Hi,
Just started recently to work with Docker and I struggle to fix one issue. I have provided the details below. I appreciate any help or guidance on how to make this work
Had to add spaces to all URLs, otherwise the forum does not allow me to post
My setup
Servers
-
Wireguard-Server
root server (cloud VM with public IPv4 and IPv6)
Configured as Wireguard server
Also configured as web server for mydomain.tld -
Docker-Host
Wireguard client connected to the Wireguard-Server via wg-quick (outside of docker)
Connected via a home router with dynamic IP (Wireguard-Server – Home-Router – Docker-Host)
Raspberry Pi 4 running Fedora Server 37 latest and Docker CE
What works
- I can run Nextcloud on the Docker-Host using Docker
- I can point a reverse proxy from Wireguard-Server to Docker-Host and access its Nextcloud instance via a subdomain (nextcloud.mydomain.tld)
- I can download updates from ports .ubuntu .com from inside a docker container
What does not yet work
- I cannot download updates from deb .debian .org
- I cannot download Nextcloud apps from githubusercontent .com (timeout after 120 ms with 0 bytes downloaded)
I have tried around and noticed that the issue is gone when I use “–network host” instead of my normal bridge network
That’s why I believe that this is related to the iptables config and I need to add some rules to allow Debian downloads
I have tried myself, but did not get it to work
So my questions are :
- Does anyone know why the connections to only those specific servers fail? Or why the others work fine? What is the difference between deb .debian .org and ports .ubuntu .com?
- Can anyone guide me how to resolve this (without using host network)?
Detailed steps
# Create bridge network
docker network create --driver bridge alpine
# Connecting to ports .ubuntu .com WORKS
docker run --rm -it --network alpine --name alpine1 alpine /usr/bin/wget --timeout=5 httx://ports .ubuntu .com/ubuntu-ports
docker run --rm -it --dns 8.8.8.8 --network alpine --name alpine2 alpine /usr/bin/wget --timeout=5 httx://ports .ubuntu .com/ubuntu-ports
docker run --rm -it --network host --name alpine3 alpine /usr/bin/wget --timeout=5 httx://ports .ubuntu .com/ubuntu-ports
# Connecting to deb .debian .org FAILS with timeout
docker run --rm -it --network alpine --name alpine1 alpine /usr/bin/wget --timeout=5 httx://deb .debian .org/debian/dists/bullseye/InRelease
Changing to Google DNS does not make a difference
docker run --rm -it --dns 8.8.8.8 --network alpine --name alpine2 alpine /usr/bin/wget --timeout=5 httx://deb .debian .org/debian/dists/bullseye/InRelease
It only works with host network
docker run --rm -it --network host --name alpine3 alpine /usr/bin/wget --timeout=5 httx://deb .debian .org/debian/dists/bullseye/InRelease
# traceroute from docker container is successful and goes via Docker-Host and Wireguard-Server
[docker@Docker-Host ~]$ docker run --rm -it --network alpine --name alpine1 alpine /usr/bin/traceroute google .com
traceroute to google .com
1 Docker-Host.local (172.23.0.1) 0.053 ms 0.065 ms 0.055 ms
2 [WG IP of Wireguard-Server] (10.19.99.0) 9.129 ms 8.819 ms 8.736 ms
…
# curl --verbose from Docker container FAILS
docker run --rm -it --network alpine --name ubuntu1 ubuntu bash
[install curl]
root@bbdbefe7aebd:/# curl --verbose httx://deb .debian .org/debian/dists/bullseye/InRelease
- Trying 146.75.122.132:80…
- Connected to deb .debian .org (146.75.122.132) port 80 (#0)
GET /debian/dists/bullseye/InRelease httx/1.1
Host: deb .debian .org
User-Agent: curl/7.81.0
Accept: /
# also FAILS for httxs
root@0ee93537d57e:/# curl --verbose httxs://deb .debian .org/debian/dists/bullseye/InRelease
- Trying 146.75.122.132:443…
- Connected to deb .debian .org (146.75.122.132) port 443 (#0)
- ALPN, offering h2
- ALPN, offering httx/1.1
- CAfile: /etc/ssl/certs/ca-certificates.crt
- CApath: /etc/ssl/certs
- TLSv1.0 (OUT), TLS header, Certificate Status (22):
- TLSv1.3 (OUT), TLS handshake, Client hello (1):
*# Wireguard config on Docker-Host
[root@Docker-Host ~]# wg
interface: wg0
public key: WHATEVER
private key: (hidden)
listening port: 34518
fwmark: 0xca6c
peer: WHATEVER
preshared key: (hidden)
endpoint: [IP OF WIREGUARD-SERVER]:123
allowed ips: 0.0.0.0/0, ::/0
latest handshake: 22 seconds ago
transfer: 2.22 GiB received, 1.10 GiB sent
persistent keepalive: every 25 seconds
[root@Docker-Host ~]# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default _gateway 0.0.0.0 UG 100 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.23.0.0 0.0.0.0 255.255.0.0 U 0 0 0 br-65fa97586f90
192.168.178.0 0.0.0.0 255.255.255.0 U 100 0 0 eth0
# iptables config
Generated by iptables-save v1.8.8 (nf_tables) on Sun Apr 16 13:00:04 2023
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o br-65fa97586f90 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-65fa97586f90 -j DOCKER
-A FORWARD -i br-65fa97586f90 ! -o br-65fa97586f90 -j ACCEPT
-A FORWARD -i br-65fa97586f90 -o br-65fa97586f90 -j ACCEPT
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i br-65fa97586f90 ! -o br-65fa97586f90 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o br-65fa97586f90 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT
Completed on Sun Apr 16 13:00:04 2023
Generated by iptables-save v1.8.8 (nf_tables) on Sun Apr 16 13:00:04 2023
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.23.0.0/16 ! -o br-65fa97586f90 -j MASQUERADE
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i br-65fa97586f90 -j RETURN
-A DOCKER -i docker0 -j RETURN
COMMIT
Completed on Sun Apr 16 13:00:04 2023
Docker-Host version
dnf list docker*
Last metadata expiration check: 2:03:58 ago on Sun 16 Apr 2023 11:27:15 AM CEST.
Installed Packages
docker-buildx-plugin.aarch64 0.10.4-1.fc37 @docker-ce-stable
docker-ce.aarch64 3:23.0.3-1.fc37 @docker-ce-stable
docker-ce-cli.aarch64 1:23.0.3-1.fc37 @docker-ce-stable
docker-ce-rootless-extras.aarch64 23.0.3-1.fc37 @docker-ce-stable
docker-compose.noarch 1.29.2-6.fc37 @fedora
docker-compose-plugin.aarch64 2.17.2-1.fc37 @docker-ce-stable