Hi! I am new here, so please let me know if I put this topic in the wrong place or anything of that sort. I also am not a networking expert by any means, so please forgive me if I am using any terminology incorrectly or misunderstanding anything. I will try to be as thorough as possible but if additional info would be helpful let me know.
I’ve been trying to set up Docker to work with my current VPN setup but have hit a dead end (i.e. I have tried everything I can think of and find online). I have been running a Wireguard setup with a killswitch that allows local network access on the 10.0.0.0/8 subnet range and some additional rules to allow external access to a couple specific ports without issues for a while now (using wg-quick up [device]). I have gotten around to trying to set up some Docker containers (I will use the Immich container I have been trying to get working as an example for this post), but they seem incompatible with my setup as I am unable to access WebUIs for the applications within the Docker containers while my VPN setup is running. When my VPN is down, I am able to access the WebUIs just fine by using the IP of the Docker container (as determined by docker inspect [container]) with associated port.
The problem seems to be that my network is directing traffic to the default Docker subnets through my VPN to the docker containers (as is expected and defined by my VPN setup). The two available workarounds for this seem to be:
- Allow the default Docker subnet as part of the local networks allowed in my VPN setup (i.e. allow local traffic to circumvent the VPN on both the 10.0.0.0/8 and 172.17.0.0/16 subnets)
or - Change the default Docker subnet range to be within 10.0.0.0/8.
Towards 1, I have tried to add an additional allowed local subnet to my WireGuard configuration as per this blog post. Unfortunately, this did not work for me - if I try starting the Docker container (docker compose up) prior to setting up my WireGuard configuration, the VPN device fails to set up with an error at this point in the setup:
[#] ip route add 172.19.0.0/16 via 10.0.0.1 dev qbr0
RTNETLINK answers: File exists
If I try to do it the other way around (put WireGuard up first, then docker compose up for my Immich container), I am simply not able to access the WebUI from my browser and get a “The connection has timed out” error. (I tried the Docker container IP, Docker Gateway IP, and localhost just in case some strange routing was going on that I was unaware of). I also tried changing MTU values as per this post to no avail.
As for 2, any attempt I made to change the default subnet for Docker to be part of the 10.0.0.0/8 range by editing /etc/docker/daemon.json were met with an error saying that all of the available IPs had already been subnetted upon trying to create the container with docker compose up.
I can’t seem to think of or find any other workarounds for this. Please let me know if I have misunderstood any part of this in these attempts and what I need to do differently or if there is another solution I haven’t found yet.
Thanks!!! (:
P.S. here is my WireGuard configuration file (with some keys, IPs, ports etc. omitted for privacy/security):
# Device: xxxx
# [Server 1] to [Server 2]
PrivateKey = xxxx
Address = xxxx
DNS = xxxx
PostUp = ip route add 172.19.0.0/16 via 10.0.0.1 dev qbr0
PostUp = iptables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 10.0.0.0/8 -j REJECT && ip6tables -I OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
PostUp = ip route add default via 10.0.0.1 dev qbr0 table ssh
PostUp = ip rule add fwmark 0x2 table ssh
PostUp = iptables -I OUTPUT -d 172.19.0.0/16 -j ACCEPT
PostUp = /sbin/iptables -A OUTPUT -t mangle -o wg0 -p tcp --sport xxxx -j MARK --set-mark 2
PostUp = /sbin/iptables -A OUTPUT -t mangle -o wg0 -p tcp --sport xxxx -j MARK --set-mark 2
PreDown = /sbin/iptables -D OUTPUT -t mangle -o wg0 -p tcp --sport xxxx -j MARK --set-mark 2
PreDown = /sbin/iptables -D OUTPUT -t mangle -o wg0 -p tcp --sport xxxx -j MARK --set-mark 2
PreDown = ip rule del fwmark 0x2 table ssh
PreDown = ip route del default via 10.0.0.1 dev qbr0 table ssh
PostDown = iptables -D OUTPUT -d 172.19.0.0/16 -j ACCEPT
PreDown = iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL ! -d 10.0.0.0/8 -j REJECT && ip6tables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
[Peer]
PublicKey = xxxx
AllowedIPs = xxxx
Endpoint = xxxx