I’ve created an attachable overlay network between three servers so I can control the addressing of various containers. Note that this is NOT in Swarm mode.
I have a container running WireGuard which I use to attach to my network when away from home. I’m trying to get the network and the WireGuard implementation to play nice with WireGuard with no significant success.
I created the gwbridge network, the ingress network, and the overlay network.
The subnet for docker_gwbridge:
$ docker network inspect docker_gwbridge | grep Subnet | awk '{print $2}' | sed 's/[",]//g'
172.28.0.0/24
The subnet for the ingress network:
$ docker network inspect id-ingress | grep Subnet | awk '{print $2}' | sed 's/[",]//g'
172.28.1.0/24
The subnet for the overlay network:
$ docker network inspect id-overlay | grep Subnet | awk '{print $2}' | sed 's/[",]//g'
172.28.2.0/24
The IP routing for the edge server host that runs the WireGuard docker container:
$ ip route show
default via 192.168.1.1 dev eth0 src 192.168.1.91 metric 202
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.28.0.0/24 dev docker_gwbridge proto kernel scope link src 172.28.0.1
192.168.1.0/24 dev eth0 proto dhcp scope link src 192.168.1.91 metric 202
The IP routing for the WireGuard docker container:
$ docker exec -it wg-easy bash
# ip route show
default via 172.28.0.1 dev eth1
10.8.0.0/24 dev wg0 proto kernel scope link src 10.8.0.1
172.28.0.0/24 dev eth1 proto kernel scope link src 172.28.0.5
172.28.2.0/24 dev eth0 proto kernel scope link src 172.28.2.36
The WireGuard docker container IP Address (a static assigned IP address on the id-overlay network):
# hostname -i
172.28.2.36
The WireGuard address (the ‘server ip address’):
# grep Address /etc/wireguard/wg0.conf | awk '{print $3}' | cut -d/ -f1
10.8.0.1
The addresses for the WireGuard docker container:
# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
link/none
inet 10.8.0.1/24 scope global wg0
valid_lft forever preferred_lft forever
350: eth0@if351: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether 02:42:ac:1c:02:24 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.28.2.36/24 brd 172.28.2.255 scope global eth0
valid_lft forever preferred_lft forever
352: eth1@if353: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:1c:00:05 brd ff:ff:ff:ff:ff:ff link-netnsid 1
inet 172.28.0.5/24 brd 172.28.0.255 scope global eth1
valid_lft forever preferred_lft forever
The list of NAT routing rules for the WireGuard container:
# iptables -L -v -t nat
Chain PREROUTING (policy ACCEPT 118 packets, 37731 bytes)
pkts bytes target prot opt in out source destination
Chain INPUT (policy ACCEPT 107 packets, 34385 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- any eth0 10.8.0.0/24 anywhere
In summary:
- 172.28.0.0/24 - docker_gwbridge
- 172.28.0.1/24 - id-ingress
- 172.28.0.2/24 - id-overlay
- wireguard container uses 172.28.0.0/24 on eth1 with address 172.28.0.5
- wireguard container uses 172.28.2.0/24 on eth0 with address 172.28.2.36
- The current PreUp commands set up a route from 10.8.0.0/24 to eth0
I think I need to change my PostUp commands, which are currently this:
PostUp = iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE; iptables -A INPUT -p udp -m udp --dport 51820 -j ACCEPT; iptables -A FORWARD -i wg0 -j ACCEPT; iptables -A FORWARD -o wg0 -j ACCEPT;
In order to provide routing to all possible IP addresses, which are:
10.8.0.0/24
172.17.0.1/16
172.28.0.0/16
192.168.1.0/24
Or I might need to add POSTROUTING entries to MASQUERADE forwards to docker_gwbridge, id-ingress, and id-overlay.
Has anyone encountered this before with overlay networking and routing tables?