New user - Set default gateway of container to other machine in LAN

I’m trying to set up a container that will use a machine in my LAN as the default gateway. This machine is an Odroid XU4 running Arch Linux currently connected to a VPN.

I have IP forwarding set and have added the iptables rules necessary to the XU4. IP forwarding is also enabled on the Fedora 26 machine (docker host) running docker 1.13.1

I have tried passing --default-gateway=192.168.1.44 in the docker.service file but this causes the service to fail to load. I have a feeling I have to create a bridge by specifying more parameters but I’m unsure how to do that. Also, I may run more than one container so I’d rather change the gateway setting per container individually.

I have also considered setting up a PPTP connection locally without encryption (to spare resources – I’m not scared of a local attack).

Does anyone know what needs to be done?

Docker is running on Fedora 26, build 27e468e/1.13.1

Thank you very much!

Are you planning to change the default gateway for container’s internet access or for some other reason?
Following parameters are available for bridge w,r,to ip and gateway.
bip
fixed-cidr
default-gateway

Did you try changing all the above 3? You can check the service logs to see what error you are getting.

Regards
Sreenivas

I only changed the --default-gateway=192.168.1.44 option for the daemon.

To give more background: My LAN is on 192.168.1.0/24. My docker host IP is 192.168.1.50/24. My regular default gateway is my router, 192.168.1.1/24, and the desired target gateway is 192.168.1.44/24.

I have tried
sudo docker network create --gateway 192.168.1.44 --subnet 192.168.1.0/24 netx
and then
sudo docker run --name vpn --cap-add=NET_ADMIN --network netx -i -t base/archlinux /bin/bash

I have also tried
sudo docker network create --aux-address "DefaultGatewayIPv4=192.168.1.44" --subnet 192.168.1.0/24 nety
and then
sudo docker run --name vpn --cap-add=NET_ADMIN --network nety -i -t base/archlinux /bin/bash

I know I’m probably missing something fundamental about bridging here. I don’t have much knowledge of the subject. I’m not sure how to create the bridge network.

Is this something that can be done or should I consider trying the PPTP option?

Hi
I am not fully clear on your usecase. What is the reason that you want to set default gateway manually in the container?
In bridge network which is the default, containers in same bridge network can talk over the bridge. When containers need to access external world, it is done using default gateway in the host and ip masquerading rule in the iptables of host will take care of it. Inside the container, the default gateway will still point to bridge ip but container will still be able to access outside world.

I want the container to use the VPN connection of the host at 192.168.1.44/24. Privacy is the reason. For everyday regular traffic, the host does not need to be connected to the VPN. I just want to protect the container if possible.

2 Likes

This is not possible with bridge driver. You can try doing this with custom bridge and iptables, but that would be quite some work. The default with bridge driver is for container to use same gateway as the host for internet access.

Has there been any update on this? What I am trying to accomplish is to run a VPN as one container and have another container route its traffic through the VPN container.

To route one container through another you can alter the default gateway using ‘ip route’.
This can be done by either adding NET_ADMIN caps to your container and executing the command on the container or if you don’t want to add caps to the container you can also execute from the host by entering the network namespace manualy using nsenter.

Note: That this will slightly break container to host communication so you might have to also set a route to your host network via the original docker gateway. Also this only works on custom bridges as the default bridge isolates containers by default.

1 Like

Hi,

I had the same need for work.
I resolved my problem creating a new bridge and use iptables to change routes.
Its hard, but works.

work on this way:
first - create a new network interface with:

~$ docker network create --attachable --opt ‘com.docker.network.bridge.name=bridge-vpn’ --opt ‘com.docker.network.bridge.enable_ip_masquerade=false’ bridge-vpn

second - configure iptables:

~$ RTABLE=10022
~$ RTNAME=bridge-vpn
~$ sudo iptables -t mangle -A PREROUTING -i ${RTNAME} -j MARK --set-xmark 0x${RTABLE}/0xffffffff
~$ sudo iptables -t mangle -A PREROUTING -i ${RTNAME} -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
~$ sudo ip rule add from all fwmark 0x${RTABLE} lookup ${RTABLE}
~$ sudo sysctl net.ipv4.conf.all.rp_filter=2

third, add default gateway to bridge:

~$ sudo ip route add $(ip route show dev bridge-tj) dev ${RTNAME} table ${RTABLE}
~$ IPVPN=$(ip address show tun0 | grep "inet " | sed -e ‘s|.inet ||’ -e 's|/.||’)
~$ sudo ip route add default via ${IPVPN} dev tun0 table ${RTABLE}

IPVPN is your vpn ip obtained from tun0 connection, change this for your purpose.

and finally, you can test with a docker busybox, like this:

~$ docker run --network=bridge-tj --rm -itd --name=container-busybox busybox
~$ docker start container-busybox
~$ docker attach container-busybox

on busybox try this:

~# traceroute 8.8.8.8

for now sysctl on rp_filter it’s necessary because I couldn’t make it work without it, when the packet go back it is dropped out.

have fun…

… this editor is painful, the symbol $ sometimes appear sometimes not, I givup to editing.

1 Like

Try this:

Create the network

$ docker network create [NETWORK_NAME]

Start both containers, the one with the VPN and the one that will use the VPN gateway

$ docker container run -d --network [NETWORK_NAME] --name [VPN_CONTAINER_NAME] [VPN_IMAGE_NAME]
$ docker container run -d --network [NETWORK_NAME] --name [CONTAINER_NAME] [IMAGE_NAME]

Get the [NEW_GATEWAY_IP] by running this

$ docker container exec [VPN_CONTAINER_NAME] /bin/bash -c “ip -4 -o addr show eth0 | awk ‘{print $4}’ | cut -d ‘/’ -f 1”

Set the correct gateway on the container that will use the VPN:

$ docker container exec --privileged -u root [CONTAINER_NAME] /bin/bash -c “ip r del default; ip r add default via [NEW_GATEWAY_IP]”

  • Also change the /etc/resolv.conf file to use other nameserver