When using the ''docker run" command to launch containers, the default behavior is to create a separate L2 network (subnet) on the docker0 bridge. dockerd will update the NAT table so that traffic from a container on this L2 network to the outside will get NATed.
In my experimental setup, I want the containers to connect to the same physical L2 network the host is connected to so they can DHCP an IP address. I know this can be done with the macvlan driver, but I also want to do this while having the containers connected to a linux bridge. The bridge itself will be created and configured by me using the brctl. I’m able to achieve this setup with LXD, it’s not clear to me how to do it with docker.
Answering my own question if anyone in the future wants to do the same.
I initially tried by changing the config of the Docker bridge driver. For example, shutting off masquerading, and setting the bridge name on the host machine. That didn’t seem to work for me. Maybe there is a way to get this to work, but if you do, you’re relying on under the hood Docker stuff to manage this.
The better way is to manually create a bridge on the host, manually create veth pairs, and hook everything up manually, then update the routing table in the container to use the veth pair instead of the eth0 interface that gets connected to the docker0 bridge (or user-defined bridge).
The second example called “With Linux Bridge devices” is the one that allows you to bypass NAT.
Sidenote, of course, another option for bypassing NAT is to use Swarm. But just an FYI, that uses VXLAN which introduces a lot of overhead on the network stack.
It seems to me that what you want is for your container to reuse the network interfaces of the host directly? In such case, do a docker run --net=host.
I don’t think that would have worked in my case. What I wanted was for each container I create to have it’s own MAC and IP address. I did not want the container to use the same IP address as the host (pretty sure this is what host networking does since your container ports map directly to your host ports).
Effectively, what I was trying to do is make it appear as though each container was just another host on the network. The container couldn’t be behind NAT (default behavior of Docker), nor could it use the host’s IP (host networking).