Services that need to know source IP; host networking only?

As a bit of background, I’ve been a systems admin for over 20 years, including network administration, so I understand the concepts and such but now that I’m getting into docker I’ve not seen an explanation that makes sense to me. So I’ve done a bit of experimenting and am trying to suss things out… while my experimenting is using Docker Desktop for Mac, the containers will eventually be deployed on a docker engine running either on the Linux box, or quite possibly a Synology. Note that the services described here are examples, and may not at all be what I end up running, but the networking questions remain so I can understand limitations and usability.

In creating some containers to run services I’ve been running on a home Linux box, a few of them need to connect to one container in particular as well as the outside world (IRC server and various support services that talk to it). My first round of experiments using a bridged user-defined network for everything worked fine, and I can expose only the outside ports on the container for incoming connections. Perfect! Using a client to connect in, though, I find the IP address of that “client” is not the IP of the host on my real network (192.168.7.0/24), but a different and specific IP (192.168.65.1) for every connection. I think I’ve figured out that Docker Desktop for Mac spins up a Linux VM, and the IP address I’m seeing as the incoming connection is from that VM. On my Synology, one of the services I run there also sees all incoming connections as coming from the same IP address that has nothing to do with the actual source IP. Is this a reverse NAT kind of thing, in which case I’ll never be able to get the real source IP in the container if I use this setting?

Seeing some other comments on things, I thought that using network_mode: host might be my answer, and tried that. After reconfiguring the daemons to talk to each other again, I found basically the same thing - while I can connect to the server, my source IP address is still 192.168.65.1 instead of the actual source I’m coming from. As you may imagine, for something like IRC, this is… not very workable.

As a test, I did create a simple alpine container on the Synology and loaded tcpdump on it listening to a single exposed port with the default bridged network, and saw similar behavior (it was an IP in the 172.16.0.0/12 range this time, forget what exactly). Next I re-ran the container with host-based networking, and this time the incoming traffic was properly sourced from the actual host on the network that was sending it. This makes me believe that the above situation using host networking on the Mac is not a problem with host networking, but specific to how it works on the Mac, so while it does mean I can’t shift containers there for temporary hosting in a situation it also means that any Linux host where I send the containers will work as I expect.

Is my logic sound here? Is there some better in-depth description of how all the networks are handled that I haven’t found and covers all these scenarios?

Your observations are correct.

Linux containers require Linux kernels. Docker Desktop always runs the Docker Engine in a Linux utility vm (even on Linux!).

The host network mode configures the absence of network namespace isolation from the host’s network namespace. Network-wise, containers (~=isolated processes) attached to the host network run like every native process on the host. Though, in Docker Desktop the utiliy vm is the host.

In bridge networks, you indeed have a nat situation. The ip you see in the ip of the container networks gateway.

I am not sure if retaining the source ip of connections actually works in any Docker Desktop version other than Docker Desktop for Windows.

With Docker-CE (what your Synology uses under the hood as well), you can retain the source ip by using the host network, or by using a macvlan, and/or ipvlan container network. All these allow to receive multicast/broadcast messages as well. Apart from ipvlan in L3 mode, all these should not be restricted to tcp/udp.

Swarm services attached to an overlay network can additionally forward single ports in host mode, which also retains the source ip. Afaik, this feature is not available for plain containers, and I am uncertain if it works with swarm services attached to bridge networks.

@rimelek made a blog post with an in-depth explanation about Docker (bridge) networks and network namespaces:

1 Like

Perfect. Thank you very much! Now I know my understanding of it all is correct, and I can continue with my testing while knowing that deploying in the production environment will work as I expect. I appreciate your time and response!

1 Like