As a bit of background, I’ve been a systems admin for over 20 years, including network administration, so I understand the concepts and such but now that I’m getting into docker I’ve not seen an explanation that makes sense to me. So I’ve done a bit of experimenting and am trying to suss things out… while my experimenting is using Docker Desktop for Mac, the containers will eventually be deployed on a docker engine running either on the Linux box, or quite possibly a Synology. Note that the services described here are examples, and may not at all be what I end up running, but the networking questions remain so I can understand limitations and usability.
In creating some containers to run services I’ve been running on a home Linux box, a few of them need to connect to one container in particular as well as the outside world (IRC server and various support services that talk to it). My first round of experiments using a bridged user-defined network for everything worked fine, and I can expose only the outside ports on the container for incoming connections. Perfect! Using a client to connect in, though, I find the IP address of that “client” is not the IP of the host on my real network (192.168.7.0/24), but a different and specific IP (192.168.65.1) for every connection. I think I’ve figured out that Docker Desktop for Mac spins up a Linux VM, and the IP address I’m seeing as the incoming connection is from that VM. On my Synology, one of the services I run there also sees all incoming connections as coming from the same IP address that has nothing to do with the actual source IP. Is this a reverse NAT kind of thing, in which case I’ll never be able to get the real source IP in the container if I use this setting?
Seeing some other comments on things, I thought that using network_mode: host might be my answer, and tried that. After reconfiguring the daemons to talk to each other again, I found basically the same thing - while I can connect to the server, my source IP address is still 192.168.65.1 instead of the actual source I’m coming from. As you may imagine, for something like IRC, this is… not very workable.
As a test, I did create a simple alpine container on the Synology and loaded tcpdump on it listening to a single exposed port with the default bridged network, and saw similar behavior (it was an IP in the 172.16.0.0/12 range this time, forget what exactly). Next I re-ran the container with host-based networking, and this time the incoming traffic was properly sourced from the actual host on the network that was sending it. This makes me believe that the above situation using host networking on the Mac is not a problem with host networking, but specific to how it works on the Mac, so while it does mean I can’t shift containers there for temporary hosting in a situation it also means that any Linux host where I send the containers will work as I expect.
Is my logic sound here? Is there some better in-depth description of how all the networks are handled that I haven’t found and covers all these scenarios?