Help with Docker networking

Greetings!

I wonder if anyone could help me with a networking issue I’m having with my Docker environment. Here is the setup.

  1. Docker containers running on a VM, which accesses internet through another VM, which is running pfsense.

  2. I’m running Docker containers with .yaml files via docker-compose. The containers work fine otherwise, but they cannot access the internet properly when they are networked in the default way, via bridge networking.

  3. I cannot see any firewall logs for denied connections on the pfsense when trying to connect to internet from the containers.

  4. Inbound traffic into the containers work fine, so I’m guessing this is some kind of a routing issue with docker?

  5. The docker host OS can connect to the internet with no issue. Docker containers can connect to the internet with no issue if the network_mode is set to host.

I would much appreciate some tips on how this could possible be solved. I’ve read the Docker Networking documents, but couldn’t find anything that would describe my scenario there.

Makes kind of sense, doesn’t it? Container networks have their own ip range, thus are foreign from the perspective of pfsense.

At least you will need to add the ip range for Docker0, which is 172.17.0.0/16. When using custom networks, you might want to check if those require additional whitlisting in pfsense.

Thanks for the reply meyay!

I understand that the docker containers have internal addresses from different private ranges, but as I understand this, they are also natted when the traffic is outbound, so pfsense would have no idea what internal addresses they are assigned in docker? Wouldn’t the traffic seem to coming to the pfsense vm from the IP of the docker host?

Anyways, I’m logging blocked traffic on pfsense and can see no blocked traffic when I try to open outbound connections from the containers.

Could this just be an issue with Docker not routing the traffic properly on the docker host VM? The traffic should be sent out via the default gateway, but docker has it’s own subnet and gateway. I fail to properly understand how these intertwine though.

I had the same problem on my Synology NAS with enabled firewall. I had to specificy enable the iprange of the docker0 bridge to permit traffic from containers to the internet.

That is interesting. Although my Docker VM has no firewall, as it depends on the pfsense vm for that.

Pfsense has proper rules regarding routing traffic from the ip of the docker vm, as it works with containers which use the host network.

Any further suggestions are most welcome. Thanks a bunch anyways!

Ok, so I found the reason for this and now it is working. The VM’s main interface has an MTU value of 1450 and Docker, by default, has an MTU value of 1500.

This is why outgoing HTTPS connections failed and it seemed like not much was working. I found out useful, additional information with the Netshoot container.

So in the end this was fixed by adding this to docker-compose .yml files

networks:
default:
driver: bridge
driver_opts:
com.docker.network.driver.mtu: 1450