Packets dropped by NAT rules

Guys,

I 've been working with dockers for almost a year, did a lot of task and when finally I thought of bringing docker to prod environment, I hit a major block. UDP and TCP traffic is not reaching my docker containers.

This issue is bit strange, on my Test environment I don’t see any packets getting drop but on the production servers where I 'm trying to run docker, it is dropping UDP as well as TCP packets. The docker version (1.8) and the host OS (OEL 7) is same in both the environments. All the packets are reaching to my host server but they don’t enter my containers, so no issue with the n/w dropping the packets.

Has it anything to do with the class of IP? My hosts on my lab setup has an ip range of 172.x.x.x while the production setup has 10.10.x.x. Has the netmask anything to do with this problem?

I 'm running out of options to check, any pointers to debug would be helpful here.

Thanks
Ashish

Networking model totally changed in Docker in the past few releases (1.8 is “old”, 1.11 is soon to drop). Is it out of the question to upgrade and see if that helps your issue?

To me, the problem smells strongly of firewall or unintended network configuration. Did anything change in that regard recently? Have you checked every single layer of firewall or network indirection you might have to ensure that the rules are good? e.g. security groups (if applicable) and iptables? Are you on VPN or proxy?

How are you verifying that packets are getting to your host instance properly independent of Docker? Running a little Python webserver or something works?

What flags is the Docker daemon run with?

Are containers reachable from localhost on the host instance? e.g. using curl

Anyway, IP range shouldn’t really matter AFAIK (but I am not a networking expert), unless you are running your containers with an explicit IP for exposing ports, e.g. docker run -d -p 127.0.0.1:8080:8080 ...

Hi @nathanleclaire, thanks for the reply.

Well upgrading the docker from 1.8 to 1.11 will take considerable efforts as many of the flags/configuration I use is deprecated. I know I will have to do it but I just don’t want to do it now. :slight_smile:

Yes, I checked almost each layer. Firewalld service is stopped and in Iptables I 've no configuration which will block the particular port. There is a VPN connection to my server but again I can see my packets reaching to the host machine.

I 'm running tcpdump inside the container and the hosts to verify the packets reaching my host machine. There is no single packet drop to my host. [quote=“nathanleclaire, post:2, topic:8023”]
Are containers reachable from localhost on the host instance? e.g. using curl
[/quote]

Flags with ports binding -p, a DNS ip which is my consul IP nothing really fancy here.

Will try out the curl thing.