Two interfaces (eth0 and eth1) with two IP’s in different networks (the example IP’s we have used, 192.168.1.200 and 192.168.2.197) in a machine with any version of Ubuntu (16.04 or 18.04). Two containers, one (container1) with -p 192.168.1.200:8080:80 and another (container2) with -p 192.168.2.197:8080:80. The expected behavior is that it could successfully load http://publi_ip_container1 and http://publi_ip_container2.
Once the traffic enters the machine, it is sent to the container through the docker0 bridge, at which point, when performing NAT for the docker network, it loses its initial origin. When the response is generated inside the container, it is sent out through the docker0 interface, but since it has lost its origin in the incoming NAT, the traffic will always be sent through the eth0. That is, the traffic coming from the container (response traffic) is always sent via eth0, despite this being received by eth1 (asymmetric routing).
Steps to reproduce the behavior
- We start from an AWS instance with docker-ce installed and with an elastic ip connected to the main private ip (eip1)
- We connect a second ENI to that instance with an elastic ip (eip2)
- We run a container with a listening port, e.g. port 80 (docker run -tid -p ip1:8080:80 image)
- From a browser we can load http://eip1:8080
- Now we start a second container (docker run -tid -p ip2:8080:80 image)
- In a browser load http://eip2:8080 and return a timeout