Multiple aws ENIs

General
1

Expected behavior

Two interfaces (eth0 and eth1) with two IP’s in different networks (the example IP’s we have used, 192.168.1.200 and 192.168.2.197) in a machine with any version of Ubuntu (16.04 or 18.04). Two containers, one (container1) with -p 192.168.1.200:8080:80 and another (container2) with -p 192.168.2.197:8080:80. The expected behavior is that it could successfully load http://publi_ip_container1 and http://publi_ip_container2.

Actual behavior

Once the traffic enters the machine, it is sent to the container through the docker0 bridge, at which point, when performing NAT for the docker network, it loses its initial origin. When the response is generated inside the container, it is sent out through the docker0 interface, but since it has lost its origin in the incoming NAT, the traffic will always be sent through the eth0. That is, the traffic coming from the container (response traffic) is always sent via eth0, despite this being received by eth1 (asymmetric routing).

Additional Information

Steps to reproduce the behavior

  1. We start from an AWS instance with docker-ce installed and with an elastic ip connected to the main private ip (eip1)
  2. We connect a second ENI to that instance with an elastic ip (eip2)
  3. We run a container with a listening port, e.g. port 80 (docker run -tid -p ip1:8080:80 image)
  4. From a browser we can load http://eip1:8080
  5. Now we start a second container (docker run -tid -p ip2:8080:80 image)
  6. In a browser load http://eip2:8080 and return a timeout

Some idea?

1 Like
2

We are having a similar issue with 2 ENIs with IPs both sharing the same subnet. I will post back if we make some progress on this but it seems that you described it well.

3

Hi guys, I’m also having this issue, did you ever resolve this?

This is defiantly a docker problem as it works if I bind something directly on the host.