I am trying to setup containers in a multi-homed host and only publishing the contianers on specific interfaces. This has proven to be a challenge for me, I guess it’s routing issues.
I have four nics on the host, ens160, ens192, ens224 and docker0. ens160 has the default gw.
I’ve setup policy based routing to make it possible to access services on ens192 from hosts on another subnet which is also supposed to be able to reach services published on ens160.
The ips on my host:
main routing table is pretty standard with default gw on 10.101.3.254
routing table ens192 has the following rule and routes:
ip rule add from 10.101.10.250 table ens192
ip route add 10.101.1.0/24 via 10.101.10.254 table ens192
The routing works just fine for pinging the host on ens192 and reaching ssh. So far all the working is great.
But reaching containers does not work at all. I’ve set up portainer to listen to all interfaces for testing, but the goal is to only publish protainer on ens192.
Trying to reach portainer via ens160 works great, and packets are forwarded properly to the docker0 interface.
But nothing at all happens on the docker0 interface when I try to reach portainer via ens192.
I don’t see where the packets disapear. I can see the syn packets getting in on ens192, but there it stops, no syn packets on docker0.
As said trying to ping or ssh to ens192 works fine and I can see that the policy based routing for ens192 kicks in perfectly. So my guess is that docker-proxy gets confused but I can’t figure out why, even less figure out how to solve.