I am seeking some Expert advice on an approach I would like to use for networking a network fabric with a tool that we are developing.
Our Requirement is that any switch in a fabric reach a swarm service without having to go through NAT. As the NAT happens through the connected physical interface it is not a reliable way to reach it. we are shying away from deploying loadbalancers to front end these services as the services are of varying nature and we still want to leverage Swarm Service reliability from the container perspective.
I figured that it is possible to assign a physical interface into ingress-sbox namespace. And essentially find a gateway into the Docker Overlay network.
I essentially moved eth2 into ingress-sbox name space.
Assigned a subnet to eth2 inside the namespace.
And in my switches I setup a gateway to sbox network via this eth2 IP address.
With the above steps I was able to reach direclty into the Default Overlay network of docker and using swarm service reach any container on the overlay. The advantage I have is I could use the overlay network in the configuration of my switches directly. And if I have multiple Gateways (one for each docker node) in the switch fabric for the overlay network. I could land on any node and reach any container using the docker load balancing.
Is this method usable practically without flaws?
Will this work with the routing Mesh that is If packet hits the VIP of the service anywhere in the swarm cluster?
If this is not a correct method, really appreciate any pointers to the right way to connect any overlay network with an external Non NATed network.