Hope someone can help me with this! I’m running a vmware cluster of redhat 7 VMs. I’ve installed docker in swarm mode and initialized a new cluster and joined a couple of other nodes to it. At which point I created and overlay network and added a service on each node of the cluster using the docker service command. The problem I’ve run into is that the nodes on different containers cannot connect to one another (ping fails for the 10.0.0.0/24 address). For example service-a on node1 cannot connect to service-b on node2. If however I install service-a and service-b on the same node, the connection works fine between the 10. overlay IPs.
Things I’ve checked -
ports can be connected from host 1 to host 2 (4789 udp, 7946 tcp/udp, 2376 tcp etc.)
Firewalls of all sorts are disabled
Upon pinging from host1 to host2 from inside a docker container (ping 10.0.0.10 for example) I get the following error in the docker daemon logs:
Dec 5 00:42:01 host1 kernel: dsa: could not get device for incoming packet; using dev=br0
Dec 5 00:42:01 host2 kernel: dsa: could not get device for incoming packet; using dev=br0
Now here’s the amazing thing. I have the same exact setup on another set of VMs that were set up a few months back, and the exact steps above yield good pings and no error in /var/log/messages.
Any ideas folks?