Connection refused only from CLI of node

[rgomez18]

Running a docker swarm cluster with 3 nodes where only 2 are workers.

All DNS names of the services are pointed towards the Node A. The problem was discovered because when using a registry repo one of the nodes would never fetch the images. When I checked it, I found that no matter what is done everything running from the Node A cli (like a curl to the Repo URL) gets a connection refused. We tested with other services and we found that nothing is accessible from Node A.

However what makes it even more tricky is that from outside (LAN network) everything runs fine. Regardless if containers are in Node A or Node B. We actually deployed a whoami stack globally and we can see traffic routed to both containers.

If in Node A we create a Host entry in hosts file pointing the DNS names to Node B IP everything works fine and a curl command works fine even if container is on Node A.

OS is Ubuntu 20.04 for the nodes.

[rimelek (Ákos Takács)]

@rgomez18

All DNS names of the services are pointed towards the Node A.

@rgomez18

If in Node A we create a Host entry in hosts file pointing the DNS names to Node B IP everything works

Have you also set the domain of node B and C to point to node A? If any swarm node uses those domains that can be a problem.

@rgomez18

However what makes it even more tricky is that from outside (LAN network) everything runs fine. Regardless if containers are in Node A or Node B.

What do you mean by outside LAN network? Do you try to access the swarm services from the WAN network on a port which is forwarded to a LAN IP/PORT? The the WAN DNS works well and the port forward uses IP addresses, that would explain everything.