Connection refused for application hosted over docker swarm

General
, ,
1

I have a docker swarm running on 2 nodes created via Oracle VirtualBox installed on CentOS 7 . I am able to deploy a stack running 6 containers equally distributed over the two machines. However, I am unable to connect to the deployed application with ports exposed.

Here’s the content of my Docker Compose File

version: "3"
services:
  web:
    image: <myusername>/friendlyhello:latest
    deploy:
      replicas: 6
      resources:
        limits:
          cpus: "0.1"
          memory: 50M
      restart_policy:
        condition: on-failure
    ports:
      - "80:80"
    networks:
      - webnet
networks:
  webnet:

Here is the output of docker-machine ls :

NAME    ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER     ERRORS
myvm1   *        virtualbox   Running   tcp://192.168.99.100:2376           v18.09.0
myvm2   -        virtualbox   Running   tcp://192.168.99.101:2376           v18.09.0

Here is the error from my curl command

curl http://192.168.99.100/
curl: (7) Failed connect to 192.168.99.100:80; Connection refused

Even though my application seems to be running fine.

docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                              PORTS
i5nw2wcir9j3        getstartedlab_web   replicated          6/6                 harmanspall/friendlyhello:latest   *:80->80/tcp

docker service ps getstartedlab_web
ID                  NAME                  IMAGE                              NODE                DESIRED STATE       CURRENT STATE            ERROR               PORTS
dn7mrfo1yvg4        getstartedlab_web.1   harmanspall/friendlyhello:latest   myvm1               Running             Running 28 minutes ago
jxkr1psbvmpc        getstartedlab_web.2   harmanspall/friendlyhello:latest   myvm2               Running             Running 28 minutes ago
jttd4t6b9gz5        getstartedlab_web.3   harmanspall/friendlyhello:latest   myvm1               Running             Running 28 minutes ago
zhs0c7ygj8cs        getstartedlab_web.4   harmanspall/friendlyhello:latest   myvm2               Running             Running 28 minutes ago
mx6gykk3qocd        getstartedlab_web.5   harmanspall/friendlyhello:latest   myvm1               Running             Running 28 minutes ago
pku7f60ij0bq        getstartedlab_web.6   harmanspall/friendlyhello:latest   myvm2               Running             Running 28 minutes ago

My Docker network list, as seen from Swarm Manager:

NETWORK ID          NAME                   DRIVER              SCOPE
5c502a957a70        bridge                 bridge              local
a3b1f749c09f        docker_gwbridge        bridge              local
80nens8mmp6i        getstartedlab_webnet   overlay             swarm
c9647a0f6c30        host                   host                local
mj60zgzhiwjf        ingress                overlay             swarm
5adba823ce78        none                   null                local

This does not seem to be an issue with connectivity to VMs since I am able to ping my Virtualbox

ping 192.168.99.100 -c 5
PING 192.168.99.100 (192.168.99.100) 56(84) bytes of data.
64 bytes from 192.168.99.100: icmp_seq=1 ttl=64 time=0.246 ms
64 bytes from 192.168.99.100: icmp_seq=2 ttl=64 time=0.215 ms
64 bytes from 192.168.99.100: icmp_seq=3 ttl=64 time=0.226 ms
64 bytes from 192.168.99.100: icmp_seq=4 ttl=64 time=0.251 ms
64 bytes from 192.168.99.100: icmp_seq=5 ttl=64 time=0.262 ms

--- 192.168.99.100 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.215/0.240/0.262/0.017 ms

It also fails when I try curl from inside the VM

docker-machine ssh myvm1 "curl http://192.168.99.100/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to 192.168.99.100 port 80: Connection refused
exit status 7

docker-machine ssh myvm1 "curl http://localhost/"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to localhost port 80: Connection refused
exit status 7

Any pointers would be appreciated.

2

We are experiencing similar inability to connect to our swarm boxes ever since the update to 18.09, previously our configuration worked with 18.06.1-ce,

we use the REST api and our ports declaration looks something like

  "EndpointSpec" => %{
        "Ports" => [
          %{
            "Protocol" => "tcp",
            "PublishedPort" => 15672,
            "TargetPort" => 15672
          },
          %{
            "Protocol" => "tcp",
            "PublishedPort" => 5672,
            "TargetPort" => 5672
          },
          %{
            "Protocol" => "tcp",
            "PublishedPort" => 4369,
            "TargetPort" => 4369
          }
        ]
      }
3

I’m having the exact same issue with version 18.06.1-ce-win73 (19507).

4

This is the problem we were running into:

downgrading to 18.06.1-ce using the instructions on the issue solved our problem for now.

5

Did anyone ever find a solution to this issue? I’m on Docker version 20.10.1, build 831ebea and experiencing the same exact issue.

6

try docker swarm join command without port number at the end. It worked for me.