Swarm on five physical servers - Not seeing any node on the master

Docker Engine Swarm
1

Hi there!
I am confused. I try to get swarm up and running, but it keeps failing, well at least I do not see the swarm.
I do have five (physical) servers, CentOS 7 (most updated) and Docker 1.9.1

Dell1950     IP 192.168.10.30
Dell2950     IP 192.168.10.57
FSC300RX IP 192.168.10.48
FSC300TX  IP 192.168.10.34
Little            IP 192.168.10.65
Master         IP 192.168.10.40

On every node I ran:

sudo docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock &

and jobs tells, that each docker daemon is running.
e.g.

[1]+  Running                  sudo docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock &

So far so good.
Then I create a token on the master:

docker run --rm swarm create | tee ~/`whoami`_`date +%Y%m%d_%H%M`_token.txt
3173c53388fc57453825d4fdc4c33562

Then I prepared the following statement and copied manually in the putty sessions:

node_ip="`ip addr | grep "inet 192" | tr -s ' ' | cut -d' ' -f 3 | cut -d'/' -f1`" &&  \
docker run -d swarm join --addr=$node_ip:2375 token://3173c53388fc57453825d4fdc4c33562
INFO[1385] POST /v1.21/containers/create
a34fe533bf35095c8078abfca13bb62d6ad0bbb927f08825c903f2e968b13542
INFO[1386] POST /v1.21/containers/a34fe533bf35095c8078abfca13bb62d6ad0bbb927f08825c903f2e968b13542/start

A docker ps -a shows a running container on each server. Great!

Now it comes to the Master.

docker run -d -p 2376:2375 swarm manage token://3173c53388fc57453825d4fdc4c33562
fbc1020058d1967f74a0448a0b25a66b9ead0c6789d591390727064b6640dd58

Looks good.

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
fbc1020058d1        swarm               "/swarm manage token:"   16 seconds ago      Up 14 seconds  0.0.0.0:2376->2375/tcp   boring_noyce

Looks also good.

docker -H tcp://0.0.0.0:2376 info
Containers: 0
Images: 0
Role: primary
Strategy: spread
Filters: health, port, dependency, affinity, constraint
Nodes: 0
CPUs: 0
Total Memory: 0 B
Name: fbc1020058d1

This is not exactly what I expected.
And a docker -H tcp://192.168.178.40:2376 info will not change anything.

All firewalls are down and netstat always says:
Nodes:

$ netstat -an | grep 2375 
tcp6       0      0 :::2375                 :::*                    LISTEN

Master:

$ netstat -an | grep 237
tcp6       0      0 :::2376                 :::*                    LISTEN
unix  3      [ ]         STREAM     VERBUNDEN     22237

I DO NOT GET IT!
deep sigh

Any suggestions? Any hint / tip will be heartly appreciated.

Thank you very much in advance
Thorsten

2

Master should be allowed to access 2375 port to each node.

Telnet node 2375 port from master.

If the connection refused, check firewall or iptables on each node.

3

Hi tonykuo!
I just double checked:

  • Firewalls are OFF (defently positive)
  • The daemons are running on all servers. (Master and Slaves)

But then I discovered something wired:
When I do a netstat -an I see the tcp6.
tcp6 != tcp(4).

Maybe this is because I am running on CentOS7?
I am confused.
Any ideas? Anyone?

4

To whom ever may read this!
I found out, that in my particular case, the IPv6 protocol was causing the trouble.
I turned it off and anything went smoothly fine.
Please find a howto for your appropiate OS on My favourite search engine (!Google)

This is a great day!

I hope this thread will help someone.

Cheers
Thorsten