Docker Community Forums

Share and learn in the Docker community.

Will docker swarm 1.12 support multiple managers?

docker
beta

(Cohenaj194) #1

When testing the new version of docker swarm, available as a built in feature of docker 1.12, I noticed that it only allowed there to be one swarm manager at a time. If that manager node went down then I would have to have every node leave the current swarm and rejoin a brand new manager in order to recover from this failure.

The older version of docker swarm supported multiple managers for failover and I was wondering if there is a way to do that with this new version? Otherwise this represents a single point of failure!

For reference I created my swarm following the documentation at: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/


(Cohenaj194) #2

It seems as if there is a function docker node promote that allows the manual promotion of swarm managers. Promoting a node changes its MANAGER STATUS to Reachable and allows me to run docker service commands from the promoted node. However if the Leader leaves swarm or the machine running the leader looses connection to the other nodes then running swarm commands from the promoted node produces the following error:

username@hostname:~$ docker service ls Error response from daemon: rpc error: code = 4 desc = context deadline exceeded


(Machzqcq) #3

I followed the steps as mentioned in docker docs. Having one leader and rest of them being worker nodes goes fine.
However, the instant we try to join a node with ‘–manager’ option and accepting it on manager node, the swarm loses its sanity. I tried ‘docker swarm update’ , ‘docker swarm leave’ on manager node, worker nodes. The whole swarm just stays there in a dangling state.

vagrant@smgr:~$ docker node ls ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS 3yqk7wiz2w7o2q9m27dp76cl4 * smgr Accepted Ready Active Leader 89bgf7seeacmd7ijt3ev3uhk8 snode2 Accepted Ready Active by4zyturqgn4uayg78ybt3tps snode1 Accepted Ready Active vagrant@smgr:~$ docker swarm update Swarm updated. vagrant@smgr:~$ docker node ls ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS 3yqk7wiz2w7o2q9m27dp76cl4 * smgr Accepted Ready Active Leader 89bgf7seeacmd7ijt3ev3uhk8 snode2 Accepted Down Active by4zyturqgn4uayg78ybt3tps snode1 Accepted Down Active vagrant@smgr:~$ docker node accept 0dq57qb235prslhqy6z6bk1l6 Node 0dq57qb235prslhqy6z6bk1l6 accepted in the swarm. vagrant@smgr:~$ docker swarm update Error response from daemon: rpc error: code = 4 desc = context deadline exceeded vagrant@smgr:~$ docker node ls ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS 0dq57qb235prslhqy6z6bk1l6 Accepted Unknown Active Reachable 3yqk7wiz2w7o2q9m27dp76cl4 * smgr Accepted Ready Active Leader 89bgf7seeacmd7ijt3ev3uhk8 snode2 Accepted Down Active by4zyturqgn4uayg78ybt3tps snode1 Accepted Down Active vagrant@smgr:~$ docker swarm update Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

Promoting the node also did not help me - The command executes fine, however the status become Unreachable

vagrant@smgr:~$ docker node promote snode1 Node snode1 promoted to a manager in the swarm. vagrant@smgr:~$ docker node ls ID NAME MEMBERSHIP STATUS AVAILABILITY MANAGER STATUS 2djvv949g8fcbypx5w26271xf snode1 Accepted Ready Active Unreachable e5hc9jq52e42it54o74jc9v6d snode2 Accepted Ready Active f2ppi8i1ddyhe7mod85dx2yvg * smgr Accepted Ready Active Leader vagrant@smgr:~$ docker swarm update Error response from daemon: rpc error: code = 4 desc = context deadline exceeded


(Dvohra) #4

Please refer


(Cohenaj194) #5

Thats the old version of swarm and does not relate to the built in version of docker 1.12


(Cohenaj194) #6

Update: I have found a solution. Apparently when creating a swarm of masters the majority of (n+1)/2 must be present and active for swarm to function. So if there are 2 managers and one goes down, swarm goes down. However if there are 3 managers and one goes down the other two keep working. The initial master can be started with:

docker swarm init --listen-addr $(hostname -i):2377 --auto-accept manager --auto-accept worker 

Then the other managers can be started with:

docker swarm join --manager --listen-addr $(hostname -i):2377 $FIRST_MANAGER_IP:2377

Can you create a swarm with two manager nodes only?
(Dejan Golja) #7

Hey is the help for docker swarm incomplete ?

When running docker swarm join --help I didn’t notice any --manager parameter …

root@swarm-master:~# docker swarm join --help

Usage:    docker swarm join [OPTIONS] HOST:PORT

Join a swarm as a node and/or manager

Options:
      --advertise-addr string   Advertised address (format: <ip|interface>[:port])
      --help                    Print usage
      --listen-addr value       Listen address (format: <ip|interface>[:port]) (default 0.0.0.0:2377)
      --token string            Token for entry into the swarm
root@swarm-master:~# docker version
Client:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 22:00:36 2016
 OS/Arch:      linux/amd64

(Xlf12) #8

Is there any update in documentation on how to setup manager for 1.12.1 … The current docs are simply out dated …

I’m investigating an error of a breaking swarm cluster.

I wonder if the problem could be, to have to many managers, so that the cluster breaks very fast.

In a simple question: Can all nodes be managers?


(Odauby) #9

You can also use the “docker node” command to promote/demote other managers in your swarm:

~$ docker node --help

Usage:	docker node COMMAND

Manage Docker Swarm nodes

Options:
      --help   Print usage

Commands:
  demote      Demote one or more nodes from manager in the swarm
  inspect     Display detailed information on one or more nodes
  ls          List nodes in the swarm
  promote     Promote one or more nodes to manager in the swarm
  rm          Remove one or more nodes from the swarm
  ps          List tasks running on a node
  update      Update a node

Worked for me.


#10

2 ways you can do it :slight_smile:

Way 1: In your master run:

docker@master2:~$ docker node promote worker2
Node worker2 promoted to a manager in the swarm.

Way2:

You can even run the below command and run the output in any node as be

> docker@machine:~$ docker swarm join-token manager
To add a manager to this swarm, run the following command:

docker swarm join --token SWMTKN-1-3ql91v9oxhgwh8mneqd4zclm954mmze2e7cmasomw3gjw9ajfr-dlpgrjaaeqmjf8xt4eyffphad 192.168.99.100:2377

Running above command in any node which you want to make as master:

docker@master2:~$ docker swarm join --token SWMTKN-1-3ql91v9oxhgwh8mneqd4zclm954mmze2e7cmasomw3gjw9ajfr-dlpgrjaaeqmjf8xt4eyffphad 192.168.99.100:2377
This node joined a swarm as a manager.