I’m trying to set up a swarm cluster with a high availability swarm manager as documented here:
I have a consul cluster up and running and am able to register services with it successfully.
I have some docker swarm nodes which run a consul agent and I can see them listed as nodes in the consul ui.
The problems I’m having are with the swarm manager leadership elections. Basically, a manager is able to acquire leadership and keeps it for about a minute, then loses it. I can see in the consul KV store that the leader key (nodes/docker/swarm/leader) value has been set.
Looking at the logs from the swarm containers, I see the following output:
time="2015-09-24T15:19:40Z" level=info msg="Leader Election: Cluster leadership acquired" time="2015-09-24T15:20:39Z" level=info msg="Leader Election: Cluster leadership lost" time="2015-09-24T15:20:39Z" level=error msg="Lock already held" time="2015-09-24T15:20:49Z" level=info msg="Leader Election: Cluster leadership lost"
I can’t seem to get a swarm manager to keep leadership, even if I use a single manager!
Can anyone help?
I can provide more info if required, just let me know what you need.