Docker Community Forums

Share and learn in the Docker community.

Swarm networking with multiple UCP HA replicas


(Jojojojo1234) #1

Hi,
In order to enable swarm networking following tutorial ( https://docs.docker.com/swarm/networking/) says to start docker engine on all nodes of swarm cluster (masters and minions) like this:

docker engine sysconfig:

OPTIONS="–cluster-advertise “NODE_IP”:12376 --cluster-store etcd://“ETCD_IP”:12379 --cluster-store-opt kv.cacertfile=/var/lib/docker/discovery_certs/ca.pem --cluster-store-opt kv.certfile=/var/lib/docker/discovery_certs/cert.pem --cluster-store-opt kv.keyfile=/var/lib/docker/discovery_certs/key.pem"

This works fine when you run just one swarm master. However if I create another UCP master replica (docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock --name ucp dockerorca/ucp join --replica -i) this installs its own copy of etcd K/V store.

Lets assume that master fails with its own etcd. Replica becomes master. Now, i have concern how swarm minions can connect to replica’s etcd K/V
IP address of etcd K/V is static and pointing to failed master: --cluster-store etcd://“ETCD_IP”:12379

There is no such thing like cluster IP.

Any ideas?

Thanks

Thanks


(Vivek Saraswat) #2

Thanks for the find! When using multi-host networking, you have to enter in all of the etcd IP addresses (master and replicas). This ensures that if the master etcd fails UCP/swarm knows to look up a replica’s k/v.

For example: --cluster-store etcd://[etcd_IP1:port],[etcd_IP2:port],[etcd_IP3:port]

It looks like we need to update the docs to reflect this. Try the above and let me know if it works for you.


(Jojojojo1234) #3

Hi Vivek

Can confirm that above suggested solution works just perfect.