Docker Community Forums

Share and learn in the Docker community.

Adding new replica gives an error


(Vanomashey) #1

hi guys,

I am having a hard time reinitialization UCP on machines. First time UCP installed without any problem. But after some tests when I broke it, I cleaned it up with uninstall and trying to reinstall it again. I also receive some errors.

This is one of them:

ERRO[0031] Server response: {“message”:“etcdserver: peerURL exists”}
ERRO[0031] Failed to start KV store. Run “docker logs ucp-kv” for more details
FATA[0031] Failed to add member to KV store: {“message”:“etcdserver: peerURL exists”}

I assume my replica is already registered in the cluster but how can i remove it then?

Regards,
Ivan


(Vanomashey) #2

Some more details :

When running docker logs ucp-kv on the primary node I get:

2016-01-20 10:31:48.446676 W | etcdserver: failed to reach the peerURL(https://:12380) of member 7cbf3c29d1b1c66e (Get https://:12380/version: dial tcp :12380: getsockopt: connection refused)
2016-01-20 10:31:48.446719 W | etcdserver: cannot get the version of member 7cbf3c29d1b1c66e (Get https://:12380/version: dial tcp :12380: getsockopt: connection refused)


(Vivek Saraswat) #3

Hey vanomashey,

It seems that etcd did not start correctly. Did the rest of the UCP system containers start and continue running? Can you provide the full logs from the install?


(Vanomashey) #4

Hi Vivek.

You are right. But It seems like it tries to connect to primary node and cannot. You see the log from the etcd on the primary machine.

I have tried to manually check if key-value for this node is presented in etcd. But it is not there.

Regards,
Ivan


(Vivek Saraswat) #5

Try doing a --fresh-install on the nodes and seeing if that solves the problem. That had solved a similar etcd issue for me in the past.


(Vanomashey) #6

Fresh install - that what i am doing right now. I guess there are some leftovers on the servers which I have to remove. Currently I do not have an idea what is wrong.


(Yvad) #7

Hi!
I’m having the same problem with the new 0.7 release.
I first upgraded the main controller to 0.7 without any problem, then I installed the second with
docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock --name ucp docker/ucp join --replica -i -san ucpuctrl02 --fresh-install

And I get this

INFO[0000] This engine will join UCP and advertise itself with host address xxxxx - If this is incorrect, please specify an alternative address with the '--host-address' flag
INFO[0000] Verifying your system is compatible with UCP
INFO[0031] Starting local swarm containers
INFO[0033] Starting Orca Controller replica containers
ERRO[0033] Server response: {"message":"etcdserver: peerURL exists"}
ERRO[0033] Failed to start KV store.  Run "docker logs ucp-kv" for more details
FATA[0033] Failed to add member to KV store: {"message":"etcdserver: peerURL exists"}

(Vivek Saraswat) #8

Hi yvad,

Were you originally trying to rejoin an existing node to the swarm cluster? This might cause issues with etcd as it perceives the node as already existing within the cluster.

One workaround for this is to remove the node from the etcd cluster and re-add manually. To do this, go to the “UCP Key/Value Store Backends” page in the documentation, and scroll down to the section on “Troubleshooting with etcdctl.” There’s a series of commands there for listing out all the nodes, and then removing the problematic node from the cluster. You can then use “ucp join” to bring the node back.


(Aavillaatyp) #9

I am using:

http://ucp-beta-docs.s3-website-us-west-1.amazonaws.com/kv_store/

examples and the command for cluster-health:

sudo docker exec -it ucp-kv etcdctl \ --endpoint https://127.0.0.1:2379 \ --ca-file /etc/docker/ssl/ca.pem \ --cert-file /etc/docker/ssl/cert.pem \ --key-file /etc/docker/ssl/key.pem cluster-health

is returning:

No help topic for ' --endpoint'

Any idea?


(Aavillaatyp) #10

The commands worked if I “ash” into the container instead of just exec the commands. Not sure why but got it working now.