Unable to find basic auth credentials when pulling image from private registry via swarm

Docker Engine Swarm
1

I’m guessing something just changed/broke in the Swarm 1.2.1 release yesterday. I can no longer pull images from from our private registry which requires a basic auth username/password. Suddenly I’m getting errors like this:

$ docker pull myreg.company.com/myorg/myrepo:mytag ip-10-1-2-208: Pulling myreg.company.com/myorg/myrepo:mytag... : Error: image cyberu/cyberui not found ip-10-1-2-81: Pulling myreg.company.com/myorg/myrepo:mytag... : Error: image cyberu/cyberui not found ip-10-1-2-209: Pulling myreg.company.com/myorg/myrepo:mytag... : Error response from daemon: Get https://myreg.company.com/v2/myorg/myrepo/manifests/mytag: no basic auth credentials ip-10-1-2-82: Pulling myreg.company.com/myorg/myrepo:mytag... : Error: image cyberu/cyberui not found ip-10-1-2-207: Pulling myreg.company.com/myorg/myrepo:mytag... : Error: image cyberu/cyberui not found ip-10-1-2-83: Pulling myreg.company.com/myorg/myrepo:mytag... : Error response from daemon: Get https://myreg.company.com/v2/myorg/myrepo/manifests/mytag: no basic auth credentials Error response from daemon: Get https://myreg.company.com/v2/myorg/myrepo/manifests/mytag: no basic auth credentials

The difference in errors from some of the nodes is because I added the --disable-legacy-registry option to the daemon on those boxes to see if that was the issue. But that clarified that the basic auth credentials are somehow not being used.

Things I’ve tried:

  • I’ve got my credentials in ~/.docker/config.json both on the local manager node and the Swarm node machines.
  • I saw an article about setting the X-Registry-Auth HTTP header via ~/.docker/config.json
  • The pull succeeds if I ssh into each node and run the pull locally

Anyone know how stored credentials are picked up, passed along, and used with Swarm? What processes/containers actually have (or attempt) access to ~/.docker/config.json?

I’m suspecting there’s a bug somewhere since it was authenticating and pulling images successfully before the latest swarm image hit.

2

Do you have any luck or help with this issue. I am also facing similar issue. just wondering if you have any work arounds to resolve this.

3

Yes. I’m not exactly sure when or where things changed. But since posting, the newest release versions of Docker Engine, Swarm (and possibly Distribution) seem to have eliminated the need for me to specify the X-Registry-Auth header in the ~/.docker/config.json file. (On a whim I took it out.) Now pulls across the swarm work with both images from my private registry server and public images from Docker Hub.

4

I am also using latest Docker version 1.12.0-rc2, build 906eacd.
I have created swarm cluster with 1 manager and 3 workers.
docker service create --replicas 3 --name somename REGISTRY_IP:PORT/IMAGE_NAME
this is how I am trying to create the containers across 3 swarm workers.
docker service ls command is showing 0/3, so no container was started properly. When I check the swarm worker logs it’s saying the image was not found. But if I run the same on swarm worker directly it’s working fine. I was able to create the container properly.
Another thing is, if I pull the image manually on all swarm workers and keep it available, then the docker service create is successfully creating the containers across all swarm workers. I am behind the firewall and proxy and not able to use public docker hub for testing.