I have a 4-node Docker Swarm, one manager and 3 workers. Some background info:
OS: CentOS 7.2.1511
Docker Version: 17.09.0-ce
Note: each of the swarm nodes are authenticated to my private Docker registry.
I’ve been using the API to execute Docker commands with my DOCKER_HOST env var set to the remote manager node. All has been working fine, except when I try to do a
docker stack deploy. Note that the result is the same if I try to execute from my Mac (OS X 10.11.6) or from another CentOS 7 box (Docker 17.12.0-ce and 17.09.0-ce, respectively).
When I try to deploy, like so:
docker --tls stack deploy --with-registry-auth --compose-file deploy.yml my-app-service
The service fails to start because of “no such image”. I see this in the logs of one of the worker nodes:
Mar 02 02:33:23 worker1.company.com dockerd: time="2018-03-02T02:33:23.078543134Z" level=error msg="pulling image failed" error="Get https://dockerhub.company.com/v2/my-app/manifests/1.0: no basic auth credentials" module="node/agent/taskmanager" node.id=xxxxx service.id=xxxxx task.id=xxxxx Mar 02 02:33:23 worker1.company.com dockerd: time="2018-03-02T02:33:23.079000935Z" level=error msg="fatal task error" error="No such image: dockerhub.company.com/my-app:1.0" module="node/agent/taskmanager" node.id=xxxxx service.id=xxxxx task.id=xxxxx
So it’s not authenticating to registry properly, even though I included the
--with-registry-auth flag. From that same worker node, I can execute
docker pull dockerhub.company.com/my-app:1.0 and the image downloads without any issues! The credentials are already stored in ~/.docker/config.json
What I’ve discovered is that if I login to the private docker registry on the machine executing the command via API (ie. my Mac), then the service starts up fine. So apparently it’s passing the basic auth credentials from the machine making the API call.
Is there any way to force the stack deploy to use the registry credentials on the remote Docker node set in DOCKER_HOST? I tried to do the deploy without
--with-registry-auth, but that results in the same “no basic auth credentials” error.
Any help or advice would be greatly appreciated!