No Such Image found error even though there is image available in private docker registry

Docker Engine Swarm
,
1

Problem Statement : I am using private docker registry for docker image. Image with correct name:tag is available in private docker registry and also able to pull it from the machine manually. There is no problem in this.

I am using docker swarm having one manager node and two worker node. When I am trying to deploy the app using jenkins, I am using following command alongwith other commands like passing docker login details :

sh returnStdout: true, script: "docker stack deploy --prune --with-registry-auth --resolve-image=always --compose-file=compose.yml ${swarmService} "

However, I am getting

No Such image found error image-name:tag…

. Moreover, when I manually pull image on the respective worker machine where the app is assigned by manager node, the app run correctly. I did not find any issue related to docker login and docker image availability in the private docker registry. I don’t see any other reasons why I am getting such error though using

–with-registry-auth

as suggested on other blog on this error. Please suggest with your valuable inputs if encounters similar error in similar circumstances.

2

Hi kumarshorav,

Did you manage to solve your problem? I am currently looking at the same problem, also with using --with-registry-auth

3

I’m looking at something similar

[root@6f43bfddf5be /]# docker images | grep node
node                              12                  1f560ce4ce7e        7 weeks ago         918MB
[root@6f43bfddf5be /]# docker rmi -f 1f560ce4ce7e
Error: No such image: 1f560ce4ce7e
[root@6f43bfddf5be /]# docker rmi 1f560ce4ce7e
Error: No such image: 1f560ce4ce7e
[root@6f43bfddf5be /]# docker images --no-trunc | grep node
node                              12                  sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5   7 weeks ago         918MB
[root@6f43bfddf5be /]# docker rmi 1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
Error: No such image: 1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
[root@6f43bfddf5be /]# docker rmi sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
+Error: No such image: sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
[root@6f43bfddf5be /]# docker rmi sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
Error: No such image: sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
[root@6f43bfddf5be /]# docker rmi  -f sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
Error: No such image: sha256:1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
[root@6f43bfddf5be /]# docker rmi  -f 1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
Error: No such image: 1f560ce4ce7ef0c10f927fccf4a85f678e976d88b176ebaffd801b6c947924f5
[root@6f43bfddf5be /]# docker image prune
WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Total reclaimed space: 0B
[root@6f43bfddf5be /]#
5

Same here on different clusters, using versions 18.09.6, 18.06.0-ce and 18.03.1-ce. Which versions are you using?

6

I’ve been struggling with this issue forever. I also use --with-registry-auth and no luck when deploying from our CI/CD pipeline, there are 3 nodes, deployment mode is global and usually 2/3 nodes are OK and 1/3 node randomly fails saying there is no image.

Honestly driving me insane, any help appreciated.

7

I agree, and need a help as well!
I (we) had been having the same trouble for years.
Such behavior happens in a node each time we add the new node to a swarm.
There is only workaround is to ‘docker pull’ image manually there, but there are tens of images, and such operations looks too weird for production environment. There is no possibility to use servers auto-scaling, at all!

8

I’m having the same problem now.

To deploy these images i need to:
1- remove the stack,
2- execute docker system prune on the worker node.
3- try again to deploy the stack.

Then the Swarm can finish the deploy with success.

But I don’t want to replay these proccess aways, we need to deal with a lots of deploys by day, we need to solve that programatically.

I think these problem can be a worker node configuration, maybe for any “limit” for disk usage, or about “cache”, without the system prune they not try again to fetch the image on registry.

Docker engine 20.10.0
Swarm mode

9

Hello.

I’m facin same problem. I’ve been researching and testing everything for more than 12 hours and I haven’t been able to solve it

Any solution? I would really appreciate it

10

If your used disk space exceeds a certain threshold, docker will start to delete what it believes are unused images. These could include images that have just been created as part of build but haven’t been pushed to a registry. There is a suggestion this is 75% (c.f. Docker Images automatically getting deleted : CAPIOT)

Try clearing some disk space on your build server and see if things improve.

Also ref Prune unused Docker objects | Docker Documentation

11

This is old, but just in case somebody stumbles across this later (as I just did), in my case I was having this issue when doing a sudo docker deploy... . If I ran it directly as root, it worked!

At least I’m up and running. Now to figure out why…

12

I encountered the same problem and I found it is because one of my swarm work nodes didn’t add the private registry’s address to /etc/docker/daemon.json.