We are trying to clean the docker containers and images after stopping the running containers using prune command.
Observation : sometimes the containers are removed and sometimes they go into dead state (they can be removed by restart of docker daemon). But want to know the reason why does the stop with prune commands, takes docker containers into dead state
we are using docker : Docker version 17.03.3-ce, build e19b718
Let me see if I get this right. You are doing:
docker stop my_containerdocker container prune
What happens when you do the docker stopcommand is that docker sends a SIGTERM to the root process (PID 1) in your container and ask is nicely to shut down. And then it waits for 10 seconds and if the container hasn’t exited by then, it sends a SIGKILL to the kernel. That is the normal way to do it.
However, I have noticed that some applications, like Nginx, don’t react gracefully on the SIGTERM (it does a fast shutdown) and looking in the documentation (http://nginx.org/en/docs/control.html) shows that it wants a SIGQUIT to do it gracefully.
But even when doing a fast shutdown, 10 seconds has not been enough in some cases. What I have noticed in these cases is that the volume used by the container gets in a read only state or sometimes just goes away. This leaves the container in a defunct state and marks it as dead.
Another case of mine was when I used a shared nfs for storing a Jenkins data and when updating to a new Jenkins started the new one before I shut down the old one. That made the old one grumpy and left it in a dead state since it lost the write privileges.
I think (qualified guess without looking in the docker code) that lost or read only volumes is one of the most common causes for a dead container.
Hey, appreciate the quick reply
But in my case we have no volumes, at least i don’t see a list when i run “docker volume ls” command .
The next time this happens:
–run a docker container ls -a
–then try and remove a container by running docker container rm -f a-container’s-id
If that fails to remove the container, display the result of both commands here.
You should also look to upgrade Docker to the latest Docker CE 18.09.1.