I have a docker swarm stack of 17 containers in a development environment (Suse Linux SLE12 Server, Docker Version 18.06.1-ce, single-node manager). Every once in a while, usually after half a day of inactivity or so, docker swarm shuts down some of the services without restarting them. The symptom is that docker service ls
tells me that there are 0/1 REPLICAs. If I look into the service with docker service ps <servicename>
I can see that the DESIRED STATE is Shutdown.
The service log of such tasks usually just says
2019/02/18 07:27:54 SIGTERM received… shutting down server in 2m0s
My impression is, for some reason docker decides that the task should shut down and then never starts a new one. I have this restart policy, according to docker service inspect
:
"RestartPolicy": {
"Condition": "any",
"Delay": 5000000000,
"MaxAttempts": 5
},
I have tried the procedures from https://success.docker.com/article/swarm-troubleshooting-methodology, but I still cannot find the reason why docker sets these tasks to desired state shutdown.
It appears as if the service is chugging along happily, when suddenly docker comes along and sinks it for no obvious reason. Hence my question: for what reasons would docker swarm decide by itself that a service should be shut down?