I run a streaming data application in a docker container using docker engine on Ubuntu. I woke up this morning to find that the docker daemon had exited mysteriously. I ran sudo journalctl -n 50 -fu docker.service
and noticed the following:
Dec 01 06:15:58 ip-172-31-11-114 systemd[1]: Stopping Docker Application Container Engine…
Dec 01 06:15:58 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:15:58.650662973Z” level=info msg=“Processing signal ‘terminated’”
Dec 01 06:16:08 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:08.741278497Z” level=info msg=“Container 675b0cd325014b323d723b688f779b8fc7f3ed535b9fc74e422ea74aa49e
3653 failed to exit within 10 seconds of signal 15 - using the force”
Dec 01 06:16:08 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:08.741278833Z” level=info msg=“Container b9f3fc1968d07eeaa5a5cd9582e76e3dabad36e5227c6654d084c18fdd8f
2025 failed to exit within 10 seconds of signal 15 - using the force”
Dec 01 06:16:09 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:09.282690931Z” level=info msg=“ignoring event” module=libcontainerd namespace=moby topic=/tasks/delet
e type="*events.TaskDelete"
Dec 01 06:16:13 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:13.653494622Z” level=error msg=“Force shutdown daemon”
Dec 01 06:16:13 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:13.653568627Z” level=info msg=“Daemon shutdown complete”
Dec 01 06:16:13 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:13.653635545Z” level=info msg=“stopping event stream following graceful shutdown” error=“context canc
eled” module=libcontainerd namespace=plugins.moby
Dec 01 06:16:13 ip-172-31-11-114 dockerd[1995]: time=“2020-12-01T06:16:13.653664594Z” level=info msg=“stopping event stream following graceful shutdown” error=“context canc
eled” module=libcontainerd namespace=moby
Dec 01 06:16:13 ip-172-31-11-114 systemd[1]: Stopped Docker Application Container Engine.
This is running on an EC2 instance on AWS. Is there anyway to dig up more information about what triggered this event? To my knowledge, nobody logged in at that the time to shutdown the service.
Thanks!