Docker Daemon unable to start

General
1

After Docker was behaving quite unstable (possibly due to running low on memory) I restarted the machine and now the Docker Daemon does not want to startup any longer. The container possibly causing the memory issues is not starting automatically.

Environment information :
Issue Type: Bug/corrupt config?
OS Version/build : Ubuntu 20.04.3 LTS
App version : Docker version 19.03.13, build 4484c46d9d

Logging output:

Systemctl

 sudo systemctl status docker
● docker.service - Docker Application Container Engine
     Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
     Active: failed (Result: core-dump) since Sat 2021-09-25 15:35:16 CEST; 1min 7s ago
TriggeredBy: ● docker.socket
       Docs: https://docs.docker.com
    Process: 1211 ExecStart=/usr/bin/dockerd -H fd:// -H=tcp://0.0.0.0:2375 --containerd=/run/containerd/containerd.sock (code=dumped, signal=BUS)
   Main PID: 1211 (code=dumped, signal=BUS)

Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: Stopped Docker Application Container Engine.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Start request repeated too quickly.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: Failed to start Docker Application Container Engine.

output of /var/log/syslog

/var/log$ cat syslog | grep docker
Sep 25 14:18:25 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:18:25 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:18:27 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 1.
Sep 25 14:18:27 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:18:27 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:18:27 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:18:29 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 2.
Sep 25 14:18:29 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:18:29 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:18:29 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:18:31 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Sep 25 14:18:31 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:18:31 Homeserver-NUC systemd[1]: docker.service: Start request repeated too quickly.
Sep 25 14:18:31 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:18:31 Homeserver-NUC systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
Sep 25 14:46:41 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:46:41 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:46:43 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 1.
Sep 25 14:46:43 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:46:43 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:46:43 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:46:46 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 2.
Sep 25 14:46:46 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:46:46 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 14:46:46 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:46:48 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Sep 25 14:46:48 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 14:46:48 Homeserver-NUC systemd[1]: docker.service: Start request repeated too quickly.
Sep 25 14:46:48 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 14:46:48 Homeserver-NUC systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
Sep 25 15:35:08 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 15:35:08 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 15:35:11 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 1.
Sep 25 15:35:11 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 15:35:11 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 15:35:11 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 2.
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Start request repeated too quickly.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
robin@Homeserver-NUC:/var/log$

when running sudo dockerd

/var/log$ dockerd
Bus error (core dumped)

journalctl

journalctl -u docker.services
-- Logs begin at Fri 2021-09-24 07:07:12 CEST, end at Sat 2021-09-25 15:46:04 CEST. --
-- No entries --

Things already tried:

  • rebooting one more time
  • checked if enough resources (16GB ram left, cpu 1%, 450GB disk space left)
  • See if new updates available, but after update, upgrade and dist-upgrade did nothing
  • Checked if pid file was there, was not there

the system is running in a production environment and I really hope not to need to remove the containers and start from scratch…

Since the logs don’t give a whole lot of info I’m not sure where else to look, does have anyone have any suggestions?

2

journalctl -xe output with a grep on docker

Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
-- The unit docker.socket has successfully entered the 'dead' state.
-- Subject: A stop job for unit docker.socket has finished
-- A stop job for unit docker.socket has finished.
-- Subject: A stop job for unit docker.socket has begun execution
-- A stop job for unit docker.socket has begun execution.
-- Subject: A start job for unit docker.socket has begun execution
-- A start job for unit docker.socket has begun execution.
-- Subject: A start job for unit docker.socket has finished successfully
-- A start job for unit docker.socket has finished successfully.
-- Subject: A start job for unit docker.service has begun execution
-- A start job for unit docker.service has begun execution.
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.service: Main process exited, code=dumped, status=7/BUS
-- An ExecStart= process belonging to unit docker.service has exited.
Sep 25 15:35:13 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
-- The unit docker.service has entered the 'failed' state with result 'core-dump'.
-- Subject: A start job for unit docker.service has failed
-- A start job for unit docker.service has finished with a failure.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Scheduled restart job, restart counter is at 3.
-- Automatic restarting of the unit docker.service has been scheduled, as the result for
-- Subject: A stop job for unit docker.service has finished
-- A stop job for unit docker.service has finished.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.socket: Succeeded.
-- The unit docker.socket has successfully entered the 'dead' state.
-- Subject: A stop job for unit docker.socket has finished
-- A stop job for unit docker.socket has finished.
-- Subject: A stop job for unit docker.socket has begun execution
-- A stop job for unit docker.socket has begun execution.
-- Subject: A start job for unit docker.socket has begun execution
-- A start job for unit docker.socket has begun execution.
-- Subject: A start job for unit docker.socket has finished successfully
-- A start job for unit docker.socket has finished successfully.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Start request repeated too quickly.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.service: Failed with result 'core-dump'.
-- The unit docker.service has entered the 'failed' state with result 'core-dump'.
-- Subject: A start job for unit docker.service has failed
-- A start job for unit docker.service has finished with a failure.
Sep 25 15:35:16 Homeserver-NUC systemd[1]: docker.socket: Failed with result 'service-start-limit-hit'.
-- The unit docker.socket has entered the 'failed' state with result 'service-start-limit-hit'.
Sep 25 15:36:23 Homeserver-NUC sudo[1401]:    robin : TTY=pts/0 ; PWD=/home/robin ; USER=root ; COMMAND=/bin/systemctl status docker