Docker CE on CentOS 8 Stream won't start

I had been running Docker for about a week. I am using the Docker CE packages from the Docker repo. Docker version 20.10.13, build a224086. This morning there were some CentOS updates that required a reboot. I was running Portainer and used it to stop 2 containers before I rebooted.

After the reboot, Docker won’t start. I don’t have much to go on as to what the reason is but maybe someone may have an idea.

I have uninstalled and re-installed Docker. I have removed the directories in /var/lib. Stil won’t start.

This is the output of systemctl status docker:

● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2022-03-23 12:56:13 EDT; 1h 11min ago
     Docs: https://docs.docker.com
  Process: 207433 ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
 Main PID: 207433 (code=exited, status=1/FAILURE)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

The .sock file exists.

That’s not much help for me maybe someone has an idea. I can post whatever file you may need.

TIA

I edited your post to fix your inserted systemctl output. Please, use the codeblock button (</>) when you want to share code or any command line output so the it will be readable keeping the indentations.

I don’t see any error message that would inidicate otherwise. Have you tried running journalctl to see the logs?

journalctl -xe -u docker

Not much help their either.

[root@ts130 ~]# journalctl -xe -u docker
-- Automatic restarting of the unit docker.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Mar 23 15:12:23 xxxxxxxxxxxxxxxx.com systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit docker.service has finished shutting down.
Mar 23 15:12:23xxxxxxxxxxxxx.com systemd[1]: docker.service: Start request repeated too quickly.
Mar 23 15:12:23 xxxxxxxxxxxx.com systemd[1]: docker.service: Failed with result 'exit-code'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit docker.service has entered the 'failed' state with result 'exit-code'.
Mar 23 15:12:23 xxxxxxxxxxxxx.com systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit docker.service has failed.
--
-- The result is failed.

When I installed Docker last week, it worked the first try. I don’t know what happened.

Then try journalctl without specifying the service:

journalctl -xe

Maybe an other dependency failed like containerd. The other thing that you could try is running dockerd without systemd to see the logs in the foreground

sudo /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

I get nothing when I run journalctl -xe
containerd is running

Output from command line:

[root@ts130 ~]# /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
INFO[2022-03-23T16:49:16.917594165-04:00] Starting up
failed to load listeners: no sockets found via socket activation: make sure the service was started by systemd

Thanks

Can you scroll up with the arrow keys or is it totally empty?

Is the docker.socket systemd unit is running? Try

systemctl status docker.socket

Or try running dockerd without the extra parameters:

/usr/bin/dockerd

I don’t know how systemd can prevent running the daemon without systemd so my guess is the containerd socket.

Actually, you can try to list the logs of containerd. Let’s hope it will not be empty.

journalctl -xe -u containerd

Yes, running journalctl -xe there is nothing there.

Running systemctl status docker.socket:

[root@ts130 ~]# systemctl status docker.socket
● docker.socket - Docker Socket for the API
   Loaded: loaded (/usr/lib/systemd/system/docker.socket; disabled; vendor preset: disabled)
   Active: failed (Result: service-start-limit-hit) since Wed 2022-03-23 16:48:37 EDT; 4h 33min ago
   Listen: /var/run/docker.sock (Stream)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

Running /usr/bin/dockered:

[root@ts130 ~]# /usr/bin/dockerd
INFO[2022-03-23T21:22:49.700094864-04:00] Starting up
INFO[2022-03-23T21:22:49.702236122-04:00] parsed scheme: "unix"                         module=grpc
INFO[2022-03-23T21:22:49.702268327-04:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2022-03-23T21:22:49.702292270-04:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2022-03-23T21:22:49.702302252-04:00] ClientConn switching balancer to "pick_first"  module=grpc
INFO[2022-03-23T21:22:49.707847133-04:00] parsed scheme: "unix"                         module=grpc
INFO[2022-03-23T21:22:49.707883784-04:00] scheme "unix" not registered, fallback to default scheme  module=grpc
INFO[2022-03-23T21:22:49.707907440-04:00] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}  module=grpc
INFO[2022-03-23T21:22:49.707917675-04:00] ClientConn switching balancer to "pick_first"  module=grpc
WARN[2022-03-23T21:22:49.851175387-04:00] Your kernel does not support cgroup blkio weight
WARN[2022-03-23T21:22:49.851206213-04:00] Your kernel does not support cgroup blkio weight_device
INFO[2022-03-23T21:22:49.851334395-04:00] Loading containers: start.
INFO[2022-03-23T21:22:49.904451654-04:00] Firewalld: docker zone already exists, returning
INFO[2022-03-23T21:22:50.507845204-04:00] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address
failed to start daemon: Error initializing network controller: Error creating default "bridge" network: Failed to program NAT chain: ZONE_CONFLICT: 'docker0' already bound to a zone
type or paste code here

Looks like something going on.

It was a kernel update I got today. Maybe that’s the problem. i think CentOS keeps the last 3 to 5 kernels available in case there is a problem. Think I should try to boot to the last kernel?

Just installed an update from the Docker repo. Sad to say things still not working.

SOLVED
I removed the docker trusted zone:

[root@ts130 ~]# firewall-cmd --get-active-zones
libvirt
  interfaces: virbr0
public
  interfaces: eno1
trusted
  interfaces: docker0
[root@ts130 ~]# firewall-cmd --permanent --zone=trusted --remove-interface=docker0
success
[root@ts130 ~]# firewall-cmd --reload
success
[root@ts130 ~]# systemctl start docker
[root@ts130 ~]# systemctl status docker
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2022-03-24 11:15:18 EDT; 9s ago
     Docs: https://docs.docker.com
 Main PID: 3813957 (dockerd)
    Tasks: 10
   Memory: 28.5M
   CGroup: /system.slice/docker.service
           └─3813957 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Mar 24 11:15:16 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:16.526057214-04:00" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Mar 24 11:15:16 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:16.526274075-04:00" level=info msg="Loading containers: start."
Mar 24 11:15:16 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:16.588615820-04:00" level=info msg="Firewalld: docker zone already exists, returning"
Mar 24 11:15:17 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:17.168963576-04:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemo>
Mar 24 11:15:17 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:17.518454010-04:00" level=info msg="Firewalld: interface docker0 already part of docker zone, returning"
Mar 24 11:15:18 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:18.073762166-04:00" level=info msg="Loading containers: done."
Mar 24 11:15:18 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:18.188700853-04:00" level=info msg="Docker daemon" commit=87a90dc graphdriver(s)=overlay2 version=20.10.14
Mar 24 11:15:18 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:18.188824222-04:00" level=info msg="Daemon has completed initialization"
Mar 24 11:15:18 ts130.xxxxxxxxxxxx.com systemd[1]: Started Docker Application Container Engine.
Mar 24 11:15:18 ts130.xxxxxxxxxxxx.com dockerd[3813957]: time="2022-03-24T11:15:18.423425573-04:00" level=info msg="API listen on /var/run/docker.sock"