After how far I got on the previous issue, I was hoping that I had figured out what went wrong last time. That was sadly not the case.
Whenever my Linux VM attempts to start the docker daemon, the daemon crashes. In addition to this, whenever I try manually running dockerd, I sometimes get something similar to the following:
➜ ~ sudo dockerd
INFO[2021-09-20T21:27:13.822623716-04:00] Starting up
INFO[2021-09-20T21:27:13.826501470-04:00] libcontainerd: started new containerd process pid=6675
INFO[2021-09-20T21:27:13.826575924-04:00] parsed scheme: "unix" module=grpc
INFO[2021-09-20T21:27:13.826607002-04:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2021-09-20T21:27:13.826647978-04:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2021-09-20T21:27:13.826682275-04:00] ClientConn switching balancer to "pick_first" module=grpc
WARN[0000] deprecated version : `1`, please switch to version `2`
INFO[2021-09-20T21:27:13.878370983-04:00] starting containerd revision=72cec4be58a9eb6b2910f5d10f1c01ca47d231c0 version=v1.5.5
INFO[2021-09-20T21:27:13.951279735-04:00] loading plugin "io.containerd.content.v1.content"... type=io.containerd.content.v1
INFO[2021-09-20T21:27:13.951425294-04:00] loading plugin "io.containerd.snapshotter.v1.aufs"... type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.957016125-04:00] skip loading plugin "io.containerd.snapshotter.v1.aufs"... error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/5.14.2-artix1-2\\n\"): skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.957140901-04:00] loading plugin "io.containerd.snapshotter.v1.btrfs"... type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.957472183-04:00] skip loading plugin "io.containerd.snapshotter.v1.btrfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.957517620-04:00] loading plugin "io.containerd.snapshotter.v1.devmapper"... type=io.containerd.snapshotter.v1
WARN[2021-09-20T21:27:13.957737573-04:00] failed to load plugin io.containerd.snapshotter.v1.devmapper error="devmapper not configured"
INFO[2021-09-20T21:27:13.957772899-04:00] loading plugin "io.containerd.snapshotter.v1.native"... type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.957834832-04:00] loading plugin "io.containerd.snapshotter.v1.overlayfs"... type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.959018879-04:00] loading plugin "io.containerd.snapshotter.v1.zfs"... type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.959314196-04:00] skip loading plugin "io.containerd.snapshotter.v1.zfs"... error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
INFO[2021-09-20T21:27:13.959360041-04:00] loading plugin "io.containerd.metadata.v1.bolt"... type=io.containerd.metadata.v1
WARN[2021-09-20T21:27:13.959412236-04:00] could not use snapshotter devmapper in metadata plugin error="devmapper not configured"
INFO[2021-09-20T21:27:13.959445303-04:00] metadata content store policy set policy=shared
INFO[2021-09-20T21:27:13.959636547-04:00] loading plugin "io.containerd.differ.v1.walking"... type=io.containerd.differ.v1
INFO[2021-09-20T21:27:13.959687851-04:00] loading plugin "io.containerd.gc.v1.scheduler"... type=io.containerd.gc.v1
INFO[2021-09-20T21:27:13.959774547-04:00] loading plugin "io.containerd.service.v1.introspection-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.960807734-04:00] loading plugin "io.containerd.service.v1.containers-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.960876667-04:00] loading plugin "io.containerd.service.v1.content-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.960913507-04:00] loading plugin "io.containerd.service.v1.diff-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.960952030-04:00] loading plugin "io.containerd.service.v1.images-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.960993641-04:00] loading plugin "io.containerd.service.v1.leases-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.961032979-04:00] loading plugin "io.containerd.service.v1.namespaces-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.961071602-04:00] loading plugin "io.containerd.service.v1.snapshots-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.961109844-04:00] loading plugin "io.containerd.runtime.v1.linux"... type=io.containerd.runtime.v1
INFO[2021-09-20T21:27:13.961250330-04:00] loading plugin "io.containerd.runtime.v2.task"... type=io.containerd.runtime.v2
INFO[2021-09-20T21:27:13.961367291-04:00] loading plugin "io.containerd.monitor.v1.cgroups"... type=io.containerd.monitor.v1
INFO[2021-09-20T21:27:13.963171368-04:00] loading plugin "io.containerd.service.v1.tasks-service"... type=io.containerd.service.v1
INFO[2021-09-20T21:27:13.963234098-04:00] loading plugin "io.containerd.internal.v1.restart"... type=io.containerd.internal.v1
INFO[2021-09-20T21:27:13.963362477-04:00] loading plugin "io.containerd.grpc.v1.containers"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963405403-04:00] loading plugin "io.containerd.grpc.v1.content"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963442513-04:00] loading plugin "io.containerd.grpc.v1.diff"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963479216-04:00] loading plugin "io.containerd.grpc.v1.events"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963515630-04:00] loading plugin "io.containerd.grpc.v1.healthcheck"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963553184-04:00] loading plugin "io.containerd.grpc.v1.images"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963589901-04:00] loading plugin "io.containerd.grpc.v1.leases"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963637313-04:00] loading plugin "io.containerd.grpc.v1.namespaces"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.963678049-04:00] loading plugin "io.containerd.internal.v1.opt"... type=io.containerd.internal.v1
INFO[2021-09-20T21:27:13.964755679-04:00] loading plugin "io.containerd.grpc.v1.snapshots"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.964840106-04:00] loading plugin "io.containerd.grpc.v1.tasks"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.964881474-04:00] loading plugin "io.containerd.grpc.v1.version"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.964916571-04:00] loading plugin "io.containerd.grpc.v1.introspection"... type=io.containerd.grpc.v1
INFO[2021-09-20T21:27:13.967152428-04:00] serving... address=/var/run/docker/containerd/containerd-debug.sock
INFO[2021-09-20T21:27:13.967289085-04:00] serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
INFO[2021-09-20T21:27:13.967416868-04:00] serving... address=/var/run/docker/containerd/containerd.sock
INFO[2021-09-20T21:27:13.967484684-04:00] containerd successfully booted in 0.092466s
INFO[2021-09-20T21:27:13.983149555-04:00] parsed scheme: "unix" module=grpc
INFO[2021-09-20T21:27:13.983383209-04:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2021-09-20T21:27:13.983932835-04:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2021-09-20T21:27:13.984350562-04:00] ClientConn switching balancer to "pick_first" module=grpc
INFO[2021-09-20T21:27:13.986569580-04:00] parsed scheme: "unix" module=grpc
INFO[2021-09-20T21:27:13.986613610-04:00] scheme "unix" not registered, fallback to default scheme module=grpc
INFO[2021-09-20T21:27:13.986672506-04:00] ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock <nil> 0 <nil>}] <nil> <nil>} module=grpc
INFO[2021-09-20T21:27:13.986696691-04:00] ClientConn switching balancer to "pick_first" module=grpc
ERRO[2021-09-20T21:27:13.996614318-04:00] failed to mount overlay: no such device storage-driver=overlay2
ERRO[2021-09-20T21:27:13.996668420-04:00] [graphdriver] prior storage driver overlay2 failed: driver not supported
INFO[2021-09-20T21:27:13.998545930-04:00] stopping event stream following graceful shutdown error="context canceled" module=libcontainerd namespace=plugins.moby
INFO[2021-09-20T21:27:13.998565871-04:00] stopping healthcheck following graceful shutdown module=libcontainerd
failed to start daemon: error initializing graphdriver: driver not supported
This persists until I uninstall and reinstall Docker, Compose, and the associated OpenRC scripts. Once I do the reinstall, I can start dockerd both manually and from OpenRC, without issue.
One of my colleagues suggested that I pull up Docker logs (/var/logs/docker.log):
Another colleague’s reaction to it:
That’s a lot of storage/cgroups related warnings
If this is so, something may be wrong with the way I’m installing/configuring Docker. Still can’t tell why these components are always missing. When I do the re-install, dockerd starts up without issues and allows me to run containers as if nothing’s wrong. What could be happening, upon VM reboot, to lead to this and the need to re-install all docker-related packages?