Thunderstorm strikes, power goes out, docker breaks!
This is the journalctl log
systemd[1]: Starting Docker Application Container Engine...
dockerd[2286]: time="2019-11-21T22:42:13.936981433+02:00" level=info msg="Starting up"
dockerd[2286]: time="2019-11-21T22:42:13.939904915+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.940385254+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.940681784+02:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" modul
dockerd[2286]: time="2019-11-21T22:42:13.940962888+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.943388049+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.943774928+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.944052903+02:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" modul
dockerd[2286]: time="2019-11-21T22:42:13.944299415+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
dockerd[2286]: time="2019-11-21T22:42:13.955803681+02:00" level=info msg="[graphdriver] using prior storage driver: overlay2"
dockerd[2286]: time="2019-11-21T22:42:13.989471092+02:00" level=warning msg="Your kernel does not support swap memory limit"
dockerd[2286]: time="2019-11-21T22:42:13.989521664+02:00" level=warning msg="Your kernel does not support cgroup cfs period"
dockerd[2286]: time="2019-11-21T22:42:13.989543793+02:00" level=warning msg="Your kernel does not support cgroup cfs quotas"
dockerd[2286]: time="2019-11-21T22:42:13.989565385+02:00" level=warning msg="Your kernel does not support cgroup rt period"
dockerd[2286]: time="2019-11-21T22:42:13.989585774+02:00" level=warning msg="Your kernel does not support cgroup rt runtime"
dockerd[2286]: time="2019-11-21T22:42:13.989926617+02:00" level=info msg="Loading containers: start."
dockerd[2286]: panic: invalid page type: 17: 10
dockerd[2286]: goroutine 1 [running]:
dockerd[2286]: github.com/docker/docker/vendor/go.etcd.io/bbolt.(*Cursor).search(0x5bb9254, 0x5c66ec0, 0xa, 0x10, 0x11, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/go.etcd.io/bbolt/cursor.go:250 +0x2cc
dockerd[2286]: github.com/docker/docker/vendor/go.etcd.io/bbolt.(*Cursor).seek(0x5bb9254, 0x5c66ec0, 0xa, 0x10, 0x15, 0x8e7958, 0x928340, 0x5ab1d30, 0x8, 0x8, ...)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/go.etcd.io/bbolt/cursor.go:159 +0x70
dockerd[2286]: github.com/docker/docker/vendor/go.etcd.io/bbolt.(*Bucket).Bucket(0x5582b0c, 0x5c66ec0, 0xa, 0x10, 0x5a0c750)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/go.etcd.io/bbolt/bucket.go:105 +0xa8
dockerd[2286]: github.com/docker/docker/vendor/go.etcd.io/bbolt.(*Tx).Bucket(...)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/go.etcd.io/bbolt/tx.go:101
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libkv/store/boltdb.(*BoltDB).List.func1(0x5582b00, 0x2c21248, 0x5582b00)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libkv/store/boltdb/boltdb.go:280 +0x60
dockerd[2286]: github.com/docker/docker/vendor/go.etcd.io/bbolt.(*DB).View(0x5a0c640, 0x5bb9354, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/go.etcd.io/bbolt/db.go:725 +0x88
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libkv/store/boltdb.(*BoltDB).List(0x554eec0, 0x5d76c40, 0x1b, 0x0, 0x0, 0x0, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libkv/store/boltdb/boltdb.go:279 +0x130
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore.(*cache).kmap(0x5d7c300, 0x2c81688, 0x5b1cc00, 0x16, 0xb687d004, 0x23b8a50)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore/cache.go:43 +0x20c
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore.(*cache).list(0x5d7c300, 0x2c81688, 0x5b1cc00, 0x0, 0x0, 0x0, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore/cache.go:164 +0x50
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore.(*datastore).List(0x5ac3050, 0x5d76c00, 0x1b, 0x2c81688, 0x5b1cc00, 0x0, 0x0, 0x0, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/datastore/datastore.go:517 +0x130
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*driver).populateNetworks(0x5634900, 0x5, 0x1ff77d2)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge_store.go:50 +0x190
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*driver).initStore(0x5634900, 0x5d2c6a0, 0x0, 0x5d2d4c0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge_store.go:35 +0x16c
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.(*driver).configure(0x5634900, 0x5d2c6a0, 0x2995320, 0x5d2c4a0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go:378 +0x15c
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge.Init(0x2c568a0, 0x5d2c680, 0x5d2c6a0, 0x5d2c6a0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drivers/bridge/bridge.go:161 +0x90
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork/drvregistry.(*DrvRegistry).AddDriver(...)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/drvregistry/drvregistry.go:72
dockerd[2286]: github.com/docker/docker/vendor/github.com/docker/libnetwork.New(0x5634880, 0x9, 0x10, 0x56c62d0, 0x588f9a0, 0x5634880, 0x9)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/docker/libnetwork/controller.go:220 +0x428
dockerd[2286]: github.com/docker/docker/daemon.(*Daemon).initNetworkController(0x5584000, 0x54bcdc0, 0x588f9a0, 0x55cba70, 0x5584000, 0x5aaf578, 0x588f9a0)
dockerd[2286]: /go/src/github.com/docker/docker/daemon/daemon_unix.go:857 +0x70
dockerd[2286]: github.com/docker/docker/daemon.(*Daemon).restore(0x5584000, 0x58816a0, 0x54c4ba0)
dockerd[2286]: /go/src/github.com/docker/docker/daemon/daemon.go:459 +0x3c4
dockerd[2286]: github.com/docker/docker/daemon.NewDaemon(0x2c6b0e8, 0x58816a0, 0x54bcdc0, 0x56c62d0, 0x0, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/daemon/daemon.go:1049 +0x2458
dockerd[2286]: main.(*DaemonCli).start(0x5945c80, 0x5422c00, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/cmd/dockerd/daemon.go:199 +0x6dc
dockerd[2286]: main.runDaemon(...)
dockerd[2286]: /go/src/github.com/docker/docker/cmd/dockerd/docker_unix.go:13
dockerd[2286]: main.newDaemonCommand.func1(0x550c140, 0x5945c40, 0x0, 0x3, 0x0, 0x0)
dockerd[2286]: /go/src/github.com/docker/docker/cmd/dockerd/docker.go:34 +0x74
dockerd[2286]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).execute(0x550c140, 0x54a2108, 0x3, 0x3, 0x550c140, 0x54a2108)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:762 +0x364
dockerd[2286]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x550c140, 0x0, 0x0, 0x10)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:852 +0x24c
dockerd[2286]: github.com/docker/docker/vendor/github.com/spf13/cobra.(*Command).Execute(...)
dockerd[2286]: /go/src/github.com/docker/docker/vendor/github.com/spf13/cobra/command.go:800
dockerd[2286]: main.main()
dockerd[2286]: /go/src/github.com/docker/docker/cmd/dockerd/docker.go:97 +0x1bc
systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
systemd[1]: docker.service: Failed with result 'exit-code'.
systemd[1]: Failed to start Docker Application Container Engine.
systemd[1]: docker.service: Service RestartSec=2s expired, scheduling restart.
systemd[1]: docker.service: Scheduled restart job, restart counter is at 2.
systemd[1]: Stopped Docker Application Container Engine.
This is docker version
Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:37:22 2019
OS/Arch: linux/arm
Experimental: false
This is /lib/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target
and this /etc/docker/daemon.json
{
"log-driver": "json-file",
"log-opts": {"max-size": "5m", "max-file": "3"}
}
I’ve got a server running on a RPi 4 with Raspbian Buster latest. It was working fine until today. I found a few other occasions where a power failure caused docker to fail but nothing I’ve tried did it for me. If anyone understands why it fails and could help, I would appreciate it.