I am facing a strange issue with the docker image pull command where the command fails with this error;
[desai@brilp0017 ~]$ docker image pull nginx:latest
latest: Pulling from library/nginx
d121f8d1c412: Extracting [==================================================>]
27.09MB/27.09MB
ebd81fc8c071: Download complete
655316c160af: Download complete
d15953c0e0f8: Download complete
2ee525c5c3cc: Download complete
failed to register layer: Error processing tar file(exit status 1): Error cleaning up after pivot: remove /.pivot_root534731447: device or resource busy
After this error the docker daemon is no longer accessible and all docker commands return following error;
[desai@brilp0017 ~]$ docker info
Client:
Debug Mode: false
Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
The command systemctl status docker however shows it as running;
[desai@brilp0017 ~]$ systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: active (running) since Fri 2020-09-11 14:25:53 BST; 14min ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 2380 (dockerd)
Tasks: 14
Memory: 249.5M
CGroup: /system.slice/docker.service
└─2380 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
The journalctl log has the same error line as encountered in the pull command;
Sep 11 14:25:52 brilp0017 systemd[1]: Starting Docker Application Container Engine...
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.345006155+01:00" level=info msg="Starting up"
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.348597478+01:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.348667479+01:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.348733420+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.348765306+01:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.353865701+01:00" level=info msg="parsed scheme: \"unix\"" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.353908904+01:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.353944835+01:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.353988191+01:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.497701794+01:00" level=info msg="[graphdriver] using prior storage driver: overlay2"
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.816295801+01:00" level=warning msg="Your kernel does not support cgroup rt period"
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.816318357+01:00" level=warning msg="Your kernel does not support cgroup rt runtime"
Sep 11 14:25:52 brilp0017 dockerd[2380]: time="2020-09-11T14:25:52.816442165+01:00" level=info msg="Loading containers: start."
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.101411528+01:00" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.125378601+01:00" level=info msg="Loading containers: done."
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.291896277+01:00" level=warning msg="Not using native diff for overlay2, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled" storage-driver=overlay2
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.292711063+01:00" level=info msg="Docker daemon" commit=48a66213fe graphdriver(s)=overlay2 version=19.03.12-ce
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.293190069+01:00" level=info msg="Daemon has completed initialization"
Sep 11 14:25:53 brilp0017 dockerd[2380]: time="2020-09-11T14:25:53.340381428+01:00" level=info msg="API listen on /run/docker.sock"
Sep 11 14:25:53 brilp0017 systemd[1]: Started Docker Application Container Engine.
Sep 11 14:32:38 brilp0017 dockerd[2380]: time="2020-09-11T14:32:38.011501405+01:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 11 14:33:11 brilp0017 dockerd[2380]: time="2020-09-11T14:33:11.592234770+01:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 11 14:34:52 brilp0017 dockerd[2380]: time="2020-09-11T14:34:52.864254519+01:00" level=info msg="Attempting next endpoint for pull after error: failed to register layer: Error processing tar file(exit status 1): Error cleaning up after pivot: remove /.pivot_root534731447: device or resource busy"
After this the error remains the same even after stopping and starting docker service multiple times with systemctl. After restarting the laptop fully and starting the docker service it starts work as expected until the next time docker pull command is used.
I have searched for solution on the internet but majority of them point to the user not being in docker group but that is not the case for me;
[desai@brilp0017 ~]$ groups
sys network power vboxusers wireshark sambashare docker lp wheel desai
Here is the output of docker version before it crashes for version details;
[desai@brilp0017 ~]$ docker version
Client:
Version: 19.03.12-ce
API version: 1.40
Go version: go1.14.5
Git commit: 48a66213fe
Built: Sat Jul 18 01:33:21 2020
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 19.03.12-ce
API version: 1.40 (minimum version 1.12)
Go version: go1.14.5
Git commit: 48a66213fe
Built: Sat Jul 18 01:32:59 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.4.0.m
GitCommit: 09814d48d50816305a8e6c1a4ae3e2bcc4ba725a.m
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.18.0
GitCommit: fec3683
I am using Manjaro linux OS;
Operating System: Manjaro Linux
KDE Plasma Version: 5.19.4
KDE Frameworks Version: 5.73.0
Qt Version: 5.15.0
Kernel Version: 4.19.141-2-MANJARO
OS Type: 64-bit
Processors: 8 × Intel® Core™ i7-8550U CPU @ 1.80GHz
Memory: 31.2 GiB of RAM
Graphics Processor: Mesa Intel® UHD Graphics 620
Any help on this would be appreciated.