Our enterprise patching team updated our system kernels overnight and as a result our Docker services were broken. We have backed out the kernel upgrade and the services are again behaving as expected, so we can be pretty confident the kernel update was the cause.
A typical error would be
open /var/lib/docker/overlay2/abcde12345/some_file: operation not permitted
This would be similar to known issue(s) e.g.
Works under:
RHEL 7
3.10.0-1160.15.2.el7.x86_64 #1 SMP Thu Jan 21 16:15:07 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
Broken under
3.10.0-1160.21
Client: Docker Engine - Community
Version: 20.10.5
API version: 1.41
Go version: go1.13.15
Git commit: 55c4c88
Built: Tue Mar 2 20:33:55 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.5
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 363e9a8
Built: Tue Mar 2 20:32:17 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.4
GitCommit: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc:
Version: 1.0.0-rc93
GitCommit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
docker-init:
Version: 0.19.0
GitCommit: de40ad0
cat /etc/docker/daemon.json
{
“storage-driver”: “overlay2”
}