Docker Commit Failed: no space left on device

General
1
  • Issue type: Docker Commit Failed

$ docker commit -m “Comments” opensive_meninsky centos_django:1.0
Error response from daemon: Error processing tar file(exit status 1): write /var/lib/docker/devicemapper/devicemapper/data: no space left on device

  • OS Version/build: CentOS 7 3.10.0-957.el7.x86_64

  • App version : Docker CE 19.03.8

  • Steps to reproduce
    when i committed container to build new image, i took about 15mins and failed with this message.

Error response from daemon: Error processing tar file(exit status 1): write /var/lib/docker/devicemapper/devicemapper/data: no space left on device

wired thing is docker use overlay2 but looks like there is problem with devicemapper data file.
(docker devicemapper data file size increasing exponentially)
and the file is inside of another container.

here is the that i found about this issue.
How can i solve this issue?
thanks.

[the problme]
$ docker commit -m “Comments” opensive_meninsky centos_django:1.0
Error response from daemon: Error processing tar file(exit status 1): write /var/lib/docker/devicemapper/devicemapper/data: no space left on device

[this container use this directories for layered filesystem]
“GraphDriver”: {
“Data”: {
“LowerDir”: “/docker_rootdir/overlay2/87b2a1ead4679ed08aab979c9ba490a30c43dd1614b3d22663a16605e308e1e4-init/diff:/docker_rootdir/overlay2/45f15eee9e94f6fd543228817bc875bb87be7fb1ba5d3625af3229dc6f5b41c0/diff”,
“MergedDir”: “/docker_rootdir/overlay2/87b2a1ead4679ed08aab979c9ba490a30c43dd1614b3d22663a16605e308e1e4/merged”,
“UpperDir”: “/docker_rootdir/overlay2/87b2a1ead4679ed08aab979c9ba490a30c43dd1614b3d22663a16605e308e1e4/diff”,
“WorkDir”: “/docker_rootdir/overlay2/87b2a1ead4679ed08aab979c9ba490a30c43dd1614b3d22663a16605e308e1e4/work”
},
“Name”: “overlay2”

[the file increasing its size exponentially]

/docker_rootdir/overlay2/fb4bc2c79af9c618b95cfae2317ceabb532fbe63cbed710d914d90221b33e629/diff/var/lib/docker/devicemapper/devicemapper/data

[after commit failed, this layer is disapeared]
/docker_rootdir/overlay2/fb4bc2c79af9c618b95cfae2317ceabb532fbe63cbed710d914d90221b33e629

[docker infomation]
$ docker version
Client: Docker Engine - Community
Version: 19.03.8
API version: 1.40
Go version: go1.12.17
Git commit: afacb8b
Built: Wed Mar 11 01:27:04 2020
OS/Arch: linux/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.8
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: afacb8b
Built: Wed Mar 11 01:25:42 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683

2

I’ve dig in this issue for a couple of days but i couldn’t find why.

here’s what i found.

i’m not sure this causes the problem but i found the container itself has docker binary and it use devicemapper.

so i decided not to use this image and will find other clean base image.

i’m not gonna drill down more and want to close this issue.