Running out of inodes

Docker Desktop
1

Expected behavior

I shouldn’t run out of inodes with ~30 images (docker images | wc -l), 25 containers (docker ps -a | wc -l), 159 volumes (docker volume ls | wc -l). Never had this problem on virtualbox, even with more images and containers. I have seen the inode problem when using overlay, though.

Actual behavior

$ docker run --rm --privileged debian:jessie df -h
Filesystem      Size  Used Avail Use% Mounted on
none             60G   36G   21G  64% /
tmpfs           2.0G     0  2.0G   0% /dev
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/vda2        60G   36G   21G  64% /etc/hosts
shm              64M     0   64M   0% /dev/shm

$ docker run --rm --privileged debian:jessie df -i
Filesystem     Inodes IUsed IFree IUse% Mounted on
none             3.8M  3.8M    57  100% /
tmpfs            494K   139  494K    1% /dev
tmpfs            494K    14  494K    1% /sys/fs/cgroup
/dev/vda2        3.8M  3.8M    57  100% /etc/hosts
shm              494K     1  494K    1% /dev/shm

Information

D5A4F18E-8A2A-42D2-970F-6C3851ECFEBB

Steps to reproduce the behavior

I don’t know, building and running a lot of images and containers. I started seeing “out of space” failures in my containers even though “docker run centos df -h” showed 20GB free.

Workaround

Ah I guess something in the 150 volumes was using up millions of inodes? This reduced the inode count to ~11%. Ran for several minutes, deleted all but a dozen volumes. Btw it hung, had to kill the docker client process, even though “docker ps -a” showed it had exited.
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --rm martin/docker-cleanup-volumes

2

Ok, we were wondering about increasing the number of inodes by default, will look at this again.

3

I don’t know if it’s something about the way I’m using volumes, but when I get up to around 100 I have to run “martin/docker-cleanup-volumes” to free up inodes.

What’s weird is that docker volume ls still shows the same number of volumes after that cleanup, so whatever it’s doing to free up millions of inodes does not require actually deleting volumes.