Docker Community Forums

Share and learn in the Docker community.

Docker files fills up disk space

I am using a docker based application which uses containers to provide microservices, Over a long run, my root filesystem is filed up.

Environment:

OS: Ubuntu 18.04.3 LTS server
Docker: Docker version 19.03.12, build 48a66213fe

Up on checking the main files which fills the disk space is /var/lib/docker/ , especially overlay2 directory.
I removed all stale docker images, did a system prune and still i am not able to freeup space.

root@ubuntu2:/home/test1# du -sch /var/lib/docker
4.3G /var/lib/docker
4.3G total
root@ubuntu2:/home/test1# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 395M 1.2M 394M 1% /run
/dev/mapper/ubuntu2–vg-root 14G 11G 2.4G 83% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/d4330d0d3bcee040588d204a8391761d87c56bd5ad6b6a8f3fac0c48fd0c51ce/merged
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/30ed1915a314fcd4cbeea7a9089f528e759bde07b1e8dd6eb30d7a647cb65266/merged
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/765167142da4fc17cf988ebfa7bf672a43266792aed252c75455e368caa680f6/merged
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/bb3fcf5eb730f23c40d8a60013c99c8844be6cdb37b8b98f67c01ac1b531b4cd/merged
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/84f3ebcb60fef50685a6f4dd4bfcf718f90f3e033c3c7ea5f7249adda07134ac/merged
shm 64M 0 64M 0% /var/lib/docker/containers/e61ce55f117e9be2528dca8053bcab1ecc6c2f5297840e0aa8dadfe3603ef548/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/4455c99beb21d04cf4daa42fdcf8494e65e419556a56ce9f99fbf6f06f054f93/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/916d2e317dec8d821c1605e331c1c8c2e48a081bee61c322a812ee3f9d7c86d1/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/53b44210516166800112fc9a3381f87efa1d7f2cb3f0d19c90165c32b8f3b587/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/b256d9795dbc597f795ea3b4984bbfd2ab76b57d667b3608ff424c3e9d7c3f91/mounts/shm
overlay 14G 11G 2.4G 83% /var/lib/docker/overlay2/a2bc956d174f1e84becea809d036a932cd96e434f886eea6fb23abf50ec31b44/merged
shm 64M 0 64M 0% /var/lib/docker/containers/dc32e0e4788b2e55d3402e4892fe4dfab7a1abd9fe57ccf0420dba8a0161f44c/mounts/shm
/dev/vdb1 20G 45M 19G 1% /mnt/backup
tmpfs 395M 0 395M 0% /run/user/1001

Please add -x to your du command. It will prevent that the virtual overlayfs files will be counted additionaly.
Thus, in your case the occupied capacity for /var/lib/docker should be even less than the 4.3G you got as a result. The whole ‘df’ output is worth nothing, as it repeatedly shows the stats of /dev/mapper/ubuntu2–vg-root for different overlay mounts and shm memory mounts.

This brings us to the next point. If your block device has 14G, from which 11G are used, but /var/lib/docker itself only occupies 4.3G… Can you share what exactly made you conclude that docker would be the reason your harddisk is filled?

root@ubuntu2:/var/lib/docker# du -sch * |grep G
4.3G overlay2
4.3G total
root@ubuntu2:/var/lib/docker# cd overlay2/
root@ubuntu2:/var/lib/docker/overlay2# du -sch * |grep M
30M 020dc519a5ced58fbe9e8e2841ad08f48eaef5d92ca5a853e19316d33abee162
28M 0c3cbc06b5564e3d88d73f053a02e5f25eeffe4a824f0dcf5d840c3e082e8bc0
28M 0ece04b23506b92e7f65f54e82ecef9f07b91043ce60d2f6d8216ecd83688a3d
74M 0fc801fe7a3461ceda4bfefe10f445e062bc4458d93de3823c9906f0d1b4d2a4
7.2M 212c489a9851069b6af4be3c6a1985f2dcaeec96917153cd052dad7d7357cd2c
7.0M 21b7cecc4db99ed1d2b7876f01dc3c6322820bbe49f1128008ca0ffd17b16916
37M 223607bc7851c83aef0a2ffd4c7aed5398c11d0df1853d305ecb455a0034baf4
45M 25b0685b563abc88d75df2ee4c0a79f16d685a7c700a38742d2f37041df9794d
73M 2f014d9ae6feaba4bc587d7b500de1e63fcc2952350341219ebcb5fd71bc916d
308M 30ed1915a314fcd4cbeea7a9089f528e759bde07b1e8dd6eb30d7a647cb65266
99M 339936f1ea899c8fb76cf12828cb8b8d1f4a8c66c0ee8b580c17472146f78a3e
2.4M 38efaa93222a33f819482080e0950292d3ab061396284554e24f23194d37e653
4.2M 3bb2e0622f6b8c66951442600af071ea0719682ad985de87fb2af98d261ea842
76M 406d8b08ebf15218cecdb8d1198490a22e7baff351e7c8acaf794641bb4089be
47M 4ace76a6fe7f7861bdeb6065b02062ba24b3531d04e6f16edbca906ee08658b6
45M 4bf199e36ed98f2b1001c0baaca4cce18d3f939008c9d020f89d4269837be4a6
1.7M 4d45dd15c08c8c3f6aade9e2ecba4755ed33edb2dc06bdd5374448e8d055d71b
45M 4dad3e62e90f2a6adff2be0944071cf12cd0dad10c1415f74218af8c4fc8a36c
2.0M 5aac1b8ce1c15e518e428a848cd87f78ef9149fbb443661a2daa0a08ef3414f5
1.3M 5ee580c1438a9251ab3faef5644c90c9cdde7ddb155ba7c3cdb518cc3aadd5d3
326M 63d9833680dcd732db699c6eb5c46bda77e53bc731412980607b396dcd09aca4
46M 64ee7ef7ec3bff40808e6bfde2a21e0134871584c02cb6f1922dcf1ab202e8ff
47M 703cf046d531fd25c97af0a0924a79e4454641838ff98fb24f62018b6b1532cc
24M 73c664da4063e55338abd9192f09acc8f0fee8b2293f3318855d672c09d271cd
105M 765167142da4fc17cf988ebfa7bf672a43266792aed252c75455e368caa680f6
63M 78b3a3378907b47e3c89235019c7fbdade7f9a1feeb578da8d257ba32a713ca9
438M 84f3ebcb60fef50685a6f4dd4bfcf718f90f3e033c3c7ea5f7249adda07134ac
5.9M 8986b8466083398936d4bf113532a5a1a5bb22b57c246fc28d487e4dec4d595a
28M 89d1c9fb6abb68c47e4a9132f6011470a6de4280ce58b2ada1ac03bf1c87785d
20M 926db153906b9a1312241e747d37bfcd0b5f0255361ebd50aa63c51527e02762
64M 93c2a8e819bbf02694968257ffaeca98b9026e2c69bd47e088b074236b2a993a
46M 99df33c4228c5683f2057acd417e54f56700eb8d4d6f947613b186463102f05d
496M a2bc956d174f1e84becea809d036a932cd96e434f886eea6fb23abf50ec31b44
74M a7e8d2b4a5513cd20335a34918503fe98e2e9dc94ea85abefc0cfe14bc1d8f7c
63M af4793ac6c6b016e654f502a44e39544e9459ada650fd25d460f3905d14a149b
69M b27f41aad77212892adb20dc017c30e1bd8eecbb9946b53ccc3ff3024842ee27
16M baea26faf8369a4b1d1641cbb40c2d2b613c6e2b0d6c53a5633ceb613d6bd292
154M bb3fcf5eb730f23c40d8a60013c99c8844be6cdb37b8b98f67c01ac1b531b4cd
2.5M be924182bda9a2f8edcb44f89934fa6496591df637cbfda94aecde2118c3cb34
45M c14fe664164df05dc5316de5438c51c82bf90e364f17cb60afe7a0d624f5b0af
312M c23c88a16f092ef9f05467cfd2423e3fa60797b92125b55e3b641b669fb2a093
2.4M c63930962bacdd9505c5e68811f209feae4fd8ed01c0a59959ba5e5a72be6f1c
23M ca5459c18344e945b44fd863b71d8d5f6417f3d658bb8787fbbc1e570e5f6c36
2.2M d0a6f0f0c142ee4e0e2ca850a7371ef3c7ae3f6b09e57a3d5d9edb04d5a65b05
711M d4330d0d3bcee040588d204a8391761d87c56bd5ad6b6a8f3fac0c48fd0c51ce
17M d898f5c8682a09641d33a63ae300aa22b5ccbd2385cc49226da5868d3d2da9fd
2.3M e343dfbb25b6376ff82eba59c09e037c0ffba90e0740f700ac1dfa8d77a837cc
74M ed2cc633dcfd09fbe9807f9e9c3167c2379d14889da78f868a64bcbde6771136
2.5M ede14c2faa9698a3863b7cc54eab411cacd7b9a0a0b137bc39edeb213e57c6c9
64M f053d0318d1c28a890edf4be7b7b9c85564ee8f4a4206aa30779dd37d77c4d3a
23M f0e8de75e255ca27ac263b9b474b788804afd8447a7cf38593b286f1cf70f4d2
root@ubuntu2:/var/lib/docker/overlay2#

I have only 6 images… why so many file in overlay2 directory

the df command o/p made me think there are duplicate file since its showing 11G occupied by overlay files. Also i can see so many directories inside overlay2 . What can be the reason.

Like I pointed out in my earlier response: your du messurement was and still is wrong…

Let me ask again: how can a 4.3G folder (which will be even less if you messure right) be responsible to fill up a 14G partition?!

May I suggest this free excellent self paced docker traing? After the training you should have at least a decent understanding of how docker works and how things work under the hood.

Good luck!

Ok. Now its showing correct values. It has come down to 7G from 11G in df output. But i did not delete any files.
My point was different.
The df o/p is misleading since its showing different overlay2 directories.

root@ubuntu2:/# df -h
Filesystem Size Used Avail Use% Mounted on
udev 1.9G 0 1.9G 0% /dev
tmpfs 395M 1.2M 394M 1% /run
/dev/mapper/ubuntu2–vg-root 14G 7.0G 6.1G 54% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/d4330d0d3bcee040588d204a8391761d87c56bd5ad6b6a8f3fac0c48fd0c51ce/merged
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/30ed1915a314fcd4cbeea7a9089f528e759bde07b1e8dd6eb30d7a647cb65266/merged
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/765167142da4fc17cf988ebfa7bf672a43266792aed252c75455e368caa680f6/merged
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/bb3fcf5eb730f23c40d8a60013c99c8844be6cdb37b8b98f67c01ac1b531b4cd/merged
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/84f3ebcb60fef50685a6f4dd4bfcf718f90f3e033c3c7ea5f7249adda07134ac/merged
shm 64M 0 64M 0% /var/lib/docker/containers/e61ce55f117e9be2528dca8053bcab1ecc6c2f5297840e0aa8dadfe3603ef548/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/4455c99beb21d04cf4daa42fdcf8494e65e419556a56ce9f99fbf6f06f054f93/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/916d2e317dec8d821c1605e331c1c8c2e48a081bee61c322a812ee3f9d7c86d1/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/53b44210516166800112fc9a3381f87efa1d7f2cb3f0d19c90165c32b8f3b587/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/b256d9795dbc597f795ea3b4984bbfd2ab76b57d667b3608ff424c3e9d7c3f91/mounts/shm
overlay 14G 7.0G 6.1G 54% /var/lib/docker/overlay2/a2bc956d174f1e84becea809d036a932cd96e434f886eea6fb23abf50ec31b44/merged
shm 64M 0 64M 0% /var/lib/docker/containers/dc32e0e4788b2e55d3402e4892fe4dfab7a1abd9fe57ccf0420dba8a0161f44c/mounts/shm
/dev/vdb1 20G 45M 19G 1% /mnt/backup
tmpfs 395M 0 395M 0% /run/user/1001
root@ubuntu2:/#
root@ubuntu2:/# man du
root@ubuntu2:/#
root@ubuntu2:/# du -xsch * | grep G
2.4G home
du: cannot access ‘proc/8079/task/8079/fd/4’: No such file or directory
du: cannot access ‘proc/8079/task/8079/fdinfo/4’: No such file or directory
du: cannot access ‘proc/8079/fd/3’: No such file or directory
du: cannot access ‘proc/8079/fdinfo/3’: No such file or directory
1.3G usr
2.7G var
7.0G total

Thank you very เกมสล็อต much my friend :DDD