Docker Community Forums

Share and learn in the Docker community.

Docker volume data seems to be duplicated (suspiciously large overlay2 folder)

I’m running a Garry’s Mod server under docker (code for the dockerfile/start script/etc can be found here). I have a bunch of data (addons, config files, etc) that I mount to the container using the -v flag. The size of the folder that I mount is about 4.3 GB, and the project that I forked from said to expect around 10GB total when running the container, so I should be using somewhere around 14.3GB. However I noticed my disk usage was really high so I ran ncdu and saw that /var/lib/docker/overlay2 was 17GB.

Inside the overlay2 folder I see 2 large folders (all the folder names look like hashes, I’m assuming these are each tied to a container/image?). The first is 10.7GB, the second is 6.1GB. Inside the larger one I see a merged folder and a diff folder (8.4GB and 2.2GB respectively), both of which have a subfolder inside called gmodserv. If I go back up and into the other container/image folder inside overlay2, I see a 6.1GB diff folder, also containing a gmodserv folder. The gmodserv folders are pretty much the entirety of the space inside their parent folders, so there’s nothing else that’s taking up this space.

If you look inside the Dockerfile, you’ll see that the gmodserv folder is where the container stores all the content for the server, including the stuff that I mount. However, upon looking inside all these gmodserv folders in overlay2, none of them contain the data I’m mounting, instead they’re all just duplicates of the base server files (source engine, GMod textures, etc). I’m only running one instance of my container, so why is this gmodserv folder showing up 3 times with pretty much the same files in all of them? It’s eating up my disk space to the point that it seems running the server outside of Docker would be better (but I would prefer to keep my Docker setup as it makes moving between machines simpler). Anyone know how to cut down these duplicated files? (and no, running docker system prune -a does not get rid of them)

The overlay2 folder should contain all image layers and the copy-on-write layer container files are written in. The gmodserv data should be there twice, one time from the layer and one time in the merged folder, which is the ultimate view the container sees. If you see it a third time, I guess it means you modifly files in the running container, which will end up in the copy-on-write layer. This should give you a clean result for du: sudo du --max-depth=1 --human-readable --no-dereference --one-file-system /var/lib/docker/overlay2

Your mounted folder has nothing todo with overlay2, it is mountend within the filesystem of the container.

You could optimize your Dockerfile and merge the RUN instructions from “Prepare Gmod server and CSS content” and “Cleanup” into a single block, the cleanup block has no meaning as the files are already fixed in the layer the first RUN instruction created. The apt upgrade -y command inside the “Prepare Gmod server and CSS content”, will result in files existing in the base-image, will be overwritte in this layer. The files of the base image still exist, but get flagged in the layer as replaced. Usualy apt upgrade without squashing the image results in a bigger image.