Newbie question containerd runs out of space

Image Builds
1

I’m a docker novice, and running into trouble with containerd running out of space on snapshots.
I am running docker on ubuntu server 24.04.3 VM on proxmox
Ubuntu server is on a 40G VM
I created a 100G disk on the VM (so as to have plenty of space) that resides on my NAS
I mounted the disc to /var/lib/docker - this seems to be functioning properly

I created a stack with FreeCAD and all went well

the next stack was with Blender and failed
the last line in the log was
/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/102/fs/usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0.2402.0: no space left on device

Then restarted containterd and docker, hoping to clear any full cache
tried to run the blender stack again and got the same error

not sure where to go from here. both of these are big containers, maybe that’s why there is not enough space?
should I increase the space on containerd? prune? thanks for advise

2

This page shows the two data directories. Including the one that containerd uses. That is what stores images in new Docker versions when the containerd snapshotter is enabled by default

https://docs.docker.com/engine/daemon/#configure-the-data-directory-location

You already know that it is /var/lib/containerd but the point is that the docker data root (/var/lib/docker) will not be big, so you should mount the disk to /var/lib/containerd instead

You will see /var/lib/docker/rootfs as a big folder but that is only an overlay filesystem from the containerd data dir, not physically there.

3

thanks! I will give that a try!

4

I am still getting the same error after mounting to /var/lib/containerd. To make sure it was not my mount, I ran without the mount, just a straight docker install on a 40 GB “thin” Ubuntu server. The first stack I created, with freecad went witthout problem, but blender stack failed with the same error message.
Is a 40G ubuntu server not enought to run 2 stacks? Or is there something else I’m missing

5

Please, share the current error message and the output of

df -h /var/lib/containerd

Also one importat question: Have you made sure the containerd folder was empty or at least really small bfore you mounted the disk? If not, your main disk is stil almost full and you just mounted the new disk over it. Then when you start a container, just some small files could fill up the main diskunder the docker data root.

Please make sure you use code blocks to format al the terminal outputs you share. More help here: https://forums.docker.com/t/how-to-format-your-forum-posts/127295#p-202628-dockerfiles-compose-files-logs-and-code-4

6

I decided to take a step back, and ceate a VM with 100G on Ubuntu server, before mounting another drive. This is the error message and output. Apparently the var/lib/container directory is capped at 20G, and not allowed to expand. This is running blender, wordpress, libreoffice, dockhand, and jelly fin. Doesn’t seem like a lot
I need to go back and make sure the mount was done correctly. the disk was created on proxmox/ubuntu, but it is NFS on a NAS

failed to copy: failed to send write: write /var/lib/containerd/io.containerd.content.v1.content/ingest/8d6e7232762495555cf80921ba46335cf712166ba7a222757f0ced871834d033/data: no space left on device

rrg@docker:~$ df -h /var/lib/containerd
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ubuntu–vg-ubuntu–lv 19G 17G 140M 100% /
rrg@docker:~$

7

Do I understand it correctly that you mount NFS to the containerd folder? OR you created the virtual disk on an NFS drive?