containerd
is just a container daemon. Doesn’t care about images. nerdctl
gives you an interface to manage containerd and it has its own folder. If you want to know where it is, you can just guess and try /var/lib/containerd
since Docker uses /var/lib/docker
by default and you would be right. The other and more interesting solution is searching for the image id on the filesystem:
id=$(nerdctl image inspect nginx --format '{{ .ID }}' | cut -d: -f2)
find / -name $id
When you run nerdctl referring to the containerd socket, you just talk to containerd. Docker not just has its own folder for images, it has multiple supported storage drivers. When you change that storage driver, Docker can’t see the images stored by the old storage driver.
You could say “Wait… `docker container inspect can tell me what the storage driver is, so nerdctl should be able to get that information from the container”.
In fact containerd does not need that information and it is not part of the “isolated environment”. Docker needs the storage driver, because the way it creates the filesystem for a container (for example merging multiple layers) depends on it. If you have a filesystem, you don’t actually need Docker or containerd to create a container. You could just use unshare to isolate your process using Linux kernel namespaces. This is actually what a container is. Of course Docker does much more and using Docker much easier than unshare.
So I guess, containerd knows only about containers and nothing about volumes and images. nerdctl
does, but it only knows the images it created, because it knows where the metadata is stored by itself.
If you want to understand more, list the containerd-shim processes when you have one Docker container and one nerdctl container running.
ps -axo command | grep containerd-shim
/usr/bin/containerd-shim-runc-v2 -namespace moby -id 18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f -address /run/containerd/containerd.sock
/usr/bin/containerd-shim-runc-v2 -namespace default -id d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7 -address /run/containerd/containerd.sock
Then use the “search for the id” approach:
find / -name 18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f | grep -v cgroup
/var/lib/containerd/io.containerd.runtime.v2.task/moby/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
/var/lib/docker/image/overlay2/layerdb/mounts/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
/var/lib/docker/containers/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
/run/docker/runtime-runc/moby/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
/run/docker/containerd/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
/run/containerd/io.containerd.runtime.v2.task/moby/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
find / -name d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7 | grep -v cgroup
/var/lib/nerdctl/1935db59/containers/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7
/var/lib/containerd/io.containerd.runtime.v2.task/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7
/run/containerd/runc/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7
/run/containerd/io.containerd.runtime.v2.task/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7
note: I filtered out the cgroup folder from the result
The last line of each result is where containerd has its metadata (not nerdctl which used /var/lib/containerd
)
Notice the namespaces in the path: moby, default
Docker:
/run/containerd/io.containerd.runtime.v2.task/moby/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f
Nerdctl:
/run/containerd/io.containerd.runtime.v2.task/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7
You can look into the files with cat
or anything you like to see what it contains, but the most important part is that it only contains the merged path of the Docker containar’s filesystem. If you have jq
on your machine, you can get that path
Docker:
jq --raw-output .root.path /run/containerd/io.containerd.runtime.v2.task/moby/18c1ee585f073d7cd7a224903e195e9d087ad7dc49398d9c573ce39d1cebf24f/config.json
Path:
/var/lib/docker/overlay2/f93847579465f15f1595d55107004e3f5a4da5cd864fef3a73d9d552b3600a56/merged
Nerdctl:
jq --raw-output .root.path /run/containerd/io.containerd.runtime.v2.task/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7/config.json
Path:
rootfs
Now it shows only “rootfs”, because it is relative to the config.json
. You can list the files there:
ls /run/containerd/io.containerd.runtime.v2.task/default/d3a64d9a57e2424c0e3e183cd4f27b774a69594eddc1c490f4d58e46b2ef8ec7/rootfs/
bin boot dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var
This was actually the first time I compared docker and containerd (and nerdctl) so almost everything I wrote here was new to me too, but I hope I could explain how it works.