ZFS pool suddenly unmounted by Docker (maybe)?

General Discussions
1

We are utilizing ZFS on Linux and we create the pool and filesystem as follows:

zpool create -o ashift=12 -m /dbs dbs_pool /dev/nvme0n1

zfs create -o recordsize=4k -o atime=off -o mountpoint=/dbs/data -o redundant_metadata=most -o compression=lz4  -o logbias=throughput -o primarycache=metadata -o xattr=sa dbs/data

zfs create -o recordsize=128k -o atime=off -o mountpoint=/dbs/ogs -o redundant_metadata=most -o compression=off -o logbias=latency -o xattr=sa dbs/logs

We then start a Docker container to access the above data and logs. After a few minutes one or more of
the above filesystems are automatically unmounted.

if we do: zfs mount -a then get back the filesystems but by that time the processes accessing the
filesystems have already died.

Some posts indicate that we have to add a dependency to mount zfs before we start Docker. So this is what we did:

sudo systemctl edit docker.service and add the following lines to it:

   After=zfs-mount.service
   Requires=zfs-mount.service
   Wants=zfs-mount.service
   BindsTo=zfs-mount.service

But it did not make any difference. It seems that something in the Linux system that is causing the unmount.

Can anyone please help with this.

Kernel version: 5.10.186-179.751
OpenZFS version: 2.1.12
Docker version 20.10.23, build 7155243

Is there some way for us to check if the problem is caused by Docker.

We are not utilizing ZFS storage in Docker. We are simply providing bind mounts to Docker and it so happens that the bind mounts have ZFS filesystem on them. Docker only sees these as volumes but otherwise is not instructed in anyway about ZFS.

2

You could check the daemon logs. If docker is involved, it would high likely leave traces of it in its logs:

Furthermore, you can check the logs in /var/log/, which I assume you already did.

3

nothing found in /var/log/* of much interest
tried dmesg
tried journalctl
docker events --since ‘10m’ – simply hangs

there is nothing that suggests that zfs should be unmounted. I am wondering if there is some linux tracing that can be enabled to determine why zfs got unmounted.

4

I couldn’t easily follow your question in your first post so I edited it to add code blocks. Please, format your posts as described in the following guide: How to format your forum posts

What happens if you don’t use Docker to mount the folders? Will they be unmounted in that case after a while?