Using lustre for docker volumes

Docker Engine General
1

Hello folks,
first time to docker forums,
i am deploying cvat application following this doc https://docs.cvat.ai/docs/administration/basics/installation/.
for container volumes i want to use the lustre storage (mounted to the server where cvat application)
try to add the below definition in /etc/docker/daemon.json file but give the error that “overlayfs: filesystem on ‘/mnt/lustre/check-overlayfs-support1943299733/upper’ not supported”

{
“storage-driver”: “overlay2”,
“data-root”: “/mnt/lustre”
}
some other logs from journalctl as well
level=error msg=“failed to mount overlay: invalid argument” storage-driver=overlay2
msg=“Failed to built-in GetDriver graph btrfs /mnt/lustre-nvme/cvat-volumes-test”
can anyone help me please

2

the host OS is CentOS Linux 7 (Core)"

3

Not all storage driver (responsible for how container filesystems are built) supports all backing filesystemw (the filesystem on which the docker data root is). You can find the compatibility info in the documentation

https://docs.docker.com/engine/storage/drivers/select-storage-driver/#supported-backing-filesystems

It looks like you have btrfs on the host so yo can’t use overlayfs (overlay2) as storage driver, only btrfs.

4

But why are you trying to set the docker data-root directory? This folder holds all docker configurations, metadata, images, containers, and named volumes with default setting.

You could a) check if a lustre volumes plugin exist and use if it does, or b) mount the lustre volumes on the host and bind those folders into container folders.

But first, you need to fix this:

5

thank you @rimelek for your response, as per the docs, i have to create to btrfs filesystem on the lustre mountpoint. which i don’t want, its been used by other team as well.
anyway to configure docker to use lustre.
there are two containers which are having issue 1) clickhouse with below error.
Configuration file ‘/etc/clickhouse-server/config.xml’ isn’t readable by user with id ‘101’
and postgress with below error
error su-exec: /usr/local/bin/docker-entrypoint.sh: Permission denied
when i deploy the CVAT using /var/lib/docker/ as default, it works and without any problem.
thanks,
ihsan

6

thank you @meyay for your response,
the reason for using docker root-directory is, local system storage is not sufficient enough to store the data for CVAT application. we have huge amount of pics and videos that’s why we want to use lustre storage for application data.
how can i check the lustre volumes plugin exists?
for option b, i will give a try and will share the results with you.
thanks,
ihsan

7

I googled it, and it doesn’t seem like there is an actively maintained docker volume plugin.

8

so what you recommend me in such scenario?
thanks,
ihsan

9

I have no idea what cvat is, but I assume you are using Lustre because you need it for high-performance computing workloads, and can’t replace it with a nfsv4 remote share. So the answer remains what I wrote earlier: