Changing data-root path to a network shared NTFS volume

General Discussions General
1

Hello docker community,

I am facing some problem issues with docker.

I would like to save docker images on a shared network volume when using docker pull <image> and to be able to run the images on the shared network volume from a host that is connected to the same network when using docker run <image> . The shared network volume is mounted using cifs protocol.

So to do that, according to the official doc of docker, it is recommended to use the daemon.json file. So I have created the /etc/docker/deamon.json and add to it the data-root = /path/to/shared/volume/docker/
For some reason, this does not worked for me. So I decided to modify the /usr/lib/systemd/system/docker.service file instead by adding --data-root option and -s overlay2 because it is the recommended filesystem in the docker documentation but without success.

I have made some tests to be able to debug, below is my tests and results.

  1. mount the volume directly to the host (via USB without the network) + overlay2
    SUCCESS
  2. mount the volume from the network using cifs to the host + overlay2
    FAILED
  3. mount the volume from the network using cifs to the host + vfs
    FAILED

The first test was to make sure that we can pull/run images using overla2 to an external volume.
The second test is the one we need to make it succeed, however, it failed with the error below (NOTE that in the first one it succeeded using overlay2!!! weird!!!)

dockerd[53541]: level=info msg="Starting up"
dockerd[53541]: level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf"
dockerd[53541]: level=info msg="[graphdriver] trying configured driver: overlay2"
dockerd[53541]: level=error msg="failed to mount overlay: invalid argument" storage-driver=overlay2
dockerd[53541]: failed to start daemon: error initializing graphdriver: driver not supported: overlay2
systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE

The third test was to see other filesystem than overlay2 because according to the error message above. So I am able to pull images on the shared network volume with success but I am not able to run them. It failed with the error below:

docker: Error response from daemon: symlink /proc/mounts /path/to/shared/volume/docker/vfs/dir/4b01720de058d734cc7769563f0592f970278c961c2570989b51935f546410a3-init/etc/mtab: operation not supported.
See 'docker run --help'.

and according to this link here they suggest to use the json file which does not worked for me.

Thank you in advance.
Best regards,

2

You can not use a remote cifs or nfs share for the data-root. If your key requirement is to store the data-root on a remote system, you could consider mounting an iscsi block device for the data-root.

The documentation covers which storage driver works with which backing filesystem: https://docs.docker.com/storage/storagedriver/select-storage-driver/#supported-backing-filesystems

1 Like
3

Hello @meyay ,

Thank you for your reply.

The goal is to store the images when runing docker pull <image> from any host connected to the same network and to be able to run it with docker run <image> as well.

I will have a look at iSCSI and keep you updated.

Many thanks,

4

No need. It will not help with this scenario.

You need to run a private container registry, so that nodes can build/push images, and other nodes can pull and use those images.

1 Like
5

Thank you for your answer.

However, after looking into container registry, the images are still pull/run on the Host. (because data-root is still /var/lib/docker)

But I would like to pull images and run them on the shared volume and not on the Host internal memory.

Thanks

6

I have found the solution.

So you need to mount the shard volume with mfsymlinks option and force the docker storage to VFS.

7

It might appear like a brilliant idea, but the docker data root is not meant to be used from more than a single docker engine. This is a call for trouble, as the folder includes several file based boltdb databases, which will eventually corrupt if more than one docker engine modifies settings in them at the same time.

Furthermore, VFS is the least efficient storage driver, because it will waste a lot of space, and will have terrible performance compared to overlayfs. According docs, the VFS storage driver is for testing only, and not meant to be used for production systems.

1 Like