Use remote files in docker container

General
1

I am trying to containerize an app, my app is using remote files mounted in folder.

I have been unsuccesfull with using mount.nfs or sshfs inside a debian container.

I tried to bind mount a host folder, and mounted all remote files to subfolders in the host, but the folders are still empty within the container, (viewing the local contents)

I prefer the 2nd since that would mean I can base on scratch, but any way is good?

What is the correct approach to this, is docker only meant to access local files?

2

Container bind host folders by inode. So whatever the inode was when the container was started, is what the container will see. When you mount something, like a nfs remote share, in a host folder, it will have a different inode. You might want to read the mount propagation section here: Bind mounts | Docker Docs

Though, the intended approach is to use a volume with the local driver and type nfs. The driver name might be misleading, as itā€™s perfectly capable of supporting remote shares. It just means it is locally managed on the host - even if run on a swarm cluster.

3

Thanks, network backed volumes is very usefull.

In general I was relucant about volumes due to them being too persistent and not easy to move arround, but with a network backend volumes this is no longer a problem.

Technical question, are the NFS backed volumes working in user mode? or it is being mounted on the host silantly, maybe it will solve issues of syscalls getting stuck in kernel when the remote host is acting up. how can i know that?

4

For mounting a host mounted folder, it might be possible to simply mount a local host folder that contains the network mount as folder inside. So the bind mount always stays the same.

5

Are you sure this would work, if yes how.

THis was exact my scenirio I think, and it did not work, it was empty from within the container, from mayas answer I understand that I can play arround with the propegation settings, but I didnā€™t tried yet, I am trying to see first how remote volumes is working.

Your feedback is appriciated regardless.

6

You will still have a volume folder in the Docker data root in (by default) /var/lib/docker/volumes/volumename/_data and the nfs filesystem will be mounted there as long as a container uses it. When all the container stops, the filesystem is unmounted. So you still have a bind mount basically, except that you donā€™t need to manually handle the mounting and unmounting part because Docker will do it, but you need to set the right parameters when creating the volume. I tried it so I can share the test files:

nfs server compose project in the nfs folder:

services:
  nfs-server:
    image: openebs/nfs-server-alpine:0.11.0
    volumes:
      - type: bind
        source: ./data
        target: /mnt/nfs
    environment:
      SHARED_DIRECTORY: /mnt/nfs
      SYNC: sync
      FILEPERMISSIONS_UID: 0
      FILEPERMISSIONS_GID: 0
      FILEPERMISSIONS_MODE: "0755"
    privileged: true
    ports:
      - 127.0.0.1:2049:2049/tcp
      - 127.0.0.1:2049:2049/udp

Creating the volume (you can use compose for this too, but this is better for my example)

docker volume create --driver local --opt type=nfs --opt o=addr=127.0.0.1,vers=4,rw --opt device=:/ nfs

And my test container

docker run -d --name nfs-client -v nfs:/nfs nginx
docker exec --rm nfs-client touch /nfs/file.txt
ls -la /var/lib/docker/volumes/nfs/_data/

drwxr-xr-x 2 root root 4096 Apr 23 23:34 .
drwx-----x 3 root root 4096 Apr 23 23:24 ..
-rw-r--r-- 1 root root    0 Apr 23 23:34 file.txt

Regarding inodes, you can test what @meyay meant probably. Start the nfs-client container and run

docker run --rm -it -v /var/lib/docker/volumes/nfs/_data/:/mnt bash ls -la /mnt/

You will see the files bind mounted into the bash container.
Now stop the nfs-client container, and run:

docker run --rm -it -v /var/lib/docker/volumes/nfs/_data/:/mnt bash

Go to /mnt

cd /mnt

Now open another terminal and start the nfs-client container. It will remount the nfs filesystem in the docker data root, but you are already in `/mnt/ in the container. Run

ls -lai

In the container. And it will show no files:

total 8
 285930 drwxr-xr-x    2 root     root          4096 Apr 23 21:24 .
 286026 drwxr-xr-x    1 root     root          4096 Apr 23 21:38 ..

but try passing the filepath to the ls command:

bash-5.2# ls -lai $PWD
total 8
 285910 drwxr-xr-x    2 root     root          4096 Apr 23 21:34 .
 286026 drwxr-xr-x    1 root     root          4096 Apr 23 21:38 ..
 286009 -rw-r--r--    1 root     root             0 Apr 23 21:34 file.txt

now you see the files even though you are checking the same folder basically, but without passing $PWD to the ls command, it still sees the old inodes.

So if your nfs filesystem is mounted after you already started a process in the container, it might not see the new files.

Now letā€™s say you mount a subfolder (note that I just use an NFS volume as source but it could be any kind if nfs mount without Docker. The point is what the container bind mounts, when it bind mounts it and what it can see).
Stop the nfs client container if it is running.
Run this on the host:

docker exec nfs-client mkdir /nfs/data
docker exec nfs-client touch /nfs/data/myfile.txt

Stop the nfs client (docker stop nfs-client)

Now in the othet terminal, run this:

docker run --rm -it -v /var/lib/docker/volumes/nfs/_data/data:/mnt bash

You are in the container, now letā€™s run ls -lai /mnt

total 8
 286049 drwxr-xr-x    2 root     root          4096 Apr 23 21:53 .
 286027 drwxr-xr-x    1 root     root          4096 Apr 23 21:53 ..

start the nfs client (docker start nfs-client), and run this

ls -lai /var/lib/docker/volumes/nfs/_data/data

It shows the dat folder and the file:

total 8
285986 drwxr-xr-x 2 root root 4096 Apr 23 23:51 .
285910 drwxr-xr-x 3 root root 4096 Apr 23 23:50 ..
285993 -rw-r--r-- 1 root root    0 Apr 23 23:51 myfile.txt

Now Check the files in the bash container:

ls -lai /mnt

But it is still empty:

total 8
 286049 drwxr-xr-x    2 root     root          4096 Apr 23 21:53 .
 286027 drwxr-xr-x    1 root     root          4096 Apr 23 21:53 ..

So if you for example mount a subfolder and then mount the nfs filesystem on a parent folder, the container will still see the original filesystem, not the NFS filesystem.

So it depends on how you mount the folders, but Docker Desktop could be different sometimes as it supports different kind of platforms and requires special features for mounting files from the actual host. Testing that would be a task for another time, but you can try to reproduce what I described and notice the first column in the ls outputs which is the column of the inode numbers.

1 Like