Docker + Autofs : Does it work?

dartsteven · March 30, 2023, 7:43pm

Hello everyone,

since I have been searching for days for info but have not found anything I am here to ask you :

Does autofs work in Docker?

Specifically I have autofs working on my debian host on which I also installed docker.
I would like docker to be able to access folders automatically mounted/unmounted by autofs outside of docker (managed by debian itself)

Is it possible to do this or is it a limitation of docker to not run autofs?

thanks

meyay · March 30, 2023, 7:53pm

If autofs is handeld by the os, than the bind mount needs to use bind propagation, so the container will actually be able to see when somthing is automaticly mounted/unmounted.

See. https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation, any of the propagations except private and rprivate should work.

dartsteven · March 30, 2023, 8:43pm

I thank you very much but I just don’t understand how to do it.

For example : if the second nas I have is turned off, docker fails to start and returns me this error

docker: Error response from daemon: invalid mount config for type “bind”: bind source path does not exist

instead from debian host even if the second nas is off, Autofs understands it and doesn’t give me error (It simply shows me an empty directory)

meyay · March 31, 2023, 6:15am

Please share the exact docker run command or compose file content that is used to create your container. Furthermore, let us know which path is base path for your autofs mounts, and give an example for mounts inside that folder.

dartsteven · March 31, 2023, 11:57am

Thank you always for your patience.
I’ll preface this by saying that I’m trying to run PLEX with docker to which two nas are to be connected in nfs autofs, here are my configurations :

fstab:

192.168.10.3:/volume1/Video /media/Synology/Film nfs defaults 0 0
192.168.10.4:/volume1/Video /media/Synology/SerieTV nfs defaults 0 0

/etc/auto.master

/- /etc/auto.mount --timeout=10

/etc/auto.mount

/media/Synology/Film -fstype=nfs4,ro 192.168.10.3:/volume1/Video
/media/Synology/SerieTV -fstype=nfs4,ro 192.168.10.4:/volume1/Video

this is what happens if I do an ls from debian and both nas are turned off

cd /media/Synology/
ls
ls: cannot access ‘Film’: No such file or directory
ls: cannot access ‘SerieTV’: No such file or directory

Film SerieTV

if I turn the first nas

ls /media/Synology/
ls: cannot access ‘/media/Synology/SerieTV’: No such file or directory
Film SerieTV

If I try to create the docker while nas is off

sudo docker-compose up -d
[+] Running 5/5
plex 4 layers [⣿⣿⣿⣿] 0B/0B Pulled 4.7s
5544ebdc0c7b Pull complete 1.4s
b36f82e974c3 Pull complete 1.8s
37820355e5b7 Pull complete 1.8s
3d16035a9415 Pull complete 3.2s
[+] Running 0/1
⠿ Container plex Starting 0.9s
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting “/media/Synology/Film” to rootfs at “/media/nvme/OSAPP/Docker/plex/film”: mount /media/Synology/Film:/media/nvme/OSAPP/Docker/plex/film (via /proc/self/fd/6), flags: 0x5000: no such file or directory: unknown

this is my docker-compose configuration file

version: "3.9"
services:
 plex:
   container_name: plex
   image: plexinc/pms-docker

   restart: unless-stopped
   ports:
    - 32400:32400/tcp
    - 3005:3005/tcp
    - 8324:8324/tcp
    - 32469:32469/tcp
    - 32410:32410/udp
    - 32412:32412/udp
    - 32413:32413/udp
    - 32414:32414/udp

   environment:
    - PUID=1000
    - PGID=1000
    - TZ=Europe/Rome
    - PLEX_CLAIM=secret
    - ADVERTISE_IP=http://192.168.10.100:32400/

   volumes:
    - ./config:/config
    - ./plex:/data

    - type: bind
      source: /media/Synology/Film
      target: /media/nvme/OSAPP/Docker/plex/film/

    - type: bind
      source: /media/Synology/SerieTV
      target: /media/nvme/OSAPP/Docker/plex/serietv/

If I run compose with only the first nas on

sudo docker-compose up -d
[+] Running 5/5
plex 4 layers [⣿⣿⣿⣿] 0B/0B Pulled 5.1s
5544ebdc0c7b Pull complete 1.5s
b36f82e974c3 Pull complete 1.9s
37820355e5b7 Pull complete 2.0s
3d16035a9415 Pull complete 3.6s
[+] Running 1/2
Network plex_default Created 0.0s
⠿ Container plex Starting 0.9s
Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting “/media/Synology/SerieTV” to rootfs at “/media/nvme/OSAPP/Docker/plex/serietv”: mount /media/Synology/SerieTV:/media/nvme/OSAPP/Docker/plex/serietv (via /proc/self/fd/6), flags: 0x5000: no such file or directory: unknown

If I turn on both nas then it works

sudo docker-compose up -d
[+] Running 1/1
Container plex Started

But beware that usually when I have plex active I use only one nas at a time and if I create the container with both nas turned on and then turn off only one of them it crashes everything.

Whereas if I use Plex “natively” on debian, I don’t experience any problems if I turn off either of the nas.
Maybe this is a limitation of Docker or am I doing something wrong?

thank you very much

meyay · March 31, 2023, 5:21pm

I am not sure if it works when the target folders are directly used as bind source.
Though, you can try it like this:

   volumes:
   ...
    - type: bind
      source: /media/Synology/Film
      target: /media/nvme/OSAPP/Docker/plex/film/
      bind:
        propagation: rshared

    - type: bind
      source: /media/Synology/SerieTV
      target: /media/nvme/OSAPP/Docker/plex/serietv/
      bind:
        propagation: rshared

I never configured a bind propagation using the long syntax, but according specs, it should look lile that.

If it complains about the mount points not being shared, you can share it like this

sudo mount --make-shared /media/Synology

I can’t tell you if it needs to be configured after each reboot, as it’s been a couple of years since I used mount propagation.

Update: fixed formating

dartsteven · March 31, 2023, 5:49pm

unfortunately it doesn’t work

Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting “/media/Synology/SerieTV” to rootfs at “/media/nvme/OSAPP/Docker/plex/serietv”: mount /media/Synology/SerieTV:/media/nvme/OSAPP/Docker/plex/serietv (via /proc/self/fd/6), flags: 0x5000: no such file or directory: unknown

meyay · March 31, 2023, 6:25pm

What happens if you only mount a single bind to the top folder of both mount points? I know it will mess with your folder name mapping, as the folder names on your share use capital letters on some place, while your container paths didn’t. It’s still worth trying:

- type: bind
  source: /media/Synology/
  target: /media/nvme/OSAPP/Docker/plex/
  bind:
    propagation: shared

dartsteven · March 31, 2023, 7:22pm

same result

Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting “/media/Synology/SerieTV” to rootfs at “/media/nvme/OSAPP/Docker/plex/serietv”: mount /media/Synology/SerieTV:/media/nvme/OSAPP/Docker/plex/serietv (via /proc/self/fd/6), flags: 0x5000: no such file or directory: unknown

But wouldn’t it be possible to compile docker with integrated autofs? In the whole planet am I the only one with this problem?

meyay · March 31, 2023, 9:06pm

Can you share your whole compose file again? The error message doesn’t make sense… You use a bind, the changes should be immediately reflected to the configuration.

You can raise an issue about a missing feature; Issues · moby/moby · GitHub.
Most people use volumes backed by nfsv4, instead of mounting remote shares on the host and binding them into a container. I wouldn’t do it any way different from that.

dartsteven · March 31, 2023, 9:30pm

Ofcourse:

version: “3.9”
services:
plex:
container_name: plex
image: plexinc/pms-docker
restart: unless-stopped
ports:
- 32400:32400/tcp
- 3005:3005/tcp
- 8324:8324/tcp
- 32469:32469/tcp
- 32410:32410/udp
- 32412:32412/udp
- 32413:32413/udp
- 32414:32414/udp

environment:
- PUID=1000
- PGID=1000
- TZ=Europe/Rome
- PLEX_CLAIM=claim-secret
- ADVERTISE_IP=http://192.168.10.100:32400/

volumes:
- ./config:/config
- ./plex:/data
- type: bind
source: /media/Synology/SerieTV
target: /media/nvme/OSAPP/Docker/plex/serietv/
bind:
propagation: rshared

meyay · March 31, 2023, 9:36pm

Like I though. Please look at this post again: Docker + Autofs : Does it work? - #8 by meyay

The idea was not to remove a bind and keep the paths of the other bind as is. It was to use the parent folder of the mount as src.

dartsteven · March 31, 2023, 9:44pm

nope, if the nas2 is turned off

flags: 0x5000: no such file or directory: unknown

meyay · April 1, 2023, 9:26am

My bad. I completely overlooked that your autofs root is /, and each mount has the full path.
In this combination, my suggestion of course can’t work, as no part of the path exists before entering the final folder.

My suggestion only makes sense, if autofs is configured like this:

/etc/auto.master

/media/Synology /etc/auto.mount --timeout=10

/etc/auto.mount

Film -fstype=nfs4,ro 192.168.10.3:/volume1/Video
SerieTV -fstype=nfs4,ro 192.168.10.4:/volume1/Video

This way the autofs mount base folder is present when the container starts.

dartsteven · April 1, 2023, 11:45am

In the indirect autofs method I have to use these strings otherwise nothing is mounted in the Movies and TV Series folders by debian itself

/etc/auto.master

/- /etc/auto.mount --timeout=10

/etc/auto.mount

/media/Synology/Film -fstype=nfs4,ro 192.168.10.3:/volume1/Video
/media/Synology/SerieTV -fstype=nfs4,ro 192.168.10.4:/volume1/Video

meyay · April 1, 2023, 12:24pm

Got it, you tried my last suggestion, and it was not working for you.

Then there is nothing that can be done here. If you feel it’s a bug or a missing feature, please raise an issue in the moby github project (link is one of the posts above).

Please keep us updated about the solution you finally decided to use, so others having the same situation can profit from your experience.

dartsteven · April 1, 2023, 12:33pm

ok perfect, thank you very much for all the help.
I will try to play with it some more now, if I find a solution I will be here to write it.
Otherwise I will open a ticket

meyay · April 1, 2023, 12:40pm

I just tried my own suggestion again:

[me@host ~ ]$ sudo cat /etc/auto.master.d/nfs.autofs
/mnt/nfs /etc/auto.nfs

[me@host ~ ]$ sudo cat /etc/auto.nfs
docker -fstype=nfs,rw,retry=0 192.168.x.x:/volume1/docker

[me@host ~ ]$ ls -l /mnt/nfs
total 0

[me@host ~ ]$ ls -l /mnt/nfs/docker
-rwxr-xr-x  1      1026 users    6375 Apr  1 14:30  file_in_share
...

You might first raise an issue to the maintainer of autofs, if it’s not possible to use a different autofs root other than /-. I use /mnt/nfs and as you can see above, it works like a charm. I know this isn’t working in raspbian on a RPi (though, it didn’t work with /- as autofs root either)

dartsteven · April 1, 2023, 12:48pm

Perfect, I did exactly as you wrote above and it mounts the folders for me too.

ls -l /mnt/nfs/
total 0
drwxrwxrwx 1 root root 148 Jan 24 19:48 film
drwxrwxrwx 1 root root 28 Mar 3 22:10 serietv

Can I ask you the last favor, could you write me the docker-compose so I can see if it works exactly? maybe I am really doing something wrong…

again thank you very much

meyay · April 1, 2023, 12:56pm

May I suggest you modify your compose file and test it. Also please share it here so others can see the new solution.

If it’s not working, please copy/paste the error message additional to your current compose file.