SSHFS mounting on the host fails with rootless docker

Open Source Projects DockerEngine
1

Hi All,

I would like to have docker volumes stored at and made available from a separate machine (VM) using SSFS mounts.

I have a working rootless docker environment. Is it at all possible to mount the entire “volumes” folder or should I be mounting each and every volume instead?

First emptied the volumes folder. Then I stopped the docker.service, mounted the empty remote folder to the local docker “volumes” folder using the same unpriviledged user.

dockerUser@uWeb:~$ rm /home/dockerUser/.local/share/docker/volumes/metadata.db                                                         dockerUser@uWeb:~$ sshfs -o idmap=user,allow_other,IdentityFile=/home/dockerUser/.ssh/id_rsa datamgr@uData.lan:/home/datamgr/data/docker/vol2/ /home/dockerUser/.local/share/docker/volumes/

When I restart docker.service, it fails. But it is not clear to me why.

dockerUser@uWeb:~$ systemctl --user restart docker.service                                                                          dockerUser@uWeb:~$ systemctl --user status docker.service
● docker.service - Docker Application Container Engine (Rootless)
     Loaded: loaded (/home/dockerUser/.config/systemd/user/docker.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2021-11-17 13:31:00 UTC; 13min ago
       Docs: https://docs.docker.com/go/rootless/
    Process: 3544 ExecStart=/usr/bin/dockerd-rootless.sh (code=exited, status=1/FAILURE)
   Main PID: 3544 (code=exited, status=1/FAILURE)

Nov 17 13:30:58 uWeb systemd[799]: docker.service: Failed with result 'exit-code'.
Nov 17 13:31:00 uWeb systemd[799]: docker.service: Scheduled restart job, restart counter is at 3.
Nov 17 13:31:00 uWeb systemd[799]: Stopped Docker Application Container Engine (Rootless).
Nov 17 13:31:00 uWeb systemd[799]: docker.service: Start request repeated too quickly.
Nov 17 13:31:00 uWeb systemd[799]: docker.service: Failed with result 'exit-code'.
Nov 17 13:31:00 uWeb systemd[799]: Failed to start Docker Application Container Engine (Rootless).

The SSHFS mountpoint seems to be working:

#from the local host uWeb
dockerUser@uWeb:~$ touch /home/dockerUser/.local/share/docker/volumes/testfile
dockerUser@uWeb:~$ ls -la /home/dockerUser/.local/share/docker/volumes/
total 8
drwx-----x  1 dockerUser   uadmin        4096 Nov 17 14:13 .
drwx--x--- 14 dockerUser  dockerUser  4096 Nov 17 14:08 ..
-rw-rw-r--  1 dockerUser   uadmin               0 Nov 17 14:13 testfile

#from the remote VM uData
datamgr@uData:~$ ls -la data/docker/vol2
total 8
drwx-----x 2 datamgr datamgr 4096 Nov 17 14:13 .
drwxrwxr-x 4 datamgr datamgr 4096 Nov 17 13:25 ..
-rw-rw-r-- 1 datamgr datamgr    0 Nov 17 14:13 testfile

Any ideas?

2

To show that docker works without any SSHFS mounts:

First I un-mounted the SHFS mountpoint using an account with sudo rights:

uadmin@uWeb:~$ sudo fusermount -u /home/dockerUser/.local/share/docker/volumes

Then restarted and confirmed docker.service was running:

dockerUser@uWeb:~$ systemctl --user start docker.service
dockerUser@uWeb:~$ systemctl --user status docker.service
● docker.service - Docker Application Container Engine (Rootless)
     Loaded: loaded (/home/dockerUser/.config/systemd/user/docker.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2021-11-17 14:22:39 UTC; 2min 16s ago
       Docs: https://docs.docker.com/go/rootless/
   Main PID: 3967 (rootlesskit)
     CGroup: /user.slice/user-1000.slice/user@1000.service/docker.service
             ├─3967 rootlesskit --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-drive>
             ├─3978 /proc/self/exe --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-dr>
             ├─3996 slirp4netns --mtu 65520 -r 3 --disable-host-loopback --enable-sandbox --enable-seccomp 3978 tap0
             ├─4003 dockerd
             └─4021 containerd --config /run/user/1000/docker/containerd/containerd.toml --log-level info

Nov 17 14:22:39 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:39.863745673Z" level=warning msg="Running modprobe bridge br_netfilter failed wit>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.048539994Z" level=info msg="Default bridge (docker0) is assigned with an IP ad>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.683119378Z" level=error msg="a39a18771ddbd9b73e57dfd304a28e66c073aa8a7dd31cad8>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.683158924Z" level=error msg="failed to start container" container=a39a18771ddb>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.705723824Z" level=error msg="failed to prepare mountpoints for container" cont>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.705766899Z" level=info msg="Loading containers: done."
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.715145003Z" level=warning msg="Not using native diff for overlay2, this may ca>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.715364353Z" level=info msg="Docker daemon" commit=e2f740d graphdriver(s)=overl>
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.715410250Z" level=info msg="Daemon has completed initialization"
Nov 17 14:22:40 uWeb dockerd-rootless.sh[4003]: time="2021-11-17T14:22:40.759075757Z" level=info msg="API listen on /run/user/1000/docker.sock"

dockerUser@uWeb:~$ systemctl --user show --property=Environment docker
Environment=PATH=/usr/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin