Cannot install vieux/sshfs plugin on rootless docker

Open Source Projects DockerEngine
1

Hi All,

I have a working rootless docker environment. Now I wanted to move the data/volumes to another VM and decided to do it using SSHFS (encrypted, easy to install and configure).

When I tried installing the vieux/sshfs plugin as per the instructions (link to plugin here), it fails eventually, due to some permission issues.

At this point I am not sure if I can use the plugin at all in a rootless environment, or if I should mount the volumes to the host directly (using SSHFS)?

dockerUser@uWeb:~/$ docker plugin install vieux/sshfs
Plugin "vieux/sshfs" is requesting the following privileges:
 - network: [host]
 - mount: [/var/lib/docker/plugins/]
 - mount: []
 - device: [/dev/fuse]
 - capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
latest: Pulling from vieux/sshfs
Digest: sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811
52d435ada6a4: Complete
Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:76: mounting "/var/lib/docker/plugins/" to rootfs at "/mnt/state" caused: stat /var/lib/docker/plugins/: permission denied: unknown
dockerUser@uWeb:~/$ docker plugin ls
ID             NAME                 DESCRIPTION               ENABLED
cb8c58758170   vieux/sshfs:latest   sshFS plugin for Docker   false

“dockerUser” has no other permissions whatsoever in the system, but is the user that I used to rollout docker.

dockerUser@uWeb:~/$ id dockerUser
uid=1000(dockerUser) gid=1000(dockerUser) groups=1000(dockerUser)
2

I saw your previous topic and wanted to recommend vieux/sshfs instead. Then I tried and failed using SSH keys. The last version is 4 years old. Now I tried with rootless Docker but I could not install any plugin yet. It looks like rootless Docker is not completely support plugins. It could work but we should be able to change the plugin directory since rootless Docker does not have permission to write /var/lib/docker/plugins

3

Good day rimelek

Eventually, I found a way to install the vieux/sshfs plugin on my rootless docker environment. I had to specify the “state.source” (which --in the absence of documentation-- I understood as the path to the plugin directory).

dockerUser@uWeb:~$ docker plugin install vieux/sshfs state.source=/home/dockerUser/.local/share/docker/plugins/ sshkey.source=/home/dockerUser/.ssh
Plugin "vieux/sshfs" is requesting the following privileges:
 - network: [host]
 - mount: [/var/lib/docker/plugins/]
 - mount: []
 - device: [/dev/fuse]
 - capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
latest: Pulling from vieux/sshfs
Digest: sha256:1d3c3e42c12138da5ef7873b97f7f32cf99fb6edde75fa4f0bcf9ed277855811
52d435ada6a4: Complete
dockerUser@uWeb:~$ docker plugin ls
ID             NAME                 DESCRIPTION               ENABLED
2d8337628b16   vieux/sshfs:latest   sshFS plugin for Docker   true

Then I can create a volume. Unfortunately, the plugin has hardcoded the volumes’ directory to “mnt/volumes/” (see config.json), where is not accessible for the rootless user. So what happens actually is, the volume creation doesn’t fail at this point, rather the volume gets created by the plugin, but there is no “actual” volume created at “mnt/volumes” of course (or anywhere else).

dockerUser@uWeb:~$ docker volume create -d vieux/sshfs -o sshcmd=datamgr@uData.lan:/home/datamgr/data/docker/volumes/ -o IdentityFile=/home/dockerUser/.ssh/id_rsa -o allow_other testsshvolume
testsshvolume
dockerUser@uWeb:~$ docker volume ls
DRIVER               VOLUME NAME
vieux/sshfs:latest   testsshvolume
dockerUser@uWeb:~$ docker volume inspect testsshvolume
[
    {
        "CreatedAt": "0001-01-01T00:00:00Z",
        "Driver": "vieux/sshfs:latest",
        "Labels": {},
        "Mountpoint": "/mnt/volumes/9f9d256a5fbcc046e8826c439083837a",
        "Name": "testsshvolume",
        "Options": {
            "IdentityFile": "/home/dockerUser/.ssh/id_rsa",
            "allow_other": "",
            "sshcmd": "datamgr@uData.lan:/home/datamgr/data/docker/volumes/"
        },
        "Scope": "local"
    }
]

Indeed the plugin hasn’t been updated in the past 4 years and nobody cared to check that it works in a rootless environment… This particular plugin is used as an example throughout the official docker documentation!

I do not know what else to do, someone needs to update the plugin I suppose…

And I cannot get the direct SSHFS mount to the host’s /volumes directory to work either! I will not setup the horrendous and slow NFS in my NAS just because SSHFS is not really supported by docker. I am thinking to have a look at podman or other container solutions.

And for the sake of completion, here is confirmation the volume doesn’t actually exist:

dockerUser@uWeb:~$ docker run -d --name sshtest -v testsshvolume:/app nginx:latest
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
eff15d958d66: Pull complete
1e5351450a59: Pull complete
2df63e6ce2be: Pull complete
9171c7ae368c: Pull complete
020f975acd28: Pull complete
266f639b35ad: Pull complete
Digest: sha256:097c3a0913d7e3a5b01b6c685a60c03632fc7a2b50bc8e35bcaa3691d788226e
Status: Downloaded newer image for nginx:latest
8b44dd517ba610900a47d6660f26745d35f4a98f708e5def2143abe3478457ae
docker: Error response from daemon: error while mounting volume '/mnt/volumes/9f9d256a5fbcc046e8826c439083837a': VolumeDriver.Mount: sshfs command execute failed: exit status 1 (read: Connection reset by peer).
4

I think /mnt/volumes is a path inside the plugin container. I don’t know exactly how these plugins work. I have never made one.

I will try it later. Maybe we can make it work. I didn’t find this parameter before.

5

I tried again but no luck unfortunately. Same errors.

But I am a little bit uncertain if I want to use this plugin at all (even if it worked) in a rootless setting. I did some reading about the “[CAP_SYS_ADMIN]” privilege (aka capability) and if I understand it right, it totally beats the purpose of going “rootless”. fyi, read this , this and this.

I wonder if the “local” volume driver also takes up the same privileges, but I didn’t find any relevant documentation (not surprised, the docker documentation is very thin).

6

Usually yes. In this case only the plugin would have this capability not the containers connecting to it. Although I was surprised you could even install the plugin without root and requiring admin capability.

Probably I wouldn’t use that either. You could mount the SSHFS folder on the host with some limitations. I know you tried that but I saw that already deleted topic and you tried to mount everything to Docker’s own volume directory. Don’t do such a thing. I didn’t try it but I would not recommend accessing (besides reading) those folders directly. If you mount a folder into .local/docker/volumes (I am not sure it was the exact path) and you create a volume from it, then Docker would probably want to create the folder which is already their. Again, I have not tested it. Use another folder in your $HOME and mount that. Example:

mkdir -p $HOME/.sshfsvolumes/project1/volume1
mkdir -p $HOME/.sshfsvolumes/project1/volume2

sshfs user@server:/project1 $HOME/.sshfsvolumes/project1

docker run --name sshfstest --rm -d -it -u $(id -u):$(id -g) -v $OME/.sshvolumes/project1:/app ubuntu:20.04 bash
docker exec -it sshfstest ls -la /app/volume1

I couldn’t mount volume1 directly. Docker sad it wanted to create the folder which was already there.

I know it is not what you originally wanted but you could create symlinks pointing to /app/volume1 from any location inside the container (docker image)

I hope there is a better way but this is what I had time for.