I have my local computer and a vps server. I want to run an experiment where I run docker on the vps but the container that I build will be on another machine (I’m not looking to add a volume to a docker container)
I can ssh in to the vps and then sshfs to mount an external drive of my local computer onto my vps
iptables -A INPUT -s 123.456.78.9 -j ACCEPT // allow my vps on my local computer
ssh root@myvps.com -i mykey // ssh to my vps
sshfs -o allow_root,default_permissions root@011.23.58.13:/mnt/d1/ /mnt/d1/ -o IdentityFile=/root/mykey.pub // mount my local drive on my server
Still on my vps I run docker using my drive
dockerd -H unix:///var/run/docker1.sock -p /var/run/docker1.pid --ip-masq=true --bridge=br1 --data-root=/mnt/d1/docker-data --exec-root=/mnt/d1/docker-exec --debug
INFO[2023-10-27T15:25:13.397535535Z] Starting up
WARN[2023-10-27T15:25:13.397782724Z] Running experimental build
DEBU[2023-10-27T15:25:14.846130491Z] Listener created for HTTP on unix (/var/run/docker1.sock)
INFO[2023-10-27T15:25:14.847085018Z] detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf
DEBU[2023-10-27T15:25:16.938257472Z] Golang's threads limit set to 2700
DEBU[2023-10-27T15:25:19.670716238Z] Cleaning up old mountid : start.
failed to start daemon: error setting up metrics plugin listener: listen unix /mnt/d1/docker-exec/metrics.sock: bind: operation not permitted
And I’m not sure why root is being denied to bind to the metrics.sock when I can see that it successfully created the root docker folders and the docker socket? Could it be something like Apparmor or the way I’m mounting?
I’m not sure I understand, can you clarify what is “another machine”? Is that your local machine? I guess because you only mentioned the VPS and local machine later.
You need Docker on the machine on which you want to run a Docker container. The container is the process that runs and the isolation. What you could have on another machine is the Docker data directory, which you are trying to mount from your local machine, but I wouldn’t do that either and I don’t think that having a unix socket on a remote server would work. That is really a local thing. The Docker data dir have special files. The best if you keep it local and mount only data from another machine.
If you really want to mount a remote docker data dir, I remember I mounted it years ago using an NFS share, but I don’t do it recently. I just mount docker volumes which you stated you don’t want, but that would be the right way.
The current scenario appears to be that you want the docker daemon process and all container processes to run on the vps host, but mount a remote sshfs share for the docker data-root folder.
Unix domain sockets are used from inter process communication on that host, as such the socket pseudo file must be “stored” on the host where the consumer and producer processes are running. Every container that depends on unix domain sockets would have the same problem. Furthermore, sshfs is not a supported backend for overlay2, so probably docker will use vfs as storage driver.
Mounting a remote share as data-root is generally a terrible idea, as overlay2, won’t work, and file locking most likely won’t be available either and fall back to vfs.
If it’s about backing up volume data, backup your volumes and store the backup on the sshfs remote share or back it up on a s3 complient storage. If it’s about not storing data on the vps as plain text, you could consider to fully encrypt the vps filesystem with LUKS, which will require authentication when booting the vps or mounting an encrypted driver or partition…