Unable to use NAS as data-root, what am I doing wrong?

Hi, I’ve been trying to set up a fresh mini-server to run some docker projects on. The server itself has next to no storage, so I was hoping to take advantage of my NAS to use as data-root, but I’m running into trouble. I hope someone here can point out where I’m going wrong. Here’s my process:

  1. Fresh install of Ubuntu Server, including setting up a user with SSH access
  2. The rest of the process is done through SSH from another machine, in case that makes any difference
  3. Install cifs-utils and setup a cifs mount point from my NAS in /etc/fstab which looks like [PATH TO NAS] /mnt/data cifs mfsymlinks,vers=3.0,credentials=[PATH TO CREDS],uid=1000,gid=1000
  4. Create /etc/docker/daemon.json and fill it with { "data-root": "/mnt/data/docker" }
  5. Install Docker Engine according to these instructions, including the post-install instructions

Running docker run hello-world then results in docker: Error response from daemon: operation not supported.

I’m presuming there’s some sort sort of permissions or something that I’m missing, but I can’t see it and I’m hoping someone with fresh eyes can point me at the obvious mistake I’m making. Thanks!

data-root will not work on a remote share with a halfway decent storage driver. It will definitly not work with overlayfs2. And the lack of file locking would high likely be a problem as well.

Have you consider to create an iSCSI target on the NAS and mount it using a iSCSI initator as blockdevice on your host?

Thanks, that’s very helpful @meyay. Once we get to that level, would you say I am better off just keeping data-root relatively stock and using the NAS for container specific volumes?

Spot on!

Considerations:

  • Keep the compose files in git
  • Make sure each compose file declares and creates networks and volume alongside the services (as in: avoid external: true)
  • Keep persistent data in docker volumes backed by nfsv4 shares
    • note: technically you can use a docker volume backed by cifs as well, but it will require you to include the credentials in the compose file… and therefor in git… this is not something you will want, not even in a homelab environment.

This way you don’t rely on any state of the Docker Host itself.

Thank you – I have a larger drive turning up today to minimise the issue for now, but I’m going to explore this route too.

One potential issue I have is that the machine is most likely to run Caprover so I’ll need to make sure that I can keep your considerations in mind there too.

Thanks for the help!

Thanks, it is really help me. An iSCSI target on the NAS work well as docker data-root.

1 Like

Of course, any of local or remote block device formatted with the correct filesysten will do.