I’m using Docker Swarm in a homelab and I’m trying to figure out how to proper use the long syntax to mount NFS volumes, as described in Services top-level elements | Docker Docs
What I’m observing is the docker compose ignore when I told it to deploy in a subpath of a NFSv4 volume from my NAS, installing the container on /mnt/storage/docker instead of /mnt/storage/docker/container.
I’m using Portainer Community 2.21.4, Docker engine v26.1.5 and docker compose v2.27.0. Below is a piece of compose as example:
What could I do to improve this? I really don’t want to go down the path of creating a different dataset on the NAS for each container, otherwise I’ll just have to continue using bind volumes.
I moved the topic to the Swarm category based on the beginning of your message, but then you write about Docker Compsoe. So what are you using? Docker Swarm or Docker Compose?
Sorry, I think I mixed up the terms. I’m using swarm and deploying stacks, which clearly use a similar structure to compose but it’s not the same thing.
Note: Volumes are immutable. Changes to volumes of a compose file will not be reflected back to the actual volume configuration. In order to apply the new configuration, the existing volume must be manually deleted, and then re-created.
I’m not following your way of thinking, sorry. Do you mean that there is not an “official” 26.1.5 version? Well, I don’t know if this can help in any way, but the docker package I’m using comes from the community version of Alpine Linux: docker - Alpine Linux packages
Yeah, I noticed that when I was troubleshooting some syntax errors on my compose file, but even re-creating it I wasn’t able to make it recognize the subpath parameter. I even tried to use relative or absolute path there to see if it make a difference (ex.: “./cloudbeaver” and “/mnt/Storage/docker/cloudbeaver” instead of just “cloudbeaver”), but no luck. I even tried to deploy with no pre-existing folder, wondering if it could create a new one or crash if none exist. Ignored as well, not even an error.
Indeed, it is not a release from docker’s repositories. It must be maintained and supported by whoever maintains the alpine package.
Even the link to the package you shared pre-dates the fix from August. Thus, it is impossible this package already has the fix that is required for subpath to work with Swarm services.
Update: I have no idea where I read about the fix being merged in August. It was already in April. Still the question remains whether this fix is already included in the Docker version you use. I would normally look it up in the docker release notes… which of course don’t exist for a version that was never released by Docker…
During the week I tried to research a little more to understand where the community version of Docker came from, but given the current state of my learning curve about Linux I see that I will have to spend a lot of energy on this, which does not seem productive to me given the size of the problem. Going down the path of uninstalling this version and installing an official one seems the most coherent, but I could not find any information (and I do not trust ChatGPT’s answer) as to whether going down this path will not lead me into a rabbit hole of incompatibilities and conflicts due to the distribution’s customizations (Alpine Linux). What do you think? Does my concern make sense or am I overreacting to something that is simpler than it seems? In any case, I will just make a backup here and take a chance.
You could try to replace the binaries with those from the binary release, but I wouldn’t be surprised if it didn’t work due to Alpine using musl as standard c library. I never tried, and I am not curious enough to try it. I always used docker-ce on supported Linux distributions.
Frankly, I would ignore the subpath setting, and configure things as if didn’t exist. Once Alpine has a v27 based package version, I would try whether subpaths works with it. If using subpath is more important than using Alpine, then I suggest using your personal preference from the list of supported distributions and install vanilla docker-ce on it.
I agree with you, from what little I understand they seem to me to be arrangements of different work philosophies. I thought of this as a way to make access to persistent volumes more granular rather than mounting a general volume folder on the host, but in reality there are other possible options, perhaps suboptimal, but they exist, so it’s not the end of the world. Two things I learned from working in engineering is that we can’t get attached to solutions and we need to choose well the problems we want to solve.
When you specify a named volume, the path specified in device: is not restricted to the exported host folder on the nfsv4 server itself. It can be any subfolder within the exported host folder as well.
E.g. if your nfs4 export is :/mnt/Storage/docker, then this will work (Given the subfolder already exists inthe remote share and permissions allow accessing it):
I understand that for simple applications like this this works well, but I was expecting to use subpath as an elegant solution when you need to provide access to several folders and files, like Traefik, for example:
In this situation, correct me if I’m wrong, I would be forced to mount each folder and file, which doesn’t seem very practical to substitute bind volume for it, and I suppose that would not be good practice too.
It is indeed not possible to append a path to a volume name on the service level declaration of a volume. You need to create an individual volume in the top level volumes section per path, and use the unique volume names in the service volume definition.
I highly doubt that top level volumes support single file mounts - from my experience a volume always represent a folder.
If you need read-only files, then passing the files in, storing their content in a config or secret would be the way go. Both allow in their service level long syntax definition to specify a target path.
Note: Kubernetes volumes/volumeMounts support subpath mounts that could be folders or files since ages.
The export on the nas is :/volume1/pve, which has a template subfolder that holds Proxmox vm templates.
docker compose up: the ls command only listed items from :/volume1/pve/template`. That’s the expected outcome.
docker stack deploy -c compose.yml test: the ls command lists evey item from :/volume1/pve. Thus subpath did not act like a filter, as it did with the compose deployment.
So even on official docker-ce 27.3.1, subpaths are not working for swarm stack deployments. This is clearly a bug!
Indeed, migrating to Kubernetes is something I’m considering in the future, both for this reason and because of certificate management and other topics, but it’s further ahead in my learning curve and current needs, but that’s a topic for another day.
Well, I’m glad someone else was able to reproduce what I’m experiencing, so apparently it’s not a problem with the distribution. Do you think it’s worth opening a issue on github?
Maybe it makes sense to create an issue in the Moby project and link the other subpath related issues, and the one from swarmkit.
Docker Desktop has an experimental feature that helps to migrate from compose file based deployments to k8s: https://docs.docker.com/compose/bridge/. In my experience it gets most things converted right, but doesn’t handle volumes well. Note: this feature is not available in docker-ce.