Nfs_export disabled with overlay2 as storage driver

Hello, I have been having some trouble getting NFS exports to work after migrating from aufs to overlay2 as the storage driver for my docker containers. I have a pretty simple container that has a directory (/srv/nfs) that contains static files that are shared via an NFS server. The files there are initialized when the image is built and are read only while the container is running.

With aufs as the storage driver, when the container starts exportfs is run and the files are exported and available. With overlay2 as the backing storage driver however, exportfs fails with the error message “exportfs: /srv/nfs does not support NFS export”.

Digging into this a bit I found the following documentation for overlay’s options (Overlay Filesystem — The Linux Kernel documentation) and set the index and nfs_export options to be enabled in the kernel module. When I run docker with these options set, the mounts used by docker to provide the containers filesystem (as seen in /etc/mtab have both settings overridden to be disabled.

A workaround that I have found is to declare /srv/nfs to be a volume in the Dockerfile, however this seems to have significant performance drawbacks as on startup the volume must be initialized with the contents from the image layers which can be a couple GB of data. On container exit it must also be cleaned up as once the volume is not used beyond that point.

The questions I have at this point that I have not been able to find documentation or answers for:
Where/how does docker override these options when starting the container?
Is there a good reason why these have to be disabled such that it is inadvisable for me to try to enable them for the container?

I’d be happy to provide additional information or snippets from my Dockerfile if details are needed to better understand the issue. Thanks!

Adding some additional details on software versions for the system:
Docker version 20.10.12, build e91ed57
Ubuntu 20.04.4 LTS running kernel 5.13.0-30-generic

I could not make it work yet, but your usecase seems to be highly unusual. Interestingly I almost tried something similar some weeks ago just to test something on NFS, but in my case I wouldn’t have stored the data in the image. so a volume or a bind mount would have worked for me.

It wouldn’t be a solution for the topic but If the image is for storing the data that you want to export, could you just download the data itself and store on your host, then bind mount into the container? Whenever you need to access those files, you could bind mount again into the NFS server or even into other containers in read-only mode.

It seems that this is intentional:

I also noticed this line in dmesg when trying to start a container while the host has index=Y and nfs_export=Y:

overlayfs: disabling nfs_export due to index=off

Unfortunately the Github discussion referenced in the commit is ~9 months after the original kernel commit so it looks like there might be a reason why it’s not possible to set index=on. I added a question to the PR to see if there’s a solution.

I’m also going to look into building the overlay driver directly and see if there is something that can be tweaked to make this work.

ps: Hi!

1 Like

Hi Cody!

Thanks for your response, this is precisely what I was looking for! I read through the PR you linked and it seems like there at least was a real issue with setting index=on.
I did try it myself though, and rebuilt the kernel and modules for the host OS to always set index=on and nfs_export=on regardless on what docker requests in the mount options. Since I have very little go experience I figured this was easier then modifying docker, but I think a similar effect could be tested by removing the index=off override from docker.
When running a container with the kernel module set to always enable index and nfs_export it appears to run smoothly and allow nfs_export as expected. I still need to test the rest of the workflow that uses the actual files on the nfs share, but so far so good.