Mounting individual config files within another Docker volume

jimtendo · June 28, 2022, 1:51am

Hi all,

I’ve searched for this, but cannot seem to find an answer (my description/search term is probably poor, but I’m struggling to think of more apt phrasing. Apologies in advance.).

I have a docker-compose.yml file with two containers that share an ‘./app’ source volume that requires different config files.
These config files are binding into that existing ‘./app’ volume (that is shared between both containers).
When launching docker-compose up, the files are being copied into the ./app volume (with one container’s configs taking precedence over the other).

This is for a development environment (using node:v16 image) whereby I need these two containers to communicate with each other (albeit with different configurations).

Relevant portions of my docker-compose.yml:

# Container 1
volumes:
	- ./app:/home/node/app # Note that this is same source mount as Container 2
	- ./.docker/container1/.env:/home/node/app/.env
	- ./.docker/container1/database.db:/home/node/app/database.db

# Container 2
volumes:
	- ./app:/home/node/app # Note that this is same source mount as Container 1
	- ./.docker/container2/.env:/home/node/app/.env
	- ./.docker/container2/database.db:/home/node/app/database.db

Is it possible with docker/docker-compose to make it so that the .env and the database.db files (from Container 1 or Container 2) do not land in the ./app folder on my local machine and exist isolated from each other?

I realize this could also be resolved by mounting the .env and database.db in a different directory on each container (e.g. /var/app/.env and /var/app/database.db), but the app does not currently have the capability to configure read from different locations (these files must be placed in the app directory).

Any help/guidance on this would be much appreciated.

Thanks all.

meyay · June 28, 2022, 5:21am

Is this a copy action within your entrypoint script?

If you would just depend on the binds, I see no reason why it shouldn’t work, because you use additional mount points for .env and database.db inside /home/node/app of each of the containers filesystem. Changes made to those two files inside the container should be written to the bound files on the host. But if you run something that generates files, based on the content of both files and write it in /home/node/app, of course bother container will be affected and the state inside this folder will be inconsistent.

Using sub-mounts in mounts works fine with docker-ce.

avbentem · June 28, 2022, 5:37am

Yes. As the part before the colon is clearly a path, not just a simple name, you’re currently using a bind mount, not a named volume. Bind mounts are shared with a path on your local machine.

As a quick test you could simply use two different bind mounts, and see if that works? If not, then I think your question need some more details.

# Container 1
volumes:
	- ./app_container_1:/home/node/app
 
# Container 2
volumes:
	- ./app_container_2:/home/node/app

But not through the /home/node/app folder, right?

jimtendo · June 28, 2022, 6:36am

Is this a copy action within your entrypoint script?

There’s no copy action within the entrypoint script. This is just a simple node:v16 image that runs npm run start and the software will not be creating these files (only reading). (Confirmed this by placing a pointless test.txt submount to /home/node/app which also appeared in my local ./app folder.)

Changes made to those two files inside the container should be written to the bound files on the host.

Perhaps I’m on an older version of Docker and Docker-Compose that behaves differently (leveraging Ubuntu reps). I’ll look into this and see if updating helps.

Using sub-mounts in mounts works fine with docker-ce.

Thanks for this, “sub-mount” is probably the term I’m after - will do some Googling

As a quick test you could simply use two different bind mounts, and see if that works? If not, then I think your question need some more details.

Same result for this. The submounted files still end up in the ./app directory, so looks like it relates to submount behaviour (and not necessarily that two containers are sharing the same ./app mount). Also did docker inspect on my containers and can see that the propagation on volumes is rprivate (which I think is correct).

But not through the /home/node/app folder, right?

That’s correct - not through /home/node/app. These communicate via ZeroMQ.

Will do some more investigation on my side and try updating Docker.

avbentem · June 28, 2022, 7:53am

The ./app folder on your local machine?

~~That should not happen. At least not for, e.g.,:~~ (I didn’t realize how the 2nd and 3rd bind mount were mounted into the target of the first; not sure if this should work.)

# Container 1
volumes:
	- ./app_container_1:/home/node/app
 	- ./.docker/container1/.env:/home/node/app/.env
	- ./.docker/container1/database.db:/home/node/app/database.db

# Container 2
volumes:
	- ./app_container_2:/home/node/app
	- ./.docker/container2/.env:/home/node/app/.env
	- ./.docker/container2/database.db:/home/node/app/database.db

You may want to share the config you’re using now.

jimtendo · June 28, 2022, 12:03pm

Note: I haven’t updated my Docker version yet (am on Docker version 20.10.12, docker-compose version 1.25.0). Will try to do this as a next step tomorrow.

The ./app folder on your local machine?

Apologies, I mis-spoke. I should’ve clarified that the files were created in their respective mounts (i.e. ./app_container_1/.env, ./app_container_2/.env, etc) on my local machine.

I’ve simplified the docker-compose.yml to try to better pin-point the issue (it doesn’t appear related to multiple containers leveraging same ./app mount).

version: "3.5"

services:

  container:
    image: "node:16"
    user: "node"
    working_dir: /home/node/app
    volumes:
      # This folder was created by me prior to running "docker-compose up", but is empty otherwise empty.
      - ./app:/home/node/app
      # File just containing text "TEST" for now.
      - ./.env:/home/node/app/.env
      # File just containing text "DATABASE" for now
      - ./database.db:/home/node/app/database.db
    # This will fail, but is enough to demonstrate behaviour.
    command: "npm run start"

Upon running with docker-compose up (as my user - jimtendo), I end up with the following in my ./app folder:

ls -lah

total 8.0K
drwxrwxr-x 2 jimtendo jimtendo 4.0K Jun 28 21:41 .
drwxrwxr-x 4 jimtendo jimtendo 4.0K Jun 28 21:39 ..
-rwxr-xr-x 1 root     root        0 Jun 28 21:41 database.db
-rwxr-xr-x 1 root     root        0 Jun 28 21:41 .env

One interesting thing of note is that these files are zero bytes each (same in previous situation - apologies, I had not realized this and thought my file-manager was misbehaving).

Permissions for the source mounts are:

drwxrwxr-x  2 jimtendo jimtendo 4.0K Jun 28 21:57 app
-rw-rw-rw-  1 jimtendo jimtendo    9 Jun 28 21:28 database.db
-rw-rw-rw-  1 jimtendo jimtendo    5 Jun 28 21:27 .env

Doing docker inspect on this container yields the following information for mounts:

"Mounts": [
		{
				"Type": "bind",
				"Source": "/home/jimtendo/Projects/Sandbox/docker-test/app",
				"Destination": "/home/node/app",
				"Mode": "rw",
				"RW": true,
				"Propagation": "rprivate"
		},
		{
				"Type": "bind",
				"Source": "/home/jimtendo/Projects/Sandbox/docker-test/.env",
				"Destination": "/home/node/app/.env",
				"Mode": "rw",
				"RW": true,
				"Propagation": "rprivate"
		},
		{
				"Type": "bind",
				"Source": "/home/jimtendo/Projects/Sandbox/docker-test/database.db",
				"Destination": "/home/node/app/database.db",
				"Mode": "rw",
				"RW": true,
				"Propagation": "rprivate"
		}
],

avbentem · June 28, 2022, 3:49pm

Ah, I guess I can imagine Docker would be getting confused when bind mounts are kind of circular references, the first binding some local folder and then other files being mounted into the target of the first. I don’t know if that’s supposed to work.

Despite the zero sizes on your local machine: did you test if you can access the config files from the container? It seems they are 9 and 5 bytes in the container, which matches the DATABASE and TEST plus a newline in those files?

meyay · June 28, 2022, 5:16pm

If you execute the mount command you will see that every mountpoint is where it belongs. But the 0 byte size indeed is strange. Docker mounts the bind volumes like the command mount --bind allows to mount a local path into another local path.

As you don’t want to propagate mountpoints with furher mountpoints in the host path, I would assume the mount progation should be fine.

meyay · June 28, 2022, 5:36pm

I prepared a simple test case for this:

my_user@host:~/test$ ls -lR
.:
total 0
drwxrwxr-x 2 my_user my_user 22 Jun 28 19:29 main
drwxrwxr-x 2 my_user my_user 38 Jun 28 19:23 subs

./main:
total 4
-rw-rw-r-- 1 my_user my_user 15 Jun 28 19:24 main.txt

./subs:
total 8
-rw-rw-r-- 1 my_user my_user 15 Jun 28 19:24 sub1.txt
-rw-rw-r-- 1 my_user my_user 15 Jun 28 19:24 sub2.txt
my_user@host:~/test$ docker run -ti --rm -v $PWD/main/:/main/ -v $PWD/subs/sub1.txt:/main/sub1.txt -v $PWD/subs/sub2.txt:/main/sub2.txt ubuntu
root@359676f31ee3:/# mount | grep main
/dev/mapper/ansible--vg-root on /main type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/ansible--vg-root on /main/sub1.txt type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
/dev/mapper/ansible--vg-root on /main/sub2.txt type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota)
root@359676f31ee3:/# ls -l /main
total 12
-rw-rw-r-- 1 1001 1001 15 Jun 28 17:24 main.txt
-rw-rw-r-- 1 1001 1001 15 Jun 28 17:24 sub1.txt
-rw-rw-r-- 1 1001 1001 15 Jun 28 17:24 sub2.txt
root@359676f31ee3:/# cat /main/main.txt
writte_in_host
root@359676f31ee3:/# cat /main/sub1.txt
writte_in_host
root@359676f31ee3:/# cat /main/sub2.txt
writte_in_host
root@359676f31ee3:/# echo "written_in_container" > /main/main.txt
root@359676f31ee3:/# echo "written_in_container" > /main/sub1.txt
root@359676f31ee3:/# echo "written_in_container" > /main/sub2.txt
root@359676f31ee3:/# cat /main/sub1.txt
written_in_container
root@359676f31ee3:/# cat /main/sub2.txt
written_in_container
root@359676f31ee3:/#
exit
my_user@host:~/test$ cat main/main.txt
written_in_container
my_user@host:~/test$ cat subs/sub1.txt
written_in_container
my_user@host:~/test$ cat subs/sub2.txt
written_in_container

I preped two folders with files,
started the container (it doesn’t matter wether with docker run or docker-compose),
checked if the expected mounts exist (they do!),
checked the contant of the files (they match what I wrote inside the files),
wrote different content in the three files,
exited the contaienr and checked the content from the host (they now have the content written from inside the container)

jimtendo · June 29, 2022, 11:54pm

Sounds like this might be expected behaviour for the time being? If not, let me know and I can create an issue.

Will look at patching the app to allow custom locations via ENV vars in the meantime.

Thanks all for the help, much appreciated.

meyay · June 30, 2022, 5:39am

I am not sure if our understanding align. My test shows that submounts inside the container are respected and that changes to it affect the host files. I did my test’s with docker-ce 20.10.16 from docker’s repository.

There is no guaranty that os repo or snap repo packages behave like a vanilla docker in all aspects. If you want to have vanila behavior, make sure you use docker-ce from docker’s repositores!

Topic		Replies	Views
Docker-compose mounting via volumes option General	0	1046	November 27, 2018
How to make containers access image contents from one another? General docker	3	5003	May 21, 2021
Mount volume Dockerfile or Docker compose General	0	1013	June 26, 2019
Looking for a way to put the same volume paths into one place in docker compose file General docker , docker-compose	7	827	September 16, 2024
Mounting volume where host directory is different on CI server General docker	5	5815	July 15, 2016

Mounting individual config files within another Docker volume

Related topics