Files on host system not seen in bind mount

General
1

I am using a Dockerfile with a docker-compose.yml file, and I have created two bind mounts. My understanding is that bind mounts map to the host file system, and that all files on the host filesystem should be visible and usable in the container via the bind mount. But that’s not happening.

In my Dockerfile I have these two lines:

VOLUME /test-data
VOLUME /nasm-data

In my docker-compose.yml file I create the bind mounts:

version: “3.8”
services:
  srv1:
    volumes:
      - type: bind
        source: /opt/Test_Data/
        target: /test-data
      - type: bind
        source: /opt/P01_SH/Complex_Calc_YZ/
        target: /nasm-data
    build:
      context: /opt/P01_SH/Dockerfiles/
      dockerfile: /opt/P01_SH/Dockerfiles/Dockerfile

volumes:
   test-data:
   nasm-data:

After I build these, I log onto the container’s command line. Both bind mount directories are there (cd /test-data). The bind mount “test-data” should be able to see and open the file “testfile.txt” in its source directory, but:

cd /test-data
cat testfile.txt
cat: testfile.txt: No such file or directory

The same thing happens with the bind mount “nasm-data.” The file in its source folder is not found.

So my questions are:

  1. Am I correct that a bind mount should be able to open and read files from the host filesystem mapped as its source?

  2. Is my procedure for creating the bind mounts, shown in the code above, correct?

  3. How can I correct this so the container can use host system files?

Thanks very much.

2

Please, format your post according to the following guide: How to format your forum posts
In short: please, use </> button to share codes, terminal outputs, error messages or anything that can contain special characters which would be interpreted by the MarkDown filter. Use the preview feature to make sure your text is formatted as you would expect it and check your post after you have sent it so you can still fix it.

Example code block:

```
services:
  service1:
    image: image1
```
3

I believe I have fixed the formatting. Please advise if you see any other formattaing issues.

Thanks.

4

I linked a complete guide which doesn’t mention using &nbsp; as non breaking space and I also shared an example using the “backtick” chraracters before and after the code. I can’t give you more advise. Please, read the guide I linked and follow the instructions next time, because the formatting guide is not just for the indentation but any special character. Thank you!

Now your yaml is readable even if not the way I suggested so let’s see your questions.

  1. A bind mount just mounts whatever folder is on the host. If it is empty, than it will be empty in the container. You mixed the volume and bind mount way. I guess because it is confusing that the “volumes” section can have the actual volume mount and also the bind mount. If it is empty, it could be that the source fodler is also a mounted, remote filesystem and it mounted the original folder not the remote or the source fodler is simply wrong. You can read more about volumes and bind mounts here:

    Volumes | Docker Docs

  2. Yes, except if you want bind mounts, you don’t need to define volumes so the volumes section after the services (not the one where the mount is described) is not needed at all. And I would never use the VOLUME instruction in a Dockerfile as it can’t be “undone” and it is not a requirement if you define bind mounts or volumes in a compose file.

  3. I believe 1. and 2. explains what the correct way is.

5

My question has evolved from what I posted above. I kept the bind mounts defined in the docker-compose.yml file, and added two new bind mounts in the docker run command:

docker run -it --mount type=bind,src=“$(pwd)”,target=/opt/P01_SH/_Debug_Wrappers_in_C --mount type=bind,src=“$(pwd)”,target=/opt/P01_SH/Complex_Calc_YZ ubuntu bash

When I cd to /opt/P01_SH/Complex_Calc_YZ and do “ls” I get the contents of /opt/P01_SH/Dockerfiles, which is the “(pwd)” in the docker run command above, but not the files that show on the host in that folder. The same happens with the other command line bind mount, /opt/P01_SH/_Debug_Wrappers_in_C.

But when I cd to the parent folder /opt/P01_SH/, it shows only the two bind mounts I created in my run command above: Complex_Calc_YZ and _Debug_Wrappers_in_C, although there three more folders in /opt/P01_SH.

I think there are two possibilities:

  1. I should not mount it do “$(pwd)” when that folder it not empty, so I mounted to /media and now ls shows no files at all.

  2. Or I need to set permissions differently in the container. In the host and in the container I am logged on as root. But the permissions on everything are set to 755, which is my standard permissions.

So to summarize my question, when I create a bind mount in the run command as shown above, why do I not see the files in the host folder (the target)? My goal is to use the host files in the container.

Thanks again for any help.

6

I solved the bind mount problems described above, and I am posting my solution in hope that it will help others in the future.

I did not use a bind mount. I switched to volume mounts. Volume mounts are created in /var/lib/docker/volumes, so to create them I use this syntax:

cd /var/lib/docker/volumes/

docker run -it -v $PWD/dv_dwc:/opt/P01_SH/_Debug_Wrappers_in_C -v $PWD/dv_ccyz:/opt/P01_SH/Complex_Calc_YZ -v $PWD/dv_iu:/opt/P01_SH/_Include_Utilities -v $PWD/dv_lib:/opt/P01_SH/_Library -v $PWD/dv_tdata:/opt/P01_SH/_Test_Data -v $PWD/dv_tres:/opt/P01_SH/_Test_Results ubuntu:22.04 /bin/bash

That creates six volume mounts at $PWD, which is whatever directory this command is run from, so it’s important to cd /var/lib/docker/volumes/ before running it.

The volume mounts show up like this:

/var/lib/docker/volumes/dv_ccyz
/var/lib/docker/volumes/dv_dwc
/var/lib/docker/volumes/dv_iu
/var/lib/docker/volumes/dv_lib
/var/lib/docker/volumes/dv_tdata
/var/lib/docker/volumes/dv_tres

However, inside the container the files at dv_ccyz are referenced in the container at /opt/P01_SH/Complex_Calc_YZ, which is what I want – those are the names that my source code will look for the files. The other mounts are referenced in the container at the paths they were mapped to (e.g. $PWD/dv_tdata:/opt/P01_SH/_Test_Data is mapped to /opt/P01_SH/_Test_Data), and that’s where the container will look.

One other very important item: volume mounts are created empty. You must copy into each of the mount points all of the files you want there. I used FTP to the cloud for that. Once you do that, the files will be shared transparently in both the container and on the host, and any changes made to files in the container will be reflected on the host, and vice-versa.

Volumes are persistent, so when your Docker session ends the data are still there.

None of the infomation I gleaned from studying for over a week and asking questions was found in one place; I had to piece it together. So I hope this post helps others in the future.

7

This is probably not documented, as you should probably not mess with the files in a Docker volume folder, as that is handled by Docker.

If you want files from the host to be mounted in a container, use a bind mount.

8

@bluepuma77 - Volume mounts are created empty. How do I use them if I can’t copy files into the container? All of the files I will be using are read-only except for one. What trouble will arise if I use a volume mount?

9

…by starting a container that maps a bind as source and volume as destination, then copy the files from within the container.

It could look like:

docker run --rm \
  -v /host/path:/src \
  -v volume:/dst \
  alpine cp -r /src/* /dst/

You should be able to use an arbitrary host path for the /src mount, unless you use the docker snap package. (Note: The docker snap package can only access files in the home directory. It is maintained and supported by Canonical)

1 Like