Exclude sub directory from mounting

General
1

Hi everyone, I immediately expose the problem.

I would like to mount a directory (.), however, excluding an sub directory (vendor).
In practice, I would like the latter to be managed only as an internal volume to docker.

In this regard, I created two files

+ tree -p
├── [-rw-rw-r--]  bind.yml
└── [-rw-rw-r--]  volume.yml

Let’s examine them

+ cat bind.yml
services:
    test:
        image: bash
        volumes:
            - type: bind
              source: .
              target: /app
            - type: bind
              source: vendor
              target: /app/vendor
volumes:
    vendor:



+ cat volume.yml
services:
    test:
        image: bash
        volumes:
            - type: bind
              source: .
              target: /app
            - type: volume
              source: vendor
              target: /app/vendor
volumes:
    vendor:

I point out the only difference

+ diff bind.yml volume.yml
8c8
<             - type: bind
---
>             - type: volume

We start doing some tests (let’s start with bind.yml)

+ docker compose -f bind.yml run --rm test echo OK
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /tmp/tmp.G55luRDlaP/vendor

mmmhhh…

Let’s try again with volume.yml

+ docker compose -f volume.yml run --rm test echo OK
OK

OK, but

+ tree -p
├── [-rw-rw-r--]  bind.yml
├── [drwxr-xr-x]  vendor
└── [-rw-rw-r--]  volume.yml

As you can see, the directory vendor was created on the host.
This thing is not good for me :frowning:

As I said at the beginning, I would like this directory to be excluded, that is, that it is managed only as an internal volume to Docker (without going to dirty the file system on the host).
How can it be done?

2

Why would you expect it to not exist on the host path, when the mount point for the volume vendor is a subdirectory of a bound host directory?

The correct test would have been to check the content of the folder. Neither should changes in the host vendor folder be visible inside the container path, nor should any changes to the container path /app/vendor be visible on the host.

3

First of all sorry for the delay. I had some problems.

???
bohhh

It seems to me that the vendor volume mount point is associated with the homonymous named volume vendor

              source: vendor
volumes:
    vendor:

However is it possible to avoid creating this directory on the host?

4

No, not like you intend to use it.

  1. you can not exclude folders when you bind host folders into container folders. It binds the same inode the host folder uses into the container. Exactly how mount --bind does it.
  2. a folder used as a mount point must exist in order to mount the source into the folder. You can not prevent docker from creating the folder for mount points you use. Trying the same on the host would just result in an error.
  3. a volume is mounted into a mount point

Please make yourself squinted how mount points in Linux work, and what happens if the directory used as mount point does not exist.

1 Like
5

But what if, instead of a directory, I wanted to exclude a file?
I tried this

$ tree -p
└── [-rw-rw-r--]  docker-compose.yml

$ cat docker-compose.yml 
services:
    test:
        image: bash
        command: sh -c "echo abc > /app/file"
        volumes:
            - type: bind
              source: .
              target: /app
            - type: volume
              source: file
              target: /app/file
volumes:
    file:

$ docker compose run --rm test
sh: can't create /app/file: Is a directory

mmmhhh…

$ tree -p
├── [-rw-rw-r--]  docker-compose.yml
└── [drwxr-xr-x]  file

$ rmdir file && touch file && tree -p
├── [-rw-rw-r--]  docker-compose.yml
└── [-rw-rw-r--]  file

Now it should work

$ docker compose run --rm test
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting “/var/lib/docker/volumes/tmpk7n5ibxa03_file/_data” to rootfs at “/app/file”: create mount destination for /app/file mount: cannot mkdir in /var/lib/docker/overlay2/6aff0f7cc6002f6a8172dd4a18cc539935e5cf5e727f88c278b2a3769a827cc4/merged/app/file: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type

mmmhhh…

Why can’t it be done?

6

Because a file can not be used as a mount point for a directory (=the volume)?
It must either be non-existent (then docker creates the directory for you) or an existing directory.

7

???

Are you asking who?

Yes, but apart from that, here is another experiment

$ tree -p
├── [-rw-rw-r--]  docker-compose.yml
├── [-rw-rw-r--]  file1
└── [-rw-rw-r--]  file2

$ cat docker-compose.yml 
services:
    test:
        image: bash
        volumes:
            - ./file1:/app/file1
            - file2:/app/file2
volumes:
    file2:

# COMMAND1
$ docker compose run --rm test sh -c "> /app/file1"

# COMMAND2
$ docker compose run --rm test sh -c "> /app/file2"
sh: can't create /app/file2: Is a directory

CONCLUSION

COMMAND1: work
COMMAND2: not work

Why?

8

in your compose, file2 specifies a named volume, not the path to your actual ‘file2’

9

OK, but there is a specific reason why a named volume does not accept files (only directories)?

10

It’s a named volume, it does not exist until you first run the container, then Docker creates it - There’s nothing specified that defines this as a file, how do you expect Docker to guess you want a file?

And no, I do not think named volumes can be files

2 Likes
11

It is not accepting or not. Named volume is a directory. Config and secret can be a file.

And even if you didn’t know that, that’s a very good question from @deanayalon :

1 Like
12

you could implement a setting to let the user decide…

13

What would it be Config?
What would it be secret?

14

Please, learn about the Docker basics and Linux. That is essential in order to understand what you are doing and why things work like they do. Searching for “docker config” and “docker secret” should give you plenty of results.

This topic was about excluding a sub directory when mounting from the host. It is not possible. Hiding on the other hand is possiblée, but if you talk about directories in the title and files in your posts, you will confuse everyone. And if you write a folder as a file in the command assuming a volume will turn into a file just by knowing you need a file not a folder, it is not surprising it will fail. These basic staffs can be learned really quickly so it is time to start from the beginning.

Recommended links to learn the basics and concepts:

Here is an example to hide a folder by mounting a volume:

services:
  test:
    image: bash
    command: ["sleep", "inf"]
    init: true
    volumes:
      - type: bind
        source: .
        target: /app
      - type: volume
        source: data
        target: /app/data
volumes:
    data:

It will show an empty folder in the container instead of “data” if data exists on the host. If not, it will be created because the mount point has to exist and and the volume will be mounted over that folder.

1 Like
15

True.
But continuing the work in my project, in addition to some directories, the need was born to “free” some files.
I don’t know if it was appropriate to open a separate thread.