Copying Directories with TOML Files into Docker Image Using Find Command

Docker Desktop
, , ,
1

I am trying to build a Docker image that copies all the .toml files from a specific directory on the host machine into the Docker image while preserving the directory structure. I want to achieve this by using the find command with the exec option.

Here is the directory structure on the host machine:

/host/path/to/toml/files
    ├── subdirectory1
    │   ├── file1.toml
    │   └── file2.toml
    ├── subdirectory2
    │   ├── file3.toml
    │   └── file4.toml
    ├── subdirectory3
    │   └── file5.toml
    └── Dockerfile

Expected Directory Structure in Docker Container:

/app
    ├── subdirectory1
    │   ├── file1.toml
    │   └── file2.toml
    ├── subdirectory2
    │   ├── file3.toml
    │   └── file4.toml
    └── subdirectory3
        └── file5.toml

Dockerfile:

FROM <base_image>

# Create app directory
WORKDIR /app

# Create a temporary directory in the container
RUN mkdir /temp

# Copy .toml files from host to temporary directory
RUN find /host/path/to/toml/files -type f -name '*.toml' -exec cp --parents {} /app/ \;

# Remove the temporary directory
RUN rm -rf /temp

# Set entry point or CMD if needed

But when I build the Docker image, it fails with an error message.

When I build the Docker image using this Dockerfile, it fails with an error message. I suspect that the issue is with the find command or the directory paths. How can I modify my Dockerfile or the find command to copy only the directories with .toml files and preserve their respective directory structure?

Additionally, the purpose of adding these special filters is for learning and customizability sake. I want to have the flexibility to modify and extend the filter criteria in the future.

2

Of course, it fails! RUN instructions can not access files from the host.

To copy files from the build context into the image, the COPY (or ADD) instruction needs to be used.

Note: It is not possible to use any host path outside the build context.

3

Thanks for your response but I do not know how I can first find those set directories and then COPY them to /app . How could I do such a thing?

4

It is not possible like that with COPY/ADD.

Like the docs I linked indicate, you can use Go’s filepath.Match rules. It doesn’t look like it allows rules that search recursively for a match.

You can try, if you get lucky using https://docs.docker.com/engine/reference/builder/#run---mount:

RUN --mount=type=bind,source=.,target=/mnt \
         find /mnt/path/to/toml/files -type f -name '*.toml' -exec cp --parents {} /app/ \;

Though, I am afraid it still can’t access the host filesystem like you intend to do.

This feature requires a custom dockerfile sytax to be added in the first line of the Dockerfile and only works if buildkit is used (which is the default for recent docker versions):

# syntax=docker/dockerfile:1.2

I am not using that feature, as I strongly believe that image builds should be repeatable, which is not the case if you fetch files from arbitrary folders of your host, which may or may not be tainted with additional files. I only use COPY to add files to my images. Thus, I won’t be able to help you with how to use run --mount.

5

I ignore the fact that I agree with @meyay and that I don’t understand why you would copy yaml files that aren’t in your docker context because you could have your reason. I hope you have :slight_smile:

Maybe I miss something, but if you know the command which you wanted to add to the Dockerfile, you can just run it on the host and copy the local “app” folder. I would of course choose a different name if you have an existing local fodler with that name. You don’t have to do everything with Docker. Writing a script that calls docker build is perfectly fine.

update:

Oh yes, I missed the Windows host. But you can run the command from WSL or use a windows equivalent command.

6

Thank you both for your valuable suggestions. I have been experimenting with Dockerfile’s flexibility regarding directory structuring and filtering in general. I am considering a plan where I copy all directories to /app and utilize a bash file executed by the Dockerfile to remove unnecessary folders. Subsequently, the Dockerfile itself would delete the bash file from the container machine after it is ran. Would you consider this approach to be more effective?

7

Even though this might be an effective solution, I doubt it will be an efficient solution…

Every COPY and RUN instruction will create a new image layer. It is not possible to delete files added in one image layer in a subsequent layer - the files just get marked deleted in the subsequent layer.