Docker Community Forums

Share and learn in the Docker community.

Glob style copy in dockerfiles

Support the ability to copy all files of a given extension to the same destination folder as the source folder.

So, for example:

On the host machine:

  • /test/test1.src
  • /test/random.rtc
  • /more/more3.src
  • /more/otherstuff.txt
  • /grapes/oranges/carrots.src

Allow me to copy all *.src files to the container, putting them into the folders they came from:

  • /test/test1.src
  • /more/more3.src
  • /grapes/oranges/carrots.src

This would be SO useful for making layers of .net applications. (It would let me pull out all the .csproj files and restore them before I do the compile.)

2 Likes

This is old suggestion butI think is indeed needed.
We build .net core projects using Docker. We have a deep tree of folders with dependent projects.
So in order to utilize docker caching mechanism we separate dotnet restore phase. For it we need to only copy *.csproj files keeping folder structure. But we cannot do it with COPY **/*.csproj ./. We have to make separate instructions to copy all layers. For example,

COPY */*.csproj ./
COPY Folder1/*/*.csproj Folder1/
COPY Folder2/*/*.csproj Folder2/
COPY Folder2/InnerFolder1/*/*.csproj Folder2/InnderFolder1/

After that we run custom script to recreate folder structure and move copied files in respected folders. This is inconvenient.

Can it be solved?

advantage of Docker cache capabilities, you could try the following workaround:

Devise a wrapper script (say, in bash) that copies the required package.json files to a separate directory (say, .deps/) built with a similar hierarchy, then call docker build …
Adapt the Dockerfile to copy (and rename) the separate directory beforehand, and then call yarn install --pure-lockfile…
All things put together, this could lead to the following files:

./build.bash

#!/bin/bash

tag=copy-example:latest

rm -f -r .deps # optional, to be sure that there is

no extraneous “package.json” from a previous build

find . -type d ( -path */.deps ) -prune -o
-type f ( -name “package.json” )
-exec bash -c ‘dest=".deps/1" && \ mkdir -p -- "(dirname “$dest”)" &&
cp -av – “$1” “$dest”’ bash ‘{}’ ;

instead of mkdir + cp, you may also want to use

rsync if it is available in your environment…

sudo docker build -t “$tag” .
and

./Dockerfile

FROM …

WORKDIR /usr/src/app

COPY package.json . # subsumed by the following command

COPY .deps .

and not “COPY .deps .deps”, to avoid doing an extra “mv”

COPY yarn.lock .
RUN yarn install --pure-lockfile

COPY . .

Notice that “COPY . .” will also copy the “.deps” folder; this is

maybe a minor issue, but it could be avoided by passing more explicit

paths than just “.” (or by adapting the Dockerfile and the script and

putting them in the parent folder of the Yarn application itself…)