COPY --from same source in different images produces a different layer

Docker Hub
,
1

Hello,

I have tried multiple things but the current version is the following:

I have one Docker image containing node_modules.
I want to COPY --from base --chown=node:node this base image node_modules directory into multiple other images.
As this is the exact same node_modules directory I’m expecting COPY --from to produce the same layer in each different image.
But what I’m getting is that COPY from is producing layers with the exact same size but different IDs.

The end goal is to leverage layer caching when pulling all the different images. The only difference between each image should be the actual code (few Mb).

Thanks :slight_smile:

2

Just to confirm: could you create another base image which you can then use with FROM derived_base directly? If you could not, then I’d guess Docker is doing the expected thing, as apparently (if you could not rewrite for some FROM derived_base rather than COPY --from base) then the Nth layer is not the same for all derived images?

You may want to show your Dockerfiles and used commands to provide some more details.

3

@avbentem Thanks for your response.
I can totally switch to using the base image without copying, I’m just trying to understand why a COPY --from is generating different layers.

Sample Dockerfiles

Base

FROM node:lts-alpine
RUN apk add --no-cache make g++ gcc python3 libc-dev libtool autoconf automake git
WORKDIR /home/node/app
COPY . ./
RUN perl -i -p -e 's/postinstall/d/g;' package.json
RUN yarn --production --frozen-lockfile  && \
    yarn run prisma:gen && \
    yarn autoclean --init && \
    echo *.ts >> .yarnclean && \
    echo *.ts.map >> .yarnclean && \
    echo *.spec.* >> .yarnclean && \
    yarn autoclean --force && \
    yarn cache clean

Actual

ARG BASETAG
FROM node:lts-alpine AS builder
ARG PROJECT
ARG PORT=5000
RUN apk add --no-cache make g++ gcc python3 libc-dev libtool autoconf automake git
WORKDIR /home/node/app
COPY . ./
RUN perl -i -p -e 's/postinstall/d/g;' package.json
RUN yarn --frozen-lockfile && \
    yarn run prisma:gen && \
    yarn nx run ${PROJECT}:build:production


ARG BASETAG
FROM <redacted>.amazonaws.com/baseimage:${BASETAG} AS deps

FROM node:lts-alpine
ARG PROJECT
ARG PORT=5000
RUN chmod g+rw /home && \
    mkdir -p /home/node/app && \
    chown -R node:node /home/node
ENV HOME /home/node
ENV NODE_ENV production
WORKDIR /home/node/app
COPY --from=deps --chown=node:node /home/node/app/node_modules /home/node/app/node_modules
COPY --from=builder --chown=node:node /home/node/app/dist/apps/nest/${PROJECT} /home/node/app/dist
USER node
CMD node ./dist/main.js
1 Like
4

So, you’re seeing the following creating a new layer in the final image for the COPY:

I’d indeed assume that caching would be possible, but not my expertise.

Some more details for others who may be able to help: do you see CACHED for the RUN command? (I guess you do.) Do you see cache being used when repeatedly creating the same target image? (I guess not.) And curious: did you try to use the image name in COPY --from <redacted>.amazonaws.com/baseimage:${BASETAG}?

As for COPY the documentation also mentions “metadata”, but I’ve no idea what that would refer to in your use case:

If anything has changed in the file(s), such as the contents and metadata, then the cache is invalidated.

5

Oh, if that gives you some “repository name must be lowercase” error, then that may be related to the (unresolved) How can I expand a variable within a COPY command in the Dockerfile? (But that is digressing from your original problem; you could simply try with a hardcoded tag name instead.)

6

I tried that and faced the same problem mentioned, it doesn’t work. But conceptually I don’t think it would be really different than using that.

ARG BASETAG
FROM <redacted>.amazonaws.com/baseimage:${BASETAG} AS deps

And to be clear I’m more interested in pull cache than build cache but one goes with the other I think.

7

It probably refers to this:

root@ta-lxlt:/var/lib/docker/image/overlay2/imagedb# tree

.
├── content
│   └── sha256
│       ├── 05d3c1e2d0c166b1654f86a5beed614065450b186a94e4761120b3458b45509d
│       ├── 0995532c7014f6467a8601ac04221325ca2436e4682f1311d8ea90e55f968a6b
│       ├── 27dbe388ad4abb0d64a301ce6cd4c352c5e0df45a6bef30cfec88f8e81174ddd
│       ├── 28f52b60203d1bf12eaafe8d2fabd7807f0954c44f425b572061957ba1797729
│       ├── 2f21415cb85f27d2069174ca001af3c0ebb95a5b3c3520bf59af30395481deb0
│       ├── 3218b38490cec8d31976a40b92e09d61377359eab878db49f025e5d464367f3b
│       ├── 3a445eeb2c32272d658cbc7af4d4b0b58e19e0233da5ea4304098ad1340260fe
│       ├── 4a30feacddebc46c52bc88da560be5971caf9147b8f67f92e93d97ff2ddf2eab
│       ├── 4f572736733873dfbe613939449456ec37afc92651d824aef37fb656287e06e5
│       ├── 6a03c8e7e2be03e010d21a78c56090566e418e6957a97e5b8906b0c8df7d4e5b
│       ├── 7b46d4496bd9ae946ed80b226df56a4a01bbe7f8e7138573ada3503710e4b13b
│       ├── 9b566eee7c2e8eff17bbedcfcb4a71dca73551b890ac73090310814e85b85c27
│       ├── a10ac28f147ab614c76f6b30740642e02804f2967219b1f09cb0e9589d54fd89
│       ├── d8ebb336ffd860b5fd25ab6757f179cacf1b671a64aeb6a6836803138695d353
│       ├── dabbfbe0c57b6e5cd4bc089818d3f664acfad496dc741c9a501e72d15e803b34
│       └── dbf758a9f11bf0c65563225e70e5afab9ed63697dca479d384c1f0f16d6128d2
└── metadata
    └── sha256
        ├── 27dbe388ad4abb0d64a301ce6cd4c352c5e0df45a6bef30cfec88f8e81174ddd
        │   └── parent
        ├── 3a445eeb2c32272d658cbc7af4d4b0b58e19e0233da5ea4304098ad1340260fe
        │   └── parent
        ├── 4f572736733873dfbe613939449456ec37afc92651d824aef37fb656287e06e5
        │   └── parent
        ├── 6a03c8e7e2be03e010d21a78c56090566e418e6957a97e5b8906b0c8df7d4e5b
        │   └── lastUpdated
        ├── 9b566eee7c2e8eff17bbedcfcb4a71dca73551b890ac73090310814e85b85c27
        │   ├── lastUpdated
        │   └── parent
        ├── a10ac28f147ab614c76f6b30740642e02804f2967219b1f09cb0e9589d54fd89
        │   └── lastUpdated
        ├── d8ebb336ffd860b5fd25ab6757f179cacf1b671a64aeb6a6836803138695d353
        │   └── lastUpdated
        └── lastUpdated

11 directories, 25 files

The cache should have worked unless a previous layer was invalidated or the “deps” changed.
Can you share your build log similar to this?

ta@ta-lxlt:~/dockertest$ docker build -t localhost/bashapp-copy-2 -f Dockerfile.copy . --build-arg BASETAG=itsziget
Sending build context to Docker daemon  1.056MB
Step 1/8 : ARG BASETAG
Step 2/8 : FROM ubuntu:20.04 as build
 ---> ba6acccedd29
Step 3/8 : RUN mkdir /build
 ---> Using cache
 ---> af180d413c81
Step 4/8 : ARG BASETAG
 ---> Using cache
 ---> 6dabd9da64b6
Step 5/8 : FROM ${BASETAG}/httpd24 as deps
 ---> 4dc82615e468
Step 6/8 : FROM ubuntu:20.04 as app
 ---> ba6acccedd29
Step 7/8 : COPY --from=deps --chown=1000:1000 /usr/local/apache2/conf /app
 ---> Using cache
 ---> 474ce9f4624a
Step 8/8 : COPY --from=build --chown=1000:1000 /etc/hosts /app/hosts
 ---> Using cache
 ---> 154e7c383b4c
Successfully built 154e7c383b4c
Successfully tagged localhost/bashapp-copy-2:latest