Appears build cache is busted once the image is used in a container. Why? How to avoid?

General Discussions
1

I have an image I build that has one RUN stage that only loads packages and another after that does some initialization. If I rebuild immediately it uses the cache for the stages as it should. For that later initialization RUN stage I set up a “manual bust”. If I "bust at that point then it uses the cache for all the former stages. It works fine as intended

BUT

If I use that image in a container and then go back and build the image the entire cache is busted and the whole image rebuilds. I’d like that not to happen as the package install takes the longest and is not required time and time again.

In short I want to test the image and if there are issues only rebuild the last initialization RUN stage not the one that installs packages but once I use the image the entire cache is busted.

Why does this happen? Is there a way to use an image but persist the build cache. I’m wondering why running an image in a container has anything to do with the build cache?

ARG BASE_IMAGE
FROM $BASE_IMAGE
ARG BASE_IMAGE
ARG KEEP
ARG SYSADMIN_PW
ARG LINUX_DISTRO=alpine
ARG BUILD_DIR=/build 
WORKDIR $BUILD_DIR
COPY .src ./

# PACKAGES
RUN echo -e "\n ************************************************* \n"\
    echo "****** Building Image from Base: $BASE_IMAGE; : Distro: $LINUX_DISTRO; *****"; \
    echo " ---- running packages install script ---"; /bin/sh ./packages.sh; \
    echo -e "\n********************************************************" 
# END PACKAGES    

ARG BUST_INIT_CACHE

# INITIALIZATION
RUN echo -e "\n ************************************************* \n" \
    echo "****** BUST_INIT_CACHE ${BUST_INIT_CACHE} "; \
    echo "****** Running Initialization Script "; \
    chmod -R +x .; \ 
    pwd; ls -la; \
    echo " ---- running init script ---"; /bin/bash ./init.sh; \
    echo -e "\n********************************************************" 
# END INITIALIZATION    

VOLUME [ "/data", "/opt", "/shell" ]
WORKDIR /opt
# ENTRYPOINT ["entrypoint.sh"]
CMD ["/bin/bash", "-l"]
2

A cache miss occurs when the value for any ARG passed in as --build-arg has a different value than in the previous build. It leads to a rebuild of that instruction and all instructions that follow it.

See: Dockerfile reference | Docker Documentation

3

Just to make sure I understand: You did absolutely nothing except ran a container and ran docker build with the same parameters as before, right? If I am not right, than most likely @meyay is right about the arguments.

Make sure you don’t run a script that does more than running a container to avoid deleting the cache. Also note that changing the sourcecode would invalidate the COPY layer. Use a .dockerignore
file similar to .gitignore to ignore any temporary files.

4

Yes, The copy command was the problem. I was invalidating the cache in my copy statement.

I needed two copy statements. I need to copy the package and init scripts via their own subfolders. Then if I changed something in the init script subfolder the package copy command is not invalidated.

That also has taken care of the issue of running a container and invaliding the cache. Side benefit is now I don’t need that manual cache busting as just changing anything in the init script folder does it.

COPY .src/packages ./packages
# PACKAGES
RUN \
    echo -e "\n ************************************************* \n"\
    echo "****** Building Image from Base: $BASE_IMAGE; : Distro: $LINUX_DISTRO; *****"; \
    echo " ---- running packages install script ---"; /bin/sh ./packages/packages.sh; \
    echo -e "\n********************************************************" 
# END PACKAGES    

COPY .src/init ./init

# INITIALIZATION
RUN echo -e "\n ************************************************* \n" \
    echo "****** Running Initialization Script "; \
    chmod -R +x .; \ 
    pwd; ls -la; \
    echo " ---- running init script ---"; /bin/bash ./init/init.sh; \
    echo -e "\n********************************************************" 
# END INITIALIZATION