Dockerfile RUN commands not persisting to final image (.NET Core/Runtime)

General
, ,
1

Hello Docker Community,

I’m facing a very perplexing issue where files and packages I install via RUN commands in the final stage of my multi-stage Dockerfile are not present in the resulting image when I inspect the running container. This includes packages installed via apt-get install python3 and files downloaded via wget to /usr/local/bin/ (specifically yt-dlp).

Dockerfile Structure (Simplified Relevant Parts):

# Base SDK for build/publish stages (details omitted for brevity, they produce /app_publish_output)
# These stages build and publish a .NET 9 application.
# FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
# ...
# FROM build AS publish
# ... RUN dotnet publish ... -o /app_publish_output ...

# Final Runtime Stage
FROM mcr.microsoft.com/dotnet/runtime:9.0 AS final
WORKDIR /app

# Copy .NET application artifacts (this part seems to work, my app files are in /app)
COPY --from=publish /app_publish_output .

# Install system dependencies including python3
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        wget \
        ca-certificates \
        python3 \
    && rm -rf /var/lib/apt/lists/*

# Install yt-dlp
RUN echo "INFO: Installing yt-dlp in final stage..." && \
    wget https://github.com/yt-dlp/yt-dlp/releases/latest/download/yt-dlp -O /usr/local/bin/yt-dlp && \
    chmod a+rx /usr/local/bin/yt-dlp && \
    echo "INFO: yt-dlp installed to /usr/local/bin/yt-dlp"

# Verification during Docker build (these steps PASS during the 'docker build' process)
RUN echo "VERIFICATION DURING BUILD:" && \
    echo "1. Python3 check:" && \
    (command -v python3 && python3 --version || echo "BUILD FAIL: python3 not found") && \
    echo "2. yt-dlp check:" && \
    (command -v yt-dlp && /usr/local/bin/yt-dlp --version || echo "BUILD FAIL: yt-dlp not found") && \
    echo "3. Listing /usr/local/bin/:" && \
    ls -lA /usr/local/bin/

RUN mkdir -p /config/output && chmod -R 777 /config/output

ENTRYPOINT ["dotnet", "Downloader.dll"]

Observations:

  • During the docker build process (with BUILDKIT_PROGRESS=plain --no-cache), the “VERIFICATION DURING BUILD” steps all pass successfully . The logs clearly show:

    • python3 is found and its version is printed.

    • /usr/local/bin/yt-dlp is found, its version is printed, and ls -lA /usr/local/bin/ shows yt-dlp with execute permissions.

  • However, after the image is built and I run a container from it:

    • When I exec into the running container (e.g., docker exec -it <container_id> /bin/sh) and try to find these files:

      • ls -l /usr/bin/python3 (or which python3) shows nothing .

      • ls -l /usr/local/bin/yt-dlp (or which yt-dlp) shows nothing . The /usr/local/bin directory appears empty or does not contain yt-dlp.

    • Consequently, my .NET application (the ENTRYPOINT) also fails at runtime because it cannot find python3 (when yt-dlp tries to run) or yt-dlp itself.

Question:

Why would installations performed via RUN commands (like apt-get install and wget to a system path like /usr/local/bin/) in the final stage of a Dockerfile seemingly succeed during the image build (as verified by subsequent RUN commands in the same stage), but the resulting files are not present in the filesystem of a container run from that final image?

It’s as if the layer modifications from these RUN commands in the final stage are not being committed to the final image, or are being somehow obscured or lost when the container is actually started.

I’ve ensured:

  • I’m building with --no-cache.

  • I’m inspecting the container run from the correctly tagged final image.

  • There are no volume mounts in my docker run command that would obscure these system paths.

  • I’m using mcr.microsoft.com/dotnet/runtime:9.0 as the base for the final stage.

Has anyone encountered a similar situation where RUN command effects in the final stage don’t persist into the running container’s filesystem, despite build-time checks passing? Is there a nuance to Docker layering or the specific base image that might cause this?

Thanks for any insights!

2

The answer must be related to how you create the container for this image. Please share how you create the container for this image (as in exact command, or if compose was used, the content of the compose file.

Furthermore, please also add the exact command you use to build the image.

3

Hello, thank you for your response. Based on your suggestion, I tried running the docker run command manually, and it worked. Previously, I was running the Dockerfile directly through the JetBrains Rider interface, using the same parameters as in the manual command. However, for some unknown reason, the docker run executed by Rider doesn’t seem to preserve the files that were installed or downloaded during the image build process.

In any case, thank you!