Docker Community Forums

Share and learn in the Docker community.

Docker - Using DOCKER_BUILDKIT to pass a token secret during build time

I have a pip installable package that was published using twine to Azure DevOps artifact.

In my build image I need to download that package and install it with pip. So I need to
authenticate against azure artifacts to get it. So I am using artifacts-keyring to do so

The pip installable URL is something like this:

https://<token>@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

Actually I am using docker image build --build-arg ... approach to succeed.
My Dockerfile is:

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.7 as intermediate
RUN apt-get update && apt-get install -y git && \
    wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    apt-get update; \
    apt-get install -y apt-transport-https && \
    apt-get update && \
    apt-get install -y dotnet-sdk-5.0
# dotnet core is required from keyring artifacts-keyring pip package below
# that package is used to authenticate against azure devops.

ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

# I created a variable to pass the Azure PAT Token
ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN

RUN \
    pip install --upgrade pip && \
    # INSTALL artifacts-keyring
    pip install pyyaml numpy lxml artifacts-keyring && \
    # INSTALL THE PACKAGE passing the token on the url
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

It works and I manage to install my private package via pip inside the image I built.

The thing is I heard a more proper approach to pass secrets during build time is Docker BuildKit. So I redefined my Dockerfile slightly of this way:

#PREREQUIREMENTS APT GET PACKAGES INSTALLED
...

ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

# Not sure If i need to define token env variable
ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN

# I mounted the secets on the destination /run/secrets/azdevopstoken in my container
RUN --mount=type=secret,id=azdevopstoken,dst=/run/secrets/azdevopstoken

RUN \
    pip install --upgrade pip && \
    # INSTALL artifacts-keyring
    pip install pyyaml numpy lxml artifacts-keyring && \

    # HOW CAN I PASS THE SECRET HERE AT THE URL SINCE IT IS SUPPOSED TO BE MOUNTED? 
    # $AZ_DEVOPS_TOKEN IS NOT VALID HERE ...
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

The thing is that my secret is not being mounted and even if I manage to do it, I don’t know how to associate it to the URL when pip install is executed:

pip install -i https://<SECRETMOUNTED>@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

Perhaps my approach is not the correct one. Can someone point me out in the right direction, please?

You might think you could ADD a secret to your Dockerfile, do what you need with it, and then rm the file. No problem, right? Nope — the secret that you added will persist in the intermediate Docker layers, which are cached. Here’s an example Dockerfile that adds a secret file to a Docker image and then deletes it (catting out secrets is done for demonstration purposes and is not a secure practice):

FROM alpine:edge
ADD file_containing_secrets.txt /tmp/file_containing_secrets.txt
RUN cat /tmp/file_containing_secrets.txt
RUN echo “pre-deletion /tmp listing:” && ls /tmp
RUN rm /tmp/file_containing_secrets.txt
RUN echo “post-deletion /tmp listing:” && ls /tmp
Since you deleted your secret from your Docker image, you could be forgiven for thinking that your secret would never see the light of day. Here, we run a Docker build with this Dockerfile.

echo “mysecretcontents” > file_containing_secrets.txt
docker build --no-cache -f Dockerfile -t foobar .
Sending build context to Docker daemon 67.58kB
Step 1/6 : FROM alpine:edge
—> 3e8d7a5561f0
Step 2/6 : ADD file_containing_secrets.txt /tmp/file_containing_secrets.txt
—> c4ffeb46c264
Step 3/6 : RUN cat /tmp/file_containing_secrets.txt
—> Running in 25ed810b9957
mysecretcontents
Removing intermediate container 25ed810b9957
—> 263ca69cebfb
Step 4/6 : RUN echo “pre-deletion /tmp listing:” && ls /tmp
—> Running in 5925c1de68ce
pre-deletion /tmp listing:
file_containing_secrets.txt
Removing intermediate container 5925c1de68ce
—> b8b441d8fe65
Step 5/6 : RUN rm /tmp/file_containing_secrets.txt
—> Running in 62faa4c8f798
Removing intermediate container 62faa4c8f798
—> 87cc8dfc9f26
Step 6/6 : RUN echo “post-deletion /tmp listing:” && ls /tmp
—> Running in 10953cc51075
post-deletion /tmp listing:
Removing intermediate container 10953cc51075
—> 34ca6fc8fd13
Successfully built 34ca6fc8fd13
Successfully tagged foobar:latest

Hi bgarcial,

i have the same probelm just with jfrog. You can check if your secret ist mounted with a cat like the example on the Docker docs. It should look like this.

It should be now shown in the build process.
Basicly my question is the same, how to tell Docker now to take this secret and use it @ the SECRET in my URL. If some one could help us, that would be nice

Blockquote
FROM internalartifactory/ubi8/ubi:8.3-227
#Disable RHSM
RUN sed -i -e 's/^(enabled)./\1=0/g’ /etc/yum/pluginconf.d/subscription-manager.conf
#YUM internal Configuration
RUN rm /etc/yum.repos.d/
.repo &&
echo “[ArtifactoryUBI8basesos]” >> /etc/yum.repos.d/artifactory.repo &&
echo “name=Artifactory UBI8 basesos” >> /etc/yum.repos.d/artifactory.repo &&
echo “enabled=1” >> /etc/yum.repos.d/artifactory.repo &&
echo “gpgcheck=0” >> /etc/yum.repos.d/artifactory.repo &&
echo “sslverify=0” >> /etc/yum.repos.d/artifactory.repo &&
#Todo pw remove
echo “baseurl=https://SECRET@internalartifactory/artifactory/ubi8-rpm-remote/x86_64/baseos/os” >> /etc/yum.repos.d/artifactory.repo

That I did was this:

RUN --mount=type=secret,id=azdevopstoken,dst=/run/secrets/azdevopstoken \
    pip install --upgrade pip && \
    pip install pyyaml numpy lxml artifacts-keyring && \
    echo pip install -i https://"$(cat /run/secrets/azdevopstoken)"@pkgs.dev.azure.com/org>/<project>/_packaging/feed/pypi/simple/ --no-cache-dir package-name

I mean since the mount point was done at the destination path /run/secrets/azdevopstoken so I am getting its value with this cat command substitution and it works. $(cat /run/secrets/azdevopstoken