Docker - Using DOCKER_BUILDKIT to pass a token secret during build time

General Discussions
1

I have a pip installable package that was published using twine to Azure DevOps artifact.

In my build image I need to download that package and install it with pip. So I need to
authenticate against azure artifacts to get it. So I am using artifacts-keyring to do so

The pip installable URL is something like this:

https://<token>@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

Actually I am using docker image build --build-arg ... approach to succeed.
My Dockerfile is:

FROM mcr.microsoft.com/azure-functions/python:3.0-python3.7 as intermediate
RUN apt-get update && apt-get install -y git && \
    wget https://packages.microsoft.com/config/ubuntu/20.10/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
    dpkg -i packages-microsoft-prod.deb && \
    apt-get update; \
    apt-get install -y apt-transport-https && \
    apt-get update && \
    apt-get install -y dotnet-sdk-5.0
# dotnet core is required from keyring artifacts-keyring pip package below
# that package is used to authenticate against azure devops.

ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

# I created a variable to pass the Azure PAT Token
ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN

RUN \
    pip install --upgrade pip && \
    # INSTALL artifacts-keyring
    pip install pyyaml numpy lxml artifacts-keyring && \
    # INSTALL THE PACKAGE passing the token on the url
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

It works and I manage to install my private package via pip inside the image I built.

The thing is I heard a more proper approach to pass secrets during build time is Docker BuildKit. So I redefined my Dockerfile slightly of this way:

#PREREQUIREMENTS APT GET PACKAGES INSTALLED
...

ARG ARTIFACTS_KEYRING_NONINTERACTIVE_MODE
ENV ARTIFACTS_KEYRING_NONINTERACTIVE_MODE=true

# Not sure If i need to define token env variable
ARG AZ_DEVOPS_TOKEN
ENV AZ_DEVOPS_TOKEN=$AZ_DEVOPS_TOKEN

# I mounted the secets on the destination /run/secrets/azdevopstoken in my container
RUN --mount=type=secret,id=azdevopstoken,dst=/run/secrets/azdevopstoken

RUN \
    pip install --upgrade pip && \
    # INSTALL artifacts-keyring
    pip install pyyaml numpy lxml artifacts-keyring && \

    # HOW CAN I PASS THE SECRET HERE AT THE URL SINCE IT IS SUPPOSED TO BE MOUNTED? 
    # $AZ_DEVOPS_TOKEN IS NOT VALID HERE ...
    pip install -i https://$AZ_DEVOPS_TOKEN@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

The thing is that my secret is not being mounted and even if I manage to do it, I don’t know how to associate it to the URL when pip install is executed:

pip install -i https://<SECRETMOUNTED>@pkgs.dev.azure.com/<org>/<project>/_packaging/<feed>/pypi/simple/ --no-cache-dir <package-name>

Perhaps my approach is not the correct one. Can someone point me out in the right direction, please?

3

Hi bgarcial,

i have the same probelm just with jfrog. You can check if your secret ist mounted with a cat like the example on the Docker docs. It should look like this.

It should be now shown in the build process.
Basicly my question is the same, how to tell Docker now to take this secret and use it @ the SECRET in my URL. If some one could help us, that would be nice

Blockquote
FROM internalartifactory/ubi8/ubi:8.3-227
#Disable RHSM
RUN sed -i -e 's/^(enabled)./\1=0/g’ /etc/yum/pluginconf.d/subscription-manager.conf
#YUM internal Configuration
RUN rm /etc/yum.repos.d/.repo &&
echo “[ArtifactoryUBI8basesos]” >> /etc/yum.repos.d/artifactory.repo &&
echo “name=Artifactory UBI8 basesos” >> /etc/yum.repos.d/artifactory.repo &&
echo “enabled=1” >> /etc/yum.repos.d/artifactory.repo &&
echo “gpgcheck=0” >> /etc/yum.repos.d/artifactory.repo &&
echo “sslverify=0” >> /etc/yum.repos.d/artifactory.repo &&
#Todo pw remove
echo “baseurl=https://**SECRET**@internalartifactory/artifactory/ubi8-rpm-remote/x86_64/baseos/os” >> /etc/yum.repos.d/artifactory.repo

4

That I did was this:

RUN --mount=type=secret,id=azdevopstoken,dst=/run/secrets/azdevopstoken \
    pip install --upgrade pip && \
    pip install pyyaml numpy lxml artifacts-keyring && \
    echo pip install -i https://"$(cat /run/secrets/azdevopstoken)"@pkgs.dev.azure.com/org>/<project>/_packaging/feed/pypi/simple/ --no-cache-dir package-name

I mean since the mount point was done at the destination path /run/secrets/azdevopstoken so I am getting its value with this cat command substitution and it works. $(cat /run/secrets/azdevopstoken