Docker-py and build secret mounts, how to handle?

General
1

Hi there,

I’m writing a python based program that I want to be able to build Docker images using the docker package.

However, I’m not entirely sure how to handle mounting build secrets since the API documentation for building doesn’t list an option for passing build secrets mount. It additionally says “This is not meant for passing secret values.” regarding the buildargs parameter.

I did find that there is a client.secrets.create() method, but that seems to store the secrets in a server, and I get an error regarding docker swarm:

docker.errors.APIError: 503 Server Error for http+docker://localhost/v1.46/secrets/create: Service Unavailable ("This node is not a swarm manager. Use "docker swarm init" or "docker swarm join" to connect this node to swarm and try again.")

I’m not trying to use swarm or store the secrets in the server anyways, as I just need to handle it locally as it builds without wanting to store it anywhere else.

The local setup that I’m trying to programmatically replicate:

My Dockerfile has these lines in it for mounting the required dotfile that contains the token, since the token is required to pip install private packages.

...
RUN --mount=type=secret,id=dotfile,dst=/root/.dotfile \
    pip install -r requirements.txt
...

And then I build it with:

DOCKER_BUILDKIT=1 docker build . --secret id=dotfile,src=/home/me/.dotfile --target my-layer --tag program:1.0.0```

Is there then a way to run the above build command but via the python docker package without needing to store the secret in a server or use Docker swarm?

2

Mounting secrets is done by buildkit which doesn’t seem to be supported by docker-py:

A little explanation what I think why it is not supported:

The legacy builder without buildkit just ran simple Docker containers for each instruction in the Dockerfile. Buildkit does not run Docker containers. As far as I know it runs containers directly with the low level container runtime called runc. So it is not something you would easily handle through the Docker API to which you connect with the Python SDK for Docker.

So when you got the error message when calling client.secrets.create it was from the Docker API. 503 Server error is just a standard HTTP server error code and message. It doesn’t mean there is an additional server where the secrets are stored. You just manage secrets through the Docker API.

But secrets are supported by Docker Swarm and there is some support for secrets in Docker Compose as well, but it is probably not managable through the Docker API without Swarm mode.

When you mount a secret using buildkit - and this is the point where I’m guessing - buildkit could mount the additional file into the root filesystem used by runc. At least this is how I imagine it.