Docker BuildKit: Wait for local image to build

Docker Engine General
1

I have the following project structure

./docker/Base/Dockerfile
./docker/ServiceA/Dockerfile
./docker/ServiceB/Dockerfile
./docker-compose.yml

ServiceA and ServiceB share the same base image to avoid having stuff like package installation duplicated. The base Dockerfile looks like this:

FROM python:3.10-slim
# Do stuff
CMD tail -f /dev/null

The ServiceA and ServiceB Dockerfiles look like this:

FROM servicebase:latest
# Do different stuff
CMD /foo/service[A|B]

In the docker-compose.yml file, it is my goal to first build and tag the base image locally, and then build the image for services A and B.

version: '3.8'

services:
  servicebase:
    image: servicebase
    build:
      context: .
      dockerfile: docker/Base/Dockerfile
    container_name: servicebase
  serviceA:
    depends_on:
      - servicebase
    build:
      context: .
      dockerfile: docker/ServiceA/Dockerfile
  serviceB:
    depends_on:
      - servicebase
    build:
      context: .
      dockerfile: docker/ServiceB/Dockerfile

The problem I face is that when I run docker compose up, the Docker BuildKit tries to parallelize the building of the images of ServiceA and ServiceB to the building of the base image. During the build of image of Service(A|B), servicebase therefore was not built yet, such that the builder then queries Dockerhub and fails, as servicebase is not on Dockerhub:

Status: pull access denied for servicebase, repository does not exist or may require ‘docker login’: denied: requested access to the resource is denied, Code: 1

When I disable buildkit (export DOCKER_BUILDKIT=0), which implies a sequential building process, everything works.

Obviously I am trying to achieve something that is not intended with BuildKit. Is there a fundamental flaw in my design or is there some configuration option for BuildKit I am missing? I don’t want to publish `servicebase´ to Dockerhub.

Thank you!

2

Since depends_on is for starting the containers and not building the images, that will not help and you should not rely on Docker Compose building any base image anyway. I would create a simple shell script that builds the base image and runs docker compose up containing only the services that actually need to run. I suppose the servicebase is only for triggering the build process not for running any container.

You could also try Multi-Stage build, which might solve your issue, but in your case I don’t find it the best solution. You could have three stages. One for the base layers (wouldn’t be tagged as an image), imageA stage for serviceA depending on the base stage, and imageB stage for serviceB depending on the base stage as well. That way I the first build could start to build the base layers, but it is also possible that two different builds would be started because at the time of starting the second, the base image layers don’t exist yet.

So I think the best is just creating a script so you can control what runs first.