Docker Community Forums

Share and learn in the Docker community.

How does --cache-from work with multiple images?

Hello,

I have few questions concerning --cache-from docker build argument. Maybe someone will be able to help.

#1. How to pass multiple images?
docker build --cache-from first-image --cache-from second-image
or
docker build --cache-from first-image second-image

#2. How does providing multiple cache-from images work? Does it take first image that is found (argument order matters)? Or does it pick the one with most cached layers?

#3. I’m trying to use it in CI, with multi-stage build…

Let’s assume we have a Dockerfile with build step ( FROM ... as builder ) and setup step ( FROM ...-alpine ).

Also, let’s assume we use git-flow and we want to use develop branch and latest tag as base cache images.

first step:

docker build \
    --target builder \
    --cache-from some-image-builder:$GIT_REF \
    --cache-from some-image-builder:develop \
    --cache-from some-image-builder:latest

second step:

docker build \
    --cache-from some-image-builder:$GIT_REF \
    --cache-from some-image-builder:develop \
    --cache-from some-image-builder:latest \
    --cache-from some-image:$GIT_REF \
    --cache-from some-image:develop \
    --cache-from some-image:latest

But that seems like a lot of images. Is this right way to do it? :thinking:

Btw, I’ve also created github issue (with similar questions), to clarify the docs, here:
https://github.com/docker/docker.github.io/issues/8531

After some experiments and reading Andrew Lock’s article it seems that all images are used. However you have to make sure that they exist on local host. Andrew suggests to pull them explicitly in hosted CI/CD scenarios and I have adopted this approach.

@paulmichalik Thanks for the answer. I do pull all the images in my CI/CD pipeline and it seems to work.

it seems that all images are used. However you have to make sure that they exist on local host.

It is kind of a bummer, if true. I want to use this in a build pipeline, where the upstreams might not all be ready yet. Some of them will be composed as a part of this build. From the looks of things you’re perhaps wrong, at least you don’t need to pull (unless you really want that to come from cache, which is exactly what I wanted), I just used cache-from directives, some of which didn’t exist yet, and docker build didn’t balk at it. I count that as a win here, thanks for your help.

The following change in my buildspec.yml worked, which should have a ready DOCKER_UPSTREAM but may not have populated BRANCH_TAG_IMAGE_URI yet:

-                - docker build --tag "$BRANCH_TAG_IMAGE_URI" .
+                - docker build --cache-from "$DOCKER_UPSTREAM" --cache-from "$BRANCH_TAG_IMAGE_URI" --tag "$BRANCH_TAG_IMAGE_URI" .

I did make sure I had called docker pull first as you suggested, it seemed to invoke “Using cache” in all of the expected places, :+1: but of course my build machine is not guaranteed or likely to repeat itself on the same host, unless that was a provision of CodePipeline that I was unaware of before.