Failure to build image FROM an image that already exists locally

I am attempting to build an image and it is failing with:

failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

This is puzzling me because the image referred to in the FROM clause is an image that already exists locally. I know this because I have just built it as I’ll demonstrate…

I have a script that builds an image that is built FROM a publicly available image and which is tagged with dataplatformbase:local. The script then builds more images that are all FROM that first image.

Here is a portion of the output from my script. Notice that the building of the first image succeeds however the building of the second image fails immediately. Notice that the image referenced in the error message is exactly the same as the one that has just been successfully built:


 => exporting to image         
 => => exporting layers        
 => => writing image sha256:e8133b7a8c49e87710ce8d3a7e63c5ac3bf6b1f0364b9d13495316c6f2b71e40
 => => naming to docker.io/library/dataplatformbase:local

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Building dataplatform image 'dataplatform:local' from /Users/joe.bloggs/path/to/dir
[+] Building 2.4s (8/8) FINISHED                                                                                                                                                                       
 => [internal] load build definition from Dockerfile
 => => transferring dockerfile: 252B
 => [internal] load .dockerignore
 => => transferring context: 34B
 => resolve image config for docker.io/docker/dockerfile:1.4
 => CACHED docker-image://docker.io/docker/dockerfile:1.4@sha256:443aab4ca21183e069e7d8b2dc68006594f40bddf1b15bbd83f5137bd93e80e2
 => [internal] load build definition from Dockerfile
 => [internal] load .dockerignore
 => ERROR [internal] load metadata for docker.io/library/dataplatformbase:local
 => [auth] library/dataplatformbase:pull token for registry-1.docker.io
------
 > [internal] load metadata for docker.io/library/dataplatformbase:local:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I find it really peculiar that the building of the second image fails given the image that its being built FROM already exists locally.

I’m hoping someone might be able to explain why this is failing because right now, I am clueless. What is it trying to authenticate to and why might it be failing?

One extra note, the first image (the one that succeeds) does not build FROM a publicly available image. It builds from an image that we host on a private registry.

I remembered I saw this issue before. It turned out I even saved it to my bookmark, but I had so many bookmarks, I forgot to go back. This is that topic

It was caused by buildkit somehow. Can you configrm that it works if you disable buildkit assuming you don’t use a syntax that supported by buildkit only?

Hi @rimelek, thank you for your response.

Actually its the other way round. For the image that built successfully we do use buildkit, for the image that failed to build we do not. Here’s a portion of the script:

echo -e "${PURPLE}Building base image '${IMAGE_AND_TAG}' from $(pwd)${NC}"
DOCKER_BUILDKIT=1 docker build --secret "id=dataplatform_pypi_jsonkeyfile,src=$PYPI_READER_KEYFILE" "$@" "$BASE_PROJ_DIR/base_image" -t "${IMAGE_AND_TAG}"

IMAGE_AND_TAG=dataplatform:"${IMAGE_TAG}"
echo -e "${PURPLE}Building dataplatform image '${IMAGE_AND_TAG}' from $(pwd)${NC}"
docker build "$@" --build-arg FROM_TAG="${IMAGE_TAG}" "$BASE_PROJ_DIR/dataplatform" -t "${IMAGE_AND_TAG}"

We have attempted to change the building of the failing image to using buildkit and got the same behaviour - it still failed.

I tried to build an image locally and use it as a base image for an other image. I tied with buildkit and without buildkit. I even tried to build with buildkit and use a base image which was built without buildkit and I tried to build without buildkit using a base image which was built with buildkit. Worked perfectly on my mac. Can you create a small example for us that we can use to reproduce the issue?

  • 1 Dockerfile for the base image
  • 1 Dockerfile for the example app
  • 1 command to build the base image
  • 1 command to build the app image

Please, use the DOCKER_BUILDKIT variable for both commands so we can make sure our default settings will not affect the result.

Maybe this error happens in special cases.

Thanks again for the reply.
I’ll do this as soon as possible, unfortunately it wont be today. The problem isn’t actually one that I am experiencing, its one that two of my colleagues are experiencing. I’ve sent them a message this morning asking them to run this script:

cd /tmp
docker login
echo "FROM ubuntu" > Dockerfile
docker build .
DOCKER_BUILDKIT=1 docker build .

which I’m confident will fail with the same problem.

Unfortunately they are both on leave today and then I am on leave from Monday for 2 weeks. Hence it might be a while until I can post a useful response here :frowning:

Hi @rimelek , thank you for helping us with this.

Just to keep the conversation going in Jamie’s absence I can confirm that my colleague and I have both tried the simple script quoted above and the build does indeed succeed.

Is there anything else we could try to help us to narrow down the issue?

Thanks in advance,

Steffan

Normally I would help you with the debugging on my machine, because I am curious, but I will have very little time in the following days.

Since it looks like you have a working and a failing script, you can try to make small changes in the working one to make it more similar to the failing one and try to remove parts from the failing script until it starts to work.

You can also check if you have the same mac version, Docker version and Docker Desktop version. I have read the topic again. It looks like wi did not talk about that. You can check the desktop settings, for example the settings of Docker Engine or network related parameters. If the difference is not in your settings then it could be in your local network or anything in your environment. I have no idea what could cause this in your environment, but I have seen strange things before :slight_smile:

Without comparing a these prameters of the working and a failing builds I don’t have more ideas.

Thanks again @rimelek . When Jamie returns from leave we will look at your suggestions and see if we can narrow the failure down.