Docker Community Forums

Share and learn in the Docker community.

How can I expand a variable within a COPY command in the Dockerfile?

I have a Dockerfile:

FROM nginx
ARG from_url
ENV REFERENCE_IMAGE_URL ${from_url}
RUN echo "Going to use reference image $REFERENCE_IMAGE_URL"
COPY --from=${REFERENCE_IMAGE_URL} /staticfiles /usr/share/nginx/html/static

I have a make target to use it:

nginx-dist:
	@echo "Using DOCKER_IMAGE_URL=$(DOCKER_IMAGE_URL)"
	@echo "Using Docker version $(shell docker --version)"
	docker build -t prior_auth_nginx:latest -f nginx.Dockerfile . --build-arg from_url=$(DOCKER_IMAGE_URL)

When I run this make target, I get the following:

Using DOCKER_IMAGE_URL=<redacted-url>:<redacted-tag>
Using Docker version Docker version 20.10.2, build 20.10.2-0ubuntu1~18.04.3
docker build -t <redacted>:latest -f nginx.Dockerfile . --build-arg from_url=<redacted-url>:<redacted-tag>
Sending build context to Docker daemon  21.73MB

Step 1/5 : FROM nginx
 ---> 08b152afcfae
Step 2/5 : ARG from_url
 ---> Using cache
 ---> 3abec8f00eec
Step 3/5 : ENV REFERENCE_IMAGE_URL ${from_url}
 ---> Running in 29487f30569c
Removing intermediate container 29487f30569c
 ---> 59020c91dc63
Step 4/5 : RUN echo "Going to use reference image $REFERENCE_IMAGE_URL"
 ---> Running in 343289f7443c
Going to use reference image <redacted-url>:<redacted-tag>
Removing intermediate container 343289f7443c
 ---> d77a60646efa
Step 5/5 : COPY --from=${REFERENCE_IMAGE_URL} /staticfiles /usr/share/nginx/html/static
invalid from flag value ${REFERENCE_IMAGE_URL}: invalid reference format: repository name must be lowercase
make: *** [nginx-dist] Error 1

Based on the documentation (Dockerfile reference | Docker Documentation), I see that COPY is supposed to accept variable expansion. As you can see from the output above, it does not. What am I doing wrong here?

The <redacted> things make it a bit hard to get the full picture. So, just to be sure, referring to the actual error message repository name must be lowercase: is the value lowercase? (Maybe it’s just that the error message does not show the substituted value, but shows the original code?)

Also, given the variable’s name, should we assume REFERENCE_IMAGE_URL is some URL including https://? If true, then I’ve never seen that being used in COPY --from (but I am no expert). I guess you tested without the variable, with the hardcoded value?

Documentation for COPY --from I see:

Optionally a name can be given to a new build stage by adding AS name to the FROM instruction. The name can be used in subsequent FROM and COPY --from=<name> instructions to refer to the image built in this stage.

…and:

Optionally COPY accepts a flag --from=<name> that can be used to set the source location to a previous build stage (created with FROM .. AS <name> ) that will be used instead of a build context sent by the user. In case a build stage with a specified name can’t be found an image with the same name is attempted to be used instead.

So, I guess Compose falls back to that very last sentence. But: should that support a URL like https://example.com/something rather than, say, simply repo-name/image-name? (I don’t know.)

Sure, the redacted bit does make that hard to troubleshoot. Thank you for pointing it out.

I can verify that the $REFERENCE_IMAGE_URL is in fact all lowercase in the echo statement (4/5).

Also, given the variable’s name, should we assume REFERENCE_IMAGE_URL is some URL including https://?

It is a URL, with a tag at the end of it, but without https:// at the front. It is the equivalent of some.aws.account.url/organization/application:tag (without protocol at the beginning).

Right?

If not:

I guess that should be okay:

(Especially as, if the above code is used, the actual error message is raised because the lower-cased value does match the above, whereas the original value does not. Hence some uppercase is present in whatever value is validated.)

So, indeed maybe no variable substitution is taking place. On the other hand, the $, { and } from ${REFERENCE_IMAGE_URL} should be rejected, even when lowercased? :thinking:

I feel like I did something like make the variable name lowercased, but it just gave me the same error. The way it echos “invalid from flag value ${REFERENCE_IMAGE_URL}: invalid reference format: repository name must be lowercase” suggests to me that it’s not being processed and replace with the relevant variable. I’m really not sure here.

I can verify that it does not have anything uppercase in it, though for proprietary info purposes I cannot put the relevant URL there.