Docker Community Forums

Share and learn in the Docker community.

Docker Cloud build environment variables not being passed to the auto-build


(Tom Kazakov) #1

Hi guys,

Our project is being built by Docker CLoud autobuild CI and I need to pass a few ENVs to the build sequence, so the public assest will be deployed on the AWS S3.

We have a few environments (devs/test/production) so I need to specify not only the AWS creds but also the specific bucket and folder which the files should be put to.

There is a build-wise ENV section on the auto-build page, but

  1. After altering the variables the get hidden and reappear only after the page reload (it must be a UI bug).
  2. The values don’t seem to be injected in the build system environment (I’ve checked via creating a variable and trying RUN touch $VARNAME / RUN mkdir $VARNAME / RUN echo $VARNAME.

What am I doing wrong? Are the variables besides mentioned in https://docs.docker.com/docker-cloud/builds/advanced/ even allowed in the build process?

What other secure options do I have? I have a couple of builder nodes and at 6 Docker Hub registries being built regularly (all registries are environment-specific, but builder nods are not).

Thanks and have a nice day!
Tom


Docker Cloud - Automated build environment variables not being set
(Pablo Chico de Guzman Huerta) #2

Hi Tom,

Thanks for using Docker Cloud CI!

Environment variables are not directly accessible from your Dockerfile instructions, they are only accessible from hooks. This is because you might have a secret as an environment variable but you don’t want to use it as a build-arg because build-args are stored on the image layers.
In ooder to make environment variables available from the Dockerfile, you can create a “hooks/build” hook like this:

    #!/bin/bash
    docker build --build-arg VAR=$VAR -t $IMAGE_NAME

You could also use a “hooks/post_push” hook to upload your artifacts to S3 if necessary.
For more information about hooks, refer here https://docs.docker.com/docker-cloud/builds/advanced/#/custom-build-phase-hooks


(Tom Kazakov) #3

Hi Pablo, thank you very much for the reply, it’s all much clearer now!

Cheers!


(Dkoo761) #4

Hi Pablo,

I need to determine, within one of my Dockerfiles, which tag triggered the build in Docker Cloud since I use the same Dockerfile for a couple different Docker Cloud build rules and have a bit of conditional logic in that Dockerfile to handle building the image for each tag slightly differently.

I’ve been trying to use a custom build hook to do this, but we have a number of build rules configured for this repo and each one could have a different value for the following fields in the Docker Cloud automated builds UI:

  • source (branch or tag)
  • Docker tag
  • Dockerfile location
  • Build context

I know there are env vars provided by Docker Cloud that give us the first 2 of these 4 values: SOURCE_BRANCH and CACHE_TAG. But I’m wondering how I would be able to choose the correct Dockerfile and Build context to pass onto my custom “docker build” command?

For example, right now I have this in my hooks/build file (leaving “source” out of this example):

#!/bin/bash
docker build --build-arg CACHE_TAG=$CACHE_TAG -t $IMAGE_NAME .

But the problem is that this is always going to use the Dockerfile filename and . as the build context which differs from most of the build rules we have for this repo. I could add extra logic to the build file to determine the Dockerfile and build context from the cache tag but then I’m basically duplicating half of the data from the Docker Cloud automated builds UI and hardcoding it into the build file which is not desired.


(Levilugato) #5

we have the same problem here, when we make de build hook ours COPY for our dockefile dont make sense because we dont know where are our build context, running an ls on Dockerfile we could see only archives like gradle, lib, share, buit we need to see our archives of our aplication


(Deanunnotech) #6

It would be clearly convenient if no hooks are needed and just clearly set the environment variable. With the DockerCloud UI, it really is hard to tell if hooks are needed and first impression would be just setting in the ENVIRONMENT VARIABLES on the CONFIGURE AUTOMATED BUILDS page will be enough to act as build arguments


(George Katsanos) #7

Exactly what @deanunnotech said.
I see “Environment variables” in the web UI, I set them one by one, and hours laters of trying I find this topic.
What are the environment variables in the UI supposed to be then?


(George Katsanos) #8

could you please provide a more complete example on how to pass secret environment variables during the build process? (it is an advanced topic but definitely necessary to use even for the newbies like us in docker).
The documentation is although long, somehow lacking.


(Deanunnotech) #9

One more thing, it will be helpful if each tag can have it’s own environment variable


(Emilmoe) #10

Did you find a solution? I have no clue too what the purpose and use is


(Inishant) #11

Its probably the most misleading name for a feature. Adding variables from the auto-build’s web ui makes them available inside the hooks. In the hook, you’ll have to use that value to set a custom build arg using --build-arg. Finally you have to use this custom build arg inside your dockerfile to manually set an environment variable using ENV command or export.

Example:
Say your want an environment variable FOO=SECRET in your build environment
Step 1.
Add this in the auto-build’s web ui for ‘build environment variables’
Step 2.
Create a custom build hook i.e create a folder called hooks in the same directory as your dockerfile. Inside the hooks folder, create a file called build. This creates the custom build hook. Docker will use this to build your image. Contents of build:

#!/bin/bash

docker build -t $IMAGE_NAME . --build-arg FOO=$FOO

Here $FOO is coming from the web ui.
Step3:
In your dockerfile

ARG FOO
RUN export FOO=$FOO

Here $FOO is coming from the custom build args.

Thats it! It should work now. Honestly they should just rename ‘build environment variables’ to ‘custom hook environment variables’.