Specifying buildtime envars for docker-compose with multitiple files

Docker Engine Compose
1

Hey,

I’m using git submodules & docker-compose to build a flexible structure for web applications. I basically have something like this:

[fv projects]$ tree -L 2
.
└── project
    ├── **backend**
    ├── docker-compose.yaml
    ├── .env
    ├── .gitmodules
    ├── **frontend**
    ├── README.md

So “project” is really just a git repo wrapper, containing the docker-compose file, where to fetch the git submodules and the Readme. Then 2 submodules, /backend and /frontend. The specific backend could be whatever (django, express-js, etc), same for frontend (vuejs, react, etc.). I have generalized my environment variables (contained under .env) to be fairly generic, I pass them to my service and they “just work” as services regardless of the specific backend I’m using. Works fairly well. Part of the docker-file to make this more specific:

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile
    image: ${BACKEND_IMAGE_NAME}
    container_name: backend
    env_file:
      - .runtime.env
    ports:
      - ${SERVER_PORT}:${SERVER_PORT}
    volumes:
      - type: bind
        source: ./backend
        target: /home/app/backend
      - type: volume
        source: venv_backend
        target: /home/app/backend/${DEPENDANCIES_FOLDER}
      - type: bind
        source: ./frontend/build
        target: /home/node/app/frontend/build

.... services for frontned, database container, nginx etc....

volumes:
  mongodb_data:
  mongodb_configs:
  node_modules_backend:
  node_modules_frontend:
  venv_backend:

However it really makes no sense to specify, say, what the dependency folder my my backend should be in root project, e.g. my ${DEPENDANCIES_FOLDER} variable in the compose file (.venv for python, node_modules for js, etc.). That info should be contained within each specific backend git repo. This way, I can just check a different branch from my backend submodule and that branch should “know” everything there is to know regarding the correct envars it needs. However right now it seems I only can specify the buildtime envars under /project/.env, and not spread them out under /project/.env (for the truly shared stuff), /project/backend/.env (for the backend-specific stuff), and /project/frontend/.env (For the frontend stuff).

Any ideas? Can buildtime envars be set in this manner somehow?

PRECISION: I know I can use --env-file to specific a different file/location for those variables. This doesn’t achieve the goal because:
1 - I don’t want to override /project/.env, I want to complement it with more variables, and need to have a few such files, not just one.
2 - At least ideally, the user should just have to run docker-compose [-f ...] up, and that’s it. He doesn’t need to know/bother that I keep some envars in some subfolder somewhere.

2

Why not make the container_name into a variable and then use that to point to your location?

3

The title is a little bit misleading, since you from the Docker Compose point of view you don’t want builtime environment variables (build args), only different variables for the submodules without the user knowing you there are additional variables.

--env-file as a parameter of docker compose is not the same as env_file in the yaml block of a service. --env-file is just an alternative of .env without specifying it for any service so Docker Compose could make those variables available as placeholders in the YAML file not in the containers, so it would not override anything directly inside the containers, but this is not necessary either…

What about using another docker cpmpose file in the modules.

docker-compose -f docker-compose.yml -f backend/docker-compose.yml up

If you don’t want the user to remember this long command every time they run compose command, you can create a small script

up.sh

#!/usr/bin/env bash

args=(
  --file docker-compose.yml
  --file backend/docker-compose.yml
)
exec docker-compose "${args[@]}" up

But it would be probably easier to just define the additional env file as a required file in the modules like:

    env_file:
      - .runtime.env
      - backend/runtime.env