Enable specific config conditionally with docker compose profiles?

General
1

I have a fairly comprehensive docker-compose setup with multiple services.
I’d like to add OpenTelemetry support and enable it via a profile, e.g. monitor.

For the OpenTelemetry Collector, this is straightforward, I can just define the collector service with:

services:
  otel-collector:
    image: grafana/otel-lgtm
    profiles:
      - monitor

The tricky part is enabling OpenTelemetry inside the existing services.

By default, tracing is disabled in those services and requires certain environment variables (e.g. OTEL_* vars) to turn it on. I only want these variables to be active when the monitor profile is enabled.

One possible solution is using variable interpolation like this:

environment:
  OTEL_ENABLED: ${OTEL_ENABLED:-false}

However, this requires setting additional environment variables externally whenever I enable the monitor profile, which feels clunky.

Is there a cleaner way to conditionally inject environment variables into existing services based on a profile? Or Is there a recommended pattern for this use case?

2

Profiles are like flags for compose services to tell whihc service you want to start only when a specific profile is set, so you will still need some kind of template system to generate a compose file and sets the env variable based on the same parameter that is used for choosing the profile.

If you want to use only compose, I would use multiple compose files.

  • compsoe.yml: Main compose file with the services that always have to start
  • compose.monitor.yml: containing the services for monitoring and also overriding environment variables in other services.

Than you could start it like:

docker compose -f compose.yml -f compose.monitor.yml up -d

Just in case you think it is too long and you can make mistakes, you can create a small script if you want and add the monitor compose yaml only when you pass “-m”. It would look like:

./run.sh up -m -d
./run.sh down -m

Then if you are afraid that you would accidentally run the compose command without a script, you could set an env variable in the shell script like

export RUN_IN_SCRIPT=1

and add this line to the beginning of the compose file:

x-run-in-script: ${RUN_IN_SCRIPT:?This compose file can only be run by using run.sh}

So it fails when the env var is not set. Which is set only in the script.

If you need help I can share a small example script later, but it would work only on Linux or macOS or if you are using WSL2 integration on Windows.

1 Like