Enable specific config conditionally with docker compose profiles?

General
1

I have a fairly comprehensive docker-compose setup with multiple services.
I’d like to add OpenTelemetry support and enable it via a profile, e.g. monitor.

For the OpenTelemetry Collector, this is straightforward, I can just define the collector service with:

services:
  otel-collector:
    image: grafana/otel-lgtm
    profiles:
      - monitor

The tricky part is enabling OpenTelemetry inside the existing services.

By default, tracing is disabled in those services and requires certain environment variables (e.g. OTEL_* vars) to turn it on. I only want these variables to be active when the monitor profile is enabled.

One possible solution is using variable interpolation like this:

environment:
  OTEL_ENABLED: ${OTEL_ENABLED:-false}

However, this requires setting additional environment variables externally whenever I enable the monitor profile, which feels clunky.

Is there a cleaner way to conditionally inject environment variables into existing services based on a profile? Or Is there a recommended pattern for this use case?

2

Profiles are like flags for compose services to tell whihc service you want to start only when a specific profile is set, so you will still need some kind of template system to generate a compose file and sets the env variable based on the same parameter that is used for choosing the profile.

If you want to use only compose, I would use multiple compose files.

  • compsoe.yml: Main compose file with the services that always have to start
  • compose.monitor.yml: containing the services for monitoring and also overriding environment variables in other services.

Than you could start it like:

docker compose -f compose.yml -f compose.monitor.yml up -d

Just in case you think it is too long and you can make mistakes, you can create a small script if you want and add the monitor compose yaml only when you pass “-m”. It would look like:

./run.sh up -m -d
./run.sh down -m

Then if you are afraid that you would accidentally run the compose command without a script, you could set an env variable in the shell script like

export RUN_IN_SCRIPT=1

and add this line to the beginning of the compose file:

x-run-in-script: ${RUN_IN_SCRIPT:?This compose file can only be run by using run.sh}

So it fails when the env var is not set. Which is set only in the script.

If you need help I can share a small example script later, but it would work only on Linux or macOS or if you are using WSL2 integration on Windows.

1 Like
4

Was doing the same for recent project (not specific to OTel). But want to share that you may need to check for orphan containers when running with profiles that may require a clean-up. You only will get a WARN on this (again not an error but for avoiding to see the message everytime)

5

The --remove-orphans argument should take care of it.