Is there a way to make `run` override `scale`/`replica`?

General
1

Hi Folks,

I am trying to use docker compose run to override the replica setting in the docker-compose.yml settings.

I have docker compose configurations along the lines of:

services:
  frontend:
    ...
    depends_on: backend
  backend:
    ...
  e2e:
    ...
    depends_on: e2e-frontend
    deploy:
      replicas: 0
  e2e-frontend:
    ...
    depends_on: backend
    deploy:
      replicas: 0

Full Source:

With these configurations, I can issue docker compose up and have frontend and backend running during normal development.

When I want to run the end-to-end (e2e) test suite, I can issue:

docker compose up \
  --scale e2e=1 \
  --scale e2e-frontend=1 \
  e2e

My problem comes when I’m trying to run a single e2e test with something along the lines of:

docker compose run e2e \
  --scale e2e=1 \
  --scale e2e-frontend=1 \
  --spec my/single/test.js

Docker compose run does not recognize the scale flag.

I tried uping first:

docker compose up \
  --scale e2e-frontend=1 \
  e2e-frontend

and then runing:

docker compose run e2e \
  --spec my/single/test.js

but run stops/exits the running e2e-frontend container (probably because of the replica setting?).

I think my workaround options are:

  • override the e2e/e2e-frontend service command when using up (could not figure out how to do this)
  • override the e2e/e2e-frontend service replica when using run (could not get this working)
  • create a separate .yml file that overrides the replica setting and append it using the --file flag (want to avoid needing another .yml file)

Is there a way to accomplish what I want with some magical run or up CLI flag?

Thank you for your time :pray:

2

docker compose run is for running a command in only one service (and optinally starting its dependencies). You can run an interactive bash shell. In that case in which instance would you run the command? Scaling with run doesn’t really make sense. But because it doesn’t make sense, you don’t need it. You can run a service even if you set replicas to 0 (Actually I didn’t know you could do that). And yet… it seems it affects the dependencies. I don’t know if it is a documented and intended behavior, but it is possible that you need multiple instances of a dependency to run only one command in an other service container. For example you want to test load balancing.

You can create an other compose file like docker-compose.run.yml or just run.yml

services:
  e2e-frontend:
    deploy:
      replicas: 1

and use this command to run:

docker compose \
  -f docker-compose.yml \
  -f docker-compose.override.yml \
  -f run.yml \
  run --rm e2e --spec my/single/test.js

You have to pass all of your docker files in the correct order because if you specify a compose file the override yml will not be used automatically. I also used --rm after run because otherwise each run would leave stopped containers behind. If you want, you can create a shell script to make it easier to run.

3

Have you considered to use --profile to start a set of services, depending on the profile?
I am not sure whether the docker compose command actualy supports the --profiles argument, but good old docker-composedoes: Using profiles with Compose | Docker Documentation,

4

That is much better than I recommended! I always forget about the profiles. It works with Docker Compose v2.

5

Good to know.

Now the question is, if profiles are a solution for what the OP was asking for. At least it appears like what the OP actualy needs.

6

Thank you @meyay; the profile feature is exactly what I was looking for. It is much nicer than the other options I was considering. I just implemented it and it works perfectly. Thank you again :pray:

Thank you @rimelek also for your input. It’s probably a moot point, but I was trying to run a different command on the e2e service/container.

Thanks again, folks :tada: