Docker-compose create multiple instances

General Discussions
1

Is there any way to create multiple instances for the same image?
more specifically, I want to have multiple instances with different container_name

my original thought is using environment variable substitute:
let’s say I have a service called app in docker-composel.yml

app:
    image: app:latest
    ports:
      - "48100:48100"
    container_name: app-${APP_NAME}
    hostname: app-${APP_NAME}
    entrypoint:
      - /app

and I have a Go program:

for i := 0; i < len(apps); i++ {
    _ = os.Setenv("APP_NAME", apps[i])
    cmd := exec.Command("docker-compose", []string{"up", "-d", "app"})
    cmd.Run()
}

but when I ran this program only the first app name will be applied, after that docker-compose will recreate the same container again

I also noticed that there’s scale option, but it seems cannot customize the container_name ?

thanks for the help

2

Scale is used for replicas of the same instance.

In docker compose a service is identified by project name and service name. Since you change neither of both, you replace the existing service. You need to change at least the project name as well. Changing the service name (which is app in your case) is not suffiecient, as compose will complain about orphaned services. By default the project name is the folder name.

add "--project-name", app[i] to your line:

    cmd := exec.Command("docker-compose", []string{ "--project-name", app[i], "up", "-d", "app"})

edit: also, you can not publish a static port and reuse the same port on all instances.

3

thanks for the clarify, in my case I don’t think I can’t change the service name otherwise docker compose don’t know which image to use right ?

Changing project name seems a great idea, definitely gonna give it a try (I almost gave up and went back to use docker run ...... with lots of option)

4

Not sure how you come to this conclusion, but image: is repsonsible to determine the image. The service name you used is app. Though, service names (the node underneath ‘services:’ ) are static in a docker-compose.yml and can not be replaced with variables.

If you want to be able to use variables everywhere in the docker-compose.yml, you might want about using go templating or envsubst in the simplest case. You can render your docker-compose.yml and pipe it into docker-compose like this: envsubst < docker-compose.template | docker-compose -f - {more options} up -d. You need to export variables in order to make envsubst render them in memory for the file docker-compose.template.

Does that make sense?

5

sorry for the bad English but that’s what I try to express: I cannot dynamically change the service name.
And since changing image name won’t work ( I’ve tested it), I think the only way might be dynamically change the project name that you mentioned earlier.

6

With what is buildin in docker: yes, the project name is your only option.

Though, my suggestion to use envsubst goes beyond what is build in into docker. You can archive simple templating (without any conditional rendering) with it without fighting with complexity :wink:

7

is it possible to use the pre-existing volumes/network if I’m going to dynamically change the project name instead of creating projectname_volume/network everytime?

EDIT:
it seems I can’t achieve this if I didn’t separate my app service to another compose file?
let’s say my original docker-compose.yml is like:

verison: '3.7'
volumes:
  data:
  other-data:

services:
  core-services:
    ....

  app:
    image: app:latest
    ports:
      - "48100:48100"
    container_name: app-${APP_NAME}
    hostname: app-${APP_NAME}
    networks:
      - core-network
    entrypoint:
      - /app

networks:
  core-network:
    driver: "bridge"
8

Can be done, though In the absence of “the big picture”, responding to your individual questions is the best I can do…

You can create networks and volume outside the lifecycle of docker-compose.

docker create network core-network

Reference the network in your compose.yml:

networks:
  core-network:
    external: true

The same can be done for volumes.

1 Like
9

sorry for making it so unclear …
I eventually separate my app to another compose file and make it join the pre-existing network.

networks:
  default:
    external:
      name: core_compose_network

with this design at least I can achieve what I want, thanks for your help