Templating Confg/Secrets

Open Source Projects Compose
1

Hi,

I’m trying to make scaling under Swarm as simple as possible and need a service to load a unique private key based on it’s task/id number. I’ve tried the following but it doesn’t seem like compose supports it:

version: '3.8'
services:
  myapp:
    image: 127.0.0.1:6666/myapp
    build: ./myapp
    secrets:
      - source: private_key_{{.Task.Slot}}
        target: private_key

secrets:
  private_key_1:
    name: private_key_1
    file: ./keys/private_key_1
  private_key_2:
    name: private_key_2
    file: ./keys/private_key_2
  private_key_3:
    name: private_key_3
    file: ./keys/private_key_3

private_key_{{.Task.Slot}} does not get variable substitution applied.

Is there a way to achieve this?

2

Nope, template vars only apply to envs, hostnames and volumes.

I assume your replicas scale out instances of the same service and require the private keys for something like mutual tls auth between members of the same replicaset? The compose reference v3.8 introduced (content) templating for configs and secrets, which doesn’t solve your problem either.

3

I realised shortly after there is limited support for templates vars. Is there a reason it doesn’t support templating the entire compose file?
Yeah more or less. I need to have some unique information per service such a private keys and some minor configuration variables.
I tried going down the templated config/secrets path but also realised it doesn’t solve what I need.
How would this normally be done? I’m starting to look at using Vault instead and have the service pull the info from that at run time based on the Task.Slot being passed in an environment var.

4

Now this is kind of confusing. Replias are ment to scale out instances per service. Replicas are not ment to be used as seperate services with distinct state. Even though some services are ment to be used with multiple replicas (on application/service level), it is better to still seperate them into distinct services to gain better control of each replica’s configuration (some candidates: etcd3/zookeper/kafka/consul).

Appart from that: the approach to leverage Task.Slot as env and use it to pull distinct specific details per replica from a remote store sounds about right (without knowing your exact use case)

5

Thanks for the input. I think I’ll go down the Vault path. A little frustrating as this could be done easily with compose if it didn’t restrict what it did variable expansion on.