Yaml and optimizing compose files

[batjko]

Hi everyone.

I am wondering what everyone’s approach is to making compose files well-formatted, re-usable and optimally configured - also in regard to individual properties and values of service definitions.

For example:
I have a placement constraint on some of my services, to avoid specific worker nodes, e.g.:

placement:
   constraints:
     - node.role == worker
     - node.hostname != worker3
     - node.hostname != worker4

But I couldn’t find anywhere in the documentation if that’s the best way of defining multiple values for the hostname constraint. Could I use comma-separated values instead of one line for each?

Or should I define an avoidWorkers array at the top of my file and then somehow refer to that in the constraint?

More generally, how much yaml cleverness do you use in your compose files, and what do you regard as best practices when setting configurations?

[dmaze (David Maze)]

@batjko

But I couldn’t find anywhere in the documentation if that’s the best way of defining multiple values for the hostname constraint. Could I use comma-separated values instead of one line for each?

Yes (YAML is extremely flexible this way); it would look like

constraints: [node.role == worker, node.hostname != worker3, node.hostname != worker4]

My loose experience from other contexts that use YAML (Kubernetes, Ansible) is that the inline form is much easier to read and write if you have single words, but worse if things run over a line; for this example I’d definitely use the block form as you have it originally.

Or should I define an avoidWorkers array at the top of my file and then somehow refer to that in the constraint?

I’ve only rarely seen the YAML reference syntax used at all and I’d probably avoid it in practice, even if you are repeating the same thing over and over. (You can, but nobody will recognize what the * and & are doing there.)

More generally, how much yaml cleverness do you use in your compose files, and what do you regard as best practices when setting configurations?

In Kubernetes land the general practice is for the YAML files to not be clever at all, but to use some sort of external templating system to create them. The Helm tool uses Go’s templating system, for instance. There you can get arbitrarily clever, with a usability/maintainability tradeoff. I’m not immediately aware of anything equivalent for Docker Compose, though I’m sure it exists.

[batjko]

@dmaze

…use some sort of external templating system to create them. The Helm tool uses Go’s templating system, for instance.

That makes sense. Keeps the config files standardized as well.

We are considering a switch away from Swarm to K8s as well, so looks like I’ll check out Helm then. Thanks for the tip!

[kinghuang (King Chung Huang)]

I love using YAML anchors and aliases (& and *) and use them extensively in my Compose files. You can use them in conjunction with extension fields, introduced in file format 3.4.

I wrote a blog post a while back about this: Don’t Repeat Yourself with Anchors, Aliases and Extensions in Docker Compose Files

And, don’t forget you can combine multiple Compose files together. Use a .env file to set a project name (COMPOSE_PROJECT_NAME) and specify the Compose files (COMPOSE_FILE) for your project. See Declare default environment variables in file for details.

[batjko]

Cool introduction, @kinghuang.

I don’t see why one wouldn’t use this kind of stuff. It takes a bit of getting used to, but clearly reduces the potential for a lot of copy/paste errors.

Thanks for the tip!