Using extension fields in docker compose yaml gives error "MISSING_CHAR" at volumes block

Docker Engine Compose
1
  • Issue type:
    I’m trying to put all my services in 1 compose file with anchors/extension fileds.
    However I get the following error in my volumes block:
    volumes:
      <<: *common-services-
      - ${DOCKER_DIR}/appdata/service_1:/config
      - ${MY_DIR}/sub_dir_3:/container_dir_3
      - ${MY_DIR}/sub_dir_4:/container_dir_4

ERROR:

A block sequence may not be used as an implicit map keyYAML
Implicit keys need to be on a single lineYAML
Implicit map keys need to be followed by map valuesYAML
Implicit map keys need to be followed by map values at line 69, column 7:

  <<: *common-services-dir
  - ${DOCKER_DIR}/appdata/service_1:/config
  ^

(MISSING_CHAR)

VSCode doesn’t give me any solution and I can’t find any info to resolve it online.
I think it has something to do with the dash “-” for listing the several volumes. But can’t find a solution.

  • OS Version/build:
    Ubuntu Server 22.04

  • App version:
    Docker: 20.10.17
    Docker Compose version v2.6.0

  • Steps to reproduce:

docker-compose.yml

##################
#### NETWORKS ####
##################
networks:
  proxy:
    external: true

####################
#### EXTENSIONS ####
####################
x-environment: &default-tz-puid-pgid
  TZ: ${TZ}
  PUID: ${PUID}
  PGID: ${PGID}

# Commmon .env-file
x-env-file: &services-env-file
  env_file:
    - .env

# Common services directories
x-services-volumes: &common-services-dir
      - ${MY_DIR}/sub_dir_1:/container_dir_1
      - ${MY_DIR}/sub_dir_2:/container_dir_2

# Common network, security_opt, restart, profile
x-common-keys-services: &common-keys-services
  networks:
    - proxy
  security_opt:
    - no-new-privileges:true
  restart: unless-stopped
  profiles:
    - service_profile

##################
#### SERVICES ####
##################
services:

###### service_1 ######
  service_1:
    image: ghcr.io/linuxserver/service_1
    container_name: service_1
    hostname: service_1
    # extensions (.env-file | network & security-opt)
    <<: [ *services-env-file, *common-keys-services ]
    environment:
      <<: *default-tz-puid-pgid
      UMASK_SET: 022 #optional
    volumes:
      <<: *common-services-dir
      - ${DOCKER_DIR}/appdata/service_1:/config
      - ${MY_DIR}/sub_dir_3:/container_dir_3
      - ${MY_DIR}/sub_dir_4:/container_dir_4
    ports:
      - 8080:80
    restart: unless-stopped

.env

TZ=Europe/Berlin
PUID=1000
PGID=1000
DOCKER_DIR=/home/user/docker
MY_DIR=/home/user/my_dir
2

I finally found the sollution. (thanks to this post)

CHANGES MADE:

Extensions block

  • x-services-volume: → put alias on new line with a dash “-” in front and removed dashes “-” from volumes listiing
x-services-volumes:
  - &common-services-dir
    ${MY_DIR}/sub_dir_1:/container_dir_1
    ${MY_DIR}/sub_dir_2:/container_dir_2
  • x-common-keys-services → added volumes block
x-common-keys-services: &common-keys-services
  networks:
    - proxy
  security_opt:
    - no-new-privileges:true
  restart: unless-stopped
  profiles:
    - service_profile
  volumes:
    - *common-services-dir

Services block

  • volumes: → don’t use extended anchor field (<<: *) and put quotes " " around the alias (to avoid “too many colons” error)
services:
  app_1:
    image: app_1:latest
    ---
    volumes:
      - "*common-services-dir"
      - ${DOCKER_DIR}/appdata/app_1:/config
      - ${MY_DIR}/sub_dir_3:/container_dir_3
      - ${MY_DIR}/sub_dir_4:/container_dir_4

Final dock-compose.yml:

##################
#### NETWORKS ####
##################
networks:
  proxy:
    external: true

####################
#### EXTENSIONS ####
####################
x-environment: &default-tz-puid-pgid
  TZ: ${TZ}
  PUID: ${PUID}
  PGID: ${PGID}

# Commmon .env-file
x-env-file: &services-env-file
  env_file:
    - .env

# Common services directories
x-services-volumes:
  - &common-services-dir
    ${MY_DIR}/sub_dir_1:/container_dir_1
    ${MY_DIR}/sub_dir_2:/container_dir_2

# Common network, security_opt, restart, profile
x-common-keys-services: &common-keys-services
  networks:
    - proxy
  security_opt:
    - no-new-privileges:true
  restart: unless-stopped
  profiles:
    - service_profile
  volumes:
    - *common-services-dir

##################
#### SERVICES ####
##################
services:

###### app_1 ######
  app_1:
    image: app_1:latest
    container_name: app_1
    hostname: app_1
    # extensions (.env-file | network & security-opt)
    <<: [ *services-env-file, *common-keys-services ]
    environment:
      <<: *default-tz-puid-pgid
      UMASK_SET: 022 #optional
    volumes:
      - "*common-services-dir"
      - ${DOCKER_DIR}/appdata/app_1:/config
      - ${MY_DIR}/sub_dir_3:/container_dir_3
      - ${MY_DIR}/sub_dir_4:/container_dir_4
    ports:
      - 8080:80
    restart: unless-stopped

.env

TZ=Europe/Berlin
PUID=1000
PGID=1000
DOCKER_DIR=/home/user/docker
MY_DIR=/home/user/my_dir
1 Like
3

Thank you for sharing your solution.

I ment to respond to your post, but was too buisy to respond and then forgot about it.

Back then I would have said: switch to the long syntax, because yaml anchors can not be used to merge lists, but then I stumbled accross accross the solution while researching something else. So yes, your solution is the way to go!

4

Hi @meyay ,

My solution actually doesn’t seem to work with multiple services.
I’m getting this error:

Error response from daemon: invalid mount config for type “volume”: invalid mount path: ‘*common-services-dir’ mount path must be absolute

Only app_3 is created and started, app_1 & app_2 are ignored.

Do you know any solution?

Thanks a lot!

dock-compose.yml:

##################
#### NETWORKS ####
##################
networks:
  proxy:
    external: true

####################
#### EXTENSIONS ####
####################
x-environment: &default-tz-puid-pgid
  TZ: ${TZ}
  PUID: ${PUID}
  PGID: ${PGID}

# Commmon .env-file
x-env-file: &services-env-file
  env_file:
    - .env

# Common services directories
x-services-volumes:
  - &common-services-dir
    ${MY_DIR}/sub_dir_1:/container_dir_1
    ${MY_DIR}/sub_dir_2:/container_dir_2

# Common network, security_opt, restart, profile
x-common-keys-services: &common-keys-services
  networks:
    - proxy
  security_opt:
    - no-new-privileges:true
  restart: unless-stopped
  profiles:
    - service_profile
  volumes:
    - *common-services-dir

##################
#### SERVICES ####
##################
services:

###### app_1 ######
  app_1:
    image: app_1:latest
    container_name: app_1
    hostname: app_1
    <<: [ *services-env-file, *common-keys-services ]
    environment:
      <<: *default-tz-puid-pgid
      UMASK_SET: 022
    volumes:
      - "*common-services-dir"
      - ${DOCKER_DIR}/appdata/app_1:/config
      - ${MY_DIR}/sub_dir_3:/container_dir_3
      - ${MY_DIR}/sub_dir_4:/container_dir_4
    ports:
      - 8080:80
    restart: unless-stopped

###### app_2 ######
  app_2:
    image: app_2:latest
    container_name: app_2
    hostname: app_2
    <<: [ *services-env-file, *common-keys-services ]
    environment:
      <<: *default-tz-puid-pgid
      UMASK_SET: 022
    volumes:
      - "*common-services-dir"
      - ${DOCKER_DIR}/appdata/app_2:/config
      - ${MY_DIR}/sub_dir_5:/container_dir_5
      - ${MY_DIR}/sub_dir_6:/container_dir_6
    ports:
      - 8433:433
    restart: unless-stopped

###### app_3 ######
  app_3:
    image: app_3:latest
    container_name: app_3
    hostname: app_3
    <<: [ *services-env-file, *common-keys-services ]
    environment:
      <<: *default-tz-puid-pgid
      UMASK_SET: 022
    volumes:
      # - "*common-services-dir"
      - ${DOCKER_DIR}/appdata/app_3:/config
      - ${MY_DIR}/sub_dir_7:/container_dir_7
      - ${MY_DIR}/sub_dir_8:/container_dir_8
    ports:
      - 8888:80
    restart: unless-stopped

.env

TZ=Europe/Berlin
PUID=1000
PGID=1000
DOCKER_DIR=/home/user/docker
MY_DIR=/home/user/my_dir
5

That post mentions only one volume definition. If you want more, you need more anchors. This is actually a YAML feature, Docker Compose just lets you use additional keys with the perfix “x-”, otherwise it would be an invalid compose file after the anchors are interpreted.

This will not work:

x-services-volumes:
  - &common-services-dir
    ${MY_DIR}/sub_dir_1:/container_dir_1
    ${MY_DIR}/sub_dir_2:/container_dir_2

becuse this is the same as:

x-services-volumes:
  - &common-services-dir ${MY_DIR}/sub_dir_1:/container_dir_1 ${MY_DIR}/sub_dir_2:/container_dir_2

so you will have only one invalid volume definition.

This just makes the “too many colons” message go away, but it will not be a valid volume. This would be valid:

x-services-volumes:
  - &common-services-dir1
    ${MY_DIR}/sub_dir_1:/container_dir_1
  - &common-services-dir2
    ${MY_DIR}/sub_dir_2:/container_dir_2

    volumes:
      - *common-services-dir1
      - *common-services-dir2
      - ${DOCKER_DIR}/appdata/app_2:/config
      - ${MY_DIR}/sub_dir_5:/container_dir_5
      - ${MY_DIR}/sub_dir_6:/container_dir_6

but it would not help you a lot. If you want to merge lists, you need a template system to generate the final yaml

6

Seems I was too optimistic and merging sequences doesn’t work after all.

Back to the original suggestion I would have given: use the long syntax for volumes and you will be just fine. The long synax is more descriptive anyways and exists for ports, configs and secrets as well. Even labels and environments can be expressed as labels. If those are used the anchors and aliases should work just fine.

Update: @rimelek reminded me that the original problem still remains: even the long syntax of volumes (and ports, configs and secrets) use a sequence.