Docker Community Forums

Share and learn in the Docker community.

Need allow docker to create folder if it no existed in source

Hello all. I heave a maybe strange request but i thinks it` important.

1st. We have docker swarm
2nd all nodes connected to /external/efs/ as nfs share

here is code:

version: "3.9"

networks:
  traefiknet:
    attachable: true
    external: true
  project_tpl_network:
    attachable: true
    external: true

services:

  rabbitmq:
    image: lovelysystems/rabbitmq-swarm-cluster
    hostname: "{{.Service.Name}}.{{.Task.Slot}}.{{.Task.ID}}"
    deploy:
      replicas: 3
      restart_policy:
        condition: any
        delay: 5s
      update_config:
        parallelism: 1
        delay: 10s
        failure_action: rollback
        monitor: 1s
#        max_failure_ratio: 1
        order: start-first
      labels:
        - traefik.enable=true
        - traefik.docker.lbswarm=true
        - traefik.docker.network=traefiknet
        - traefik.http.routers.project_tpl-rabbitmq.rule=Host(`rabbitmqdomain_name_tpl`)
        - traefik.http.routers.project_tpl-rabbitmq.tls=true
        - traefik.http.routers.project_tpl-rabbitmq.tls.certresolver=le
        - traefik.http.routers.project_tpl-rabbitmq.entrypoints=websecure
        - traefik.http.routers.project_tpl-rabbitmq.middlewares=project_tpl-rabbitmq_whitelist
#        - traefik.http.routers.project_tpl-rabbitmq.entrypoints=web
        - traefik.http.routers.project_tpl-rabbitmq.service=project_tpl-rabbitmq
        - traefik.http.services.project_tpl-rabbitmq.loadbalancer.server.port=15672
        - traefik.http.services.project_tpl-rabbitmq.loadbalancer.server.scheme=http
        - traefik.http.services.project_tpl-rabbitmq.loadbalancer.passhostheader=true
        - traefik.http.services.project_tpl-rabbitmq.loadbalancer.sticky=true
        - "traefik.http.middlewares.project_tpl-rabbitmq.headers.accesscontrolallowmethods=GET,OPTIONS,PUT"
        - "traefik.http.middlewares.project_tpl-rabbitmq.headers.accesscontrolalloworiginlist=*"
        - "traefik.http.middlewares.project_tpl-rabbitmq.headers.accesscontrolmaxage=100"
        - "traefik.http.middlewares.project_tpl-rabbitmq.headers.addvaryheader=true"
        - "traefik.http.middlewares.project_tpl-rabbitmq_whitelist.ipwhitelist.sourcerange=127.0.0.1/32, 0.0.0.0/0"
    environment:
      - RABBITMQ_ERLANG_COOKIE=abc
#      - RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="-setcookie abc"
      - RABBITMQ_USE_LONGNAME=true
      - RABBITMQ_MNESIA_DIR=/var/lib/rabbitmq/mnesia
#      - RABBITMQ_PLUGINS_EXPAND_DIR=/var/lib/rabbitmq/mnesia/plugins-expand
      - RABBITMQ_PLUGINS_EXPAND_DIR=/var/lib/rabbitmq/plugins-expand
      - RABBITMQ_HIPE_COMPILE=1
      - RABBITMQ_DEFAULT_USER=admin
      - RABBITMQ_DEFAULT_PASS=rabbitmq_password_tpl
      - SERVICE_NAME={{.Service.Name}}
      - SLOT={{.Task.Slot}}
      - MASTER_SLOT=1
# not working with mounted external :(
    # volumes:
    #   - /external/efs/services/project_tpl/rabbitmq/mnesia/:/var/lib/rabbitmq/mnesia
#    ports:
#      - "5672:5672"   # amqp
#      - "15672:15672" # web ui
    # healthcheck:
    #   test: [ "CMD", "nc", "-z", "localhost", "5672" ]
    #   interval: 1m30s
    #   timeout: 15s
    #   retries: 3
    #   start_period: 45s

    networks:
      - traefiknet
      - project_tpl_network

The problem is: in this configuration cluster is not working because 3 containers all see one storage and it`s a problem.

will be nice to allow docker create folders for each containers automatically if it allowed !!!
example:

   volumes:
     - external/efs/services/project_tpl/rabbitmq/mnesia/{{.Service.Name}}/{{.Task.Slot}}/{{.Task.ID}}:/var/lib/rabbitmq/mnesia

this solution allow connect container to his own folder
If replicas will be increased from 3 to 4, next container will have his folder.
If replicas will be increased from 4 to 5, next container will have his folder.
Then replicas will deceased to 2.
If replicas will be increased from 2 to 3, container 3 connect to old folder (3)

It`s really needed.

THANKS A LOT

The classic solution would be to actualy use a named volume baked by nfsv4, pointing to the mnesia folder of the remote share. Then use the named volume (instead of bind-mounts) and map it into /var/lib/rabbitmq/mnesia.

Then set use the environment variable RABBITMQ_MNESIA_DIR=/var/lib/rabbitmq/mnesia/{{.Service.Name}}/{{.Task.Slot}}

Though, I would strongly advise to NOT use the {{.Task.ID}} as it is prone to change. See here for the template placeholds and example values.

N.B.: do not use the template placeholders in named volume declarations, as once a volume is created, its configuration is immutable. So whatever the current value for {{.Service.Name}}/{{.Task.Slot}}/{{.Task.ID}} will be when the named volume is created, it is carved into stone one that node (until you manualy delete it and let it re-create). This is an undesired behavior! Been there, done that, didn’t like the outcome :wink: