Stack deploy, cloudstor plugin, compose format

General
1

Expected behavior

I’m defining services / networks / volumes in a v3 docker-compose format … volumes are of type “cloudstor:aws”. When the stack is deployed (docker stack deploy -c docker-compose.yml <stack-name>) I expect to see the services mount the cloudstor volumes, no matter what node that service gets deployed to.

services:
  app:
    image: <my-registry>:<my-app>
    volumes:
      - "data_vol:/data"
volumes:
  data_vol:
    external:
      name: "my_volume"

Actual behavior

The cloudstor volume doesn’t get mounted on the node where the service is deployed (unless that node already has the plugin “loaded” / is the node where the volumes were created)

running docker volume ls on the manager where I created the cloudstor volumes, lists the created volumes.
running docker volume ls on a different manager, only shows local volumes … perhaps this is expected?

Additional Information

I have tried adding the “driver” option in the compose file:-

volumes:
  data_vol:
    driver: "cloudstor:aws"
    external:
      name: "my_volume"

Steps to reproduce the behavior

  1. Manager1: create a cloudstor volume
  2. create a stack (docker-compose v3 file, with volume reference) … and deploy the stack
  3. if it deploys to Manager1, it will work … if it deploys to Manager/Worker, it doesn’t

any pointers / embarrassingly obvious mistakes?

thanks
e

2

Hi @eoinverling – unfortunately it seems that to use the --mount in a Compose file is not supported quite yet, although this PR adds support for it and hopefully should land in a release soon.

In the interim, I think you will need to use the suggested CLI method to take advantage of Cloudstor:

$ docker service create \
    --replicas 5 \
    --name ping1 \
    --mount type=volume,volume-driver=docker4x/cloudstor:aws-v17.03.0-ce,source=sharedvol1,destination=/shareddata \
    alpine ping docker.com
3

@eoinverling As @nathanleclaire pointed out above, creating/mounting a distinct volume (backed by cloudstor) for each task is not possible right now through Compose. However, if all your tasks for the service can share the same volume safely (i.e. they don’t overwrite each other’s data at the same time), you can achieve that today with something along the lines of:

version: "3"

services:
  stack-app:
    image: alpine
    command: ping docker.com
    volumes:
      - "data_vol:/data"
    deploy:
      mode: replicated
      replicas: 2

volumes:
  data_vol:
    driver: "cloudstor:aws"

If you now look at the nodes where the tasks for the above service got deployed, you should be able to see volumes named stackname_data_vol when you issue docker volume ls

4

Hi,

great, thanks for that … so I can “ping” all the volumes I need, on the nodes I need (based on a constraint for example) … then the volumes will be available.

I know this is early days for this … but just to note that when I remove a volume, it still lists as available on other nodes

thanks
e

5

That is true … we will make listing/removal of volumes more “globally” aware across the swarm nodes soon.

6

@ddebroy any updates on this?

7

@frezbo all volumes managed by cloudstor are now visible/enumerable across the swarm cluster. So if you create/remove a volume from one node, it should be visible/absent respectively from docker volume ls in other nodes too. You can find further details and options at https://docs.docker.com/docker-for-aws/persistent-data-volumes/#relocatable-cloudstor-volumes