Docker Swarm: Persisting Data Volumes linked to Container instances?

Does Docker Swarm “link” a single instance Data Volume with a single instance Container running?

Does this scale up with multiple instances of both the containers and data volumes? IOW, if I scale=5 a specific container, would I get 5 matching Data Volumes - spread out across the Swarm Nodes?

What happens when that instance is destroyed? Does the Data Volume hang around (if so, I’d want a way to clean up old data volumes no longer in use)?

Docker swarm 1.13
Docker-compose 3

I am looking for a way to use a single docker-compose file for both local development, and deploying to a small docker swarm cluster. Since this is simply helping out a school part-time, as little maintenance as possible is what I am after. Hence, a set of docker-compose files for dev and production (using the override pattern).

My goal is to to run:

docker deploy --compose-file docker-compose.production.yml

To keep everything together and manageable for simplicity, if I can come back and manage something a few months from now.

I think I am looking for a feature of Docker called “Data Volumes”, but with no name and not mounted from the host node. Just a plain Data Volume, complete with its own dynamically named ContainerID and all, linked to that running Container.

Use case: CockroachDB

Most specifically: I have a datastore, CockroachDB, that I want to host within Swarm.

  • CockroachDB normally stores its own copy of data local to its instance, per machine.
  • CockroachDB gossips with other CockroachDBs to sync its data, which each stores their own copy of data.
  • When a new CockroachDB joins the cluster, it simply builds its local datastore from scratch via the gossip protocol.

Therefore, if running CockroachDB in Docker Swarm, each instance of Cockroach would need its own instance of a Data Volume tied to it to persist data between restarts and/or image updates/upgrades.

Is this possible with Docker Swarm?

I am not looking for a way to share a single EBS volume for data storage across multiple instances (i know there are drivers for that, and then there’s Flocker and other stuff). I just want a Data Volume tied to a specific Instance that is launched, and sticks around between restarts of that instance.

I ask because I keep reading about limitations with Docker Swarm about how certain data persistance features have been deprecated, talk of beta stuff, etc - and frankly, I have no idea what the current state is with Docker Swarm 1.13.

Thank you!

1 Like

Having exactly same scenario and looking for answers on same questions.
@eduncan911, did you get any progress with it?