Having studied the official Docker documentation I am still not clear on the best approach to creating and deploying a multi-container app with one container in each of multiple AWS instances.
Obviously “docker swarm” comes into it, but what about “docker stack”, and “docker network”, and where if anywhere does “docker-compose” still come into the picture? (I currently use docker-compose to build the set of apps to deploy on one host.)
There seems to be a bewildering variety of overlapping docker command functionality, which I suppose reflects the changing networking capabilities of Docker over recent years, especially in relation to cloud services.
So I wondered if some knowledgeable Docker guru could sketch the best current practice for dealing with the following set of containers, each running on a different AWS instance (“host”), and assuming I can use the latest Docker version:
Postgres Django model database (one container)
Django web server (two containers, load balanced, possibly more in future)
Cassandra database container (two containers, possibly more in future)
Docker Compose (the cli tool) is a single node multi container orchestrator.
Docker Swarm is a multi node multi container orchestrator.
Both share that they use the Compose file format to describe their deployments. Though, Swarm requires at least v3.x of the specification and has some swarm specific configuration items. docker stack is used to controll (deploy/remove) the stack. A stack deployment will create swarm services, which are controlled with docker service. It sill makes sense to use docker network if you need to create a network that needs to exist regardless wethere a stack is deployed or not. Typicaly reverse proxies containers are located in such externaly created networks.
Docker Compose (the cli tool) is compatible with a Compose file with the swarm specific additons, but it will completly ignore them.
I am not sure what type of input you expect here. Just write down you compose file, plan which container is ment to be deployed on which node(s). Add dedicated node labels on that nodes, add the labels to the placement constraints and during deployment swarm will schedule the containers according your constraints.
Use replicas wherer they apply: instances of a single service that should add as ONE scalled out service. By default a swarm service gets a VIP which forwards traffic to the replicas.
For Cassandra, you will want to search the forum. I do remember a thread from some weeks ago where Cassandra missbehaved on Swarm and required an extra configuration (I forget the exact problem). Update: the problem was related deploying Cassandra directly as service, instead of beeing deployed as part of a stack. The problemdoes not apply for swarm stack deployments - the drafted solution is not required with stack deployments .(see: here)
Many thanks for your prompt reply. I see you referred to Docker stack, and that is part of my confusion, and no doubt I’m not the only person confused by this.
If “docker swarm” can set up a multi-host, multi-container application then what extra does “docker stack” buy one? Putting it another way, when would “docker stack” have to be used, or be better to use than “docker swarm” or possibly in addition to it?
The official Docker documentation covers all the details, such as command options and Dockerfile formats. But nowhere, as far as I can see, is there anywhere an overall summary that allows one to see the wood for the trees! A set of realistic use cases, and the optimal combination of docker tools to use for each, would be very useful.
Actualy it is quite simple: docker stack is the multi node equivalent to docker-compose - both require a docker-compose.yml, both can deploy multiple services, networks and volumes within a single deployment. A docker stack can additionaly deploy secrets and configs. When a docker stack is deployed it creates swarm services and all the other objects.
Instead of docker-compose up -d, you will have to use docker stack deploy -c docker-compose.yml {stackname}.
if you prefer docker run over docker-compose you can use docker service create to create your swarm services, which creates the container(s).
I must admit when I switched to swarm years ago, the move from docker-compose.yml based docker-compose deployments to docker-compose.yml based swarm deployments (= docker stacks) came natural to me.
Many thanks again. That all makes sense, but it seems from “docker swarm”, summarised in your first reply, and “docker stack” in your second that they are both extensions of “docker-compose” for multiple nodes.
So this is the heart of my slight confusion: Why does one need both? What does one do that the other doesn’t, and vice versa?!