Scaling parallel pipelines using docker swarm

General Discussions General
1

I have been trying to scale my application recently. Consider my application having 3 services which work fine with a docker-compose file.

As a first step, I was able to scale them across devices using docker-swarm and replica factor. With this, the docker swarm was able to run a few services in a few devices and few other services in a few other devices. These were working fine as intended. In this use-case any replica of service1 talks to any replica of sevice2 and any replica of service2 talks to any replica of service3 .

Now I want a modification to this architecture as shown belowenter image description here As you can see, here each replica talks within itself and do not talk with the other replica. In other words Service1.1 should talk only with Service2.1 and should not talk with Service2.2 , Service2.3 and Service2.4 . How can I make this possible with the help of a docker swarm?

2

Replicas are exact(!) clones with the same configuration and part of the same network. They are not just a different instance of the container that can be used with a different configuration (e.g. a dedicated instance for a different customer or a seperate testing environment).

Docker Swarm does not provide container affinity: with replicated services, there is no guaranty that all replicas of a service will be spread around all nodes or worst case be run on the same node. You could use global mode instead, to at least guaranty that each node runs a single instance of a service. I am not aware of any way that restricts the communication to a replica in way that only the local instance is queried - the communication will take place randomly amongst the replicas, regardless on which node they are executed.