Beginner question about Docker swarm in practice: microservices and network latency

Open Source Projects Swarm
1

I’ve read in several places that it’s recommended to keep to the 1 service:1 container practice according to the microservices paradigm. I understand that it allows for separation of concerns, flexibility for scaling and so on. But at the same time I’ve been finding that some people opt to couple two services that will do a lot of communication with each other in one container, because in their mind it reduces complexity. Personally, I’m concerned that deploying services in separate containers creates the potential for increased network latency/an additional network hop if the containers end up on separate nodes in the Swarm. Then again, I have trouble judging whether this is worth worrying about since I’m lacking in expertise.

Can people share where they draw the line when it comes to microservices and what are the considerations you take when deciding? For instance, if you have Python web app, do you separate your wsgi server and Nginx?

Currently I have nginx and my Django backend/GraphQL server coupled together in one container, and my React frontend in another. Since i’m fretting about the network latency, I’ve been wondering if in theory I should try to ensure that each container is deployed alongside the other on the same node, so that each request from the frontend can go directly to the backend on the same node (instead of hopping over to a replica of the backend on another node). Does anyone try to do this, or is it overengineering?

I know I don’t REALLY need to worry about this at this point in my project, but I tend to be preoccupied by what’s considered ‘best practice’ and would like to have a better sense of what that is. Thanks for any input, I appreciate it!

2

Hi Rona,

Good question. This is one of the reasons why I have not found “one process per container” to be a good rallying cry. As you note, “one service per container” or “one concern per container” is more descriptive. To answer this directly:

For instance, if you have Python web app, do you separate your wsgi server and Nginx?

If you have tightly coupled components (which nginx <=> wsgi <=> python is) and putting them in the same container makes implementation a lot easier, then nope! Just put them in the same. Rigging up things like Apache/Nginx serving PHP in separate containers is a nightmare, for instance.

With only one service/concern, don’t fret. It’s more important to break apart concerns like the data layer (e.g., MySQL), service-agnostic reverse proxies, separate programs with independent business logic, and so on. A good guide is to ask if the programs are “cheating” by sharing state. In nginx/wsgi’s case, it does not share state with the business layer of the Python app, so I find putting them together more acceptable. If they were two separate apps reading and writing to a shared local filesystem, this would be more likely to cause problems.

Whatever helps you get the job done effectively is a good thing.

Indeed, this type of tight coupling is one reason why pods, groups of tightly associated co-scheduled resources which share a network namespace, are a first-class construct in Kubernetes.

By the way, while your instinct to be concerned about the network overhead of various , it’s unlikely that it will be a practical issue until very large scale. You can examine the latency yourself of things like Docker’s built-in bridge network and proxy by using a tool like Apache Benchmark. For instance, let’s take an unscientific look at the distribution of request latencies with a locally running nginx on my Linux laptop vs. one running in a container. (Note: this is different than container-to-container latencies on an overlay network, which is likely to be higher).

Running 30000 HTTP requests, 100 concurrently outside of a container:

$ ab -c 100 -n 30000 http://127.0.0.1/
Percentage of the requests served within a certain time (ms)
  50%      3
  66%      3
  75%      3
  80%      3
  90%      4
  95%      4
  98%      4
  99%      5
 100%     16 (longest request)

Running 30000 HTTP requests, 100 concurrently to nginx inside a container:

$ ab -c 100 -n 30000 http://127.0.0.1:8888/
Percentage of the requests served within a certain time (ms)
  50%      9
  66%     10
  75%     12
  80%     12
  90%     16
  95%     21
  98%     32
  99%     64
 100%    214 (longest request)

The long tail of 64 vs. 5 ms for 99 percentile latency certainly is noteworthy, but even the fastest Python webservers might take up to 900ms to load a query into an ORM and serve a request under load. So if Docker is introducing 60ms of latency in the 99th percentile, even a fast Python webserver is chewing through multiples of that on its own for every request.

My point is, optimize for performance bottlenecks when they become a problem, not before, because these types of system-level problems are unlikely to be your main issue. Misbehaving SQL queries, unreliability of software you’re depending on, naughty API client behavior, etc. will probably cause you many more headaches.

3

Hi Nathan!

Thanks so much for clarifying the one service : one container idea to me. What you say makes a lot of sense and will give me a good guideline for thinking about how to distribute my components in containers for now on.

Also, thanks for the example with Apache Benchmark! I’ve never gotten into the benchmarking territory but this example makes it seem like a straightforward thing to try when time is available :slight_smile:

And you make a really good point about when to optimize. I’ve seen it mentioned before that premature optimization is often misdirected, but the examples really help! It gives me a little peace of mind to get the reminder that I can always optimize later, but I’m also glad I got some context for the impact of different choices from your response. :+1: Thanks again!