We’ve been running on Docker Engine Swarm for AWS for a bit under a year now and have multiple clusters.
Last week when we provisioned a new cluster our containers started having connectivity issue. Some times, they can’t see other services within Swarm. That is very inconsistent and out of 13 services it’s alway at least one would have a problem. We re-deployed from docker-compose about 20 times and every time some containers would not be able to see some others within same overlay network.
I’ve put more details in this ticket for moby project - https://github.com/moby/moby/issues/34015
Not sure its a right place. Let me know if you see similar problems or where else I should create a ticket.
Workaround - turning off overlay network encryption or getting rid of all custom networks and using only a default ones. Which breaks lots of security best practices…