I have a swarm cluster with 4 nodes which’s running a simple gRPC service docker container on (the gRPC service container when start up and run it will generate a Guid value and return to client in the response).
The problem is gRPC service are using http2, that meaning it will need to keep the connection alive between client and server . when I deployed the gRPC service stack to swarm cluster with the replicas=1 and try to create 3 gRPC client to connect and call the service continuously , then all connection is routed one gRPC service container then I try to use docker service update <containerid> --replicas=3 to increase the number of gRPC service container instances but that 3 clients still connect to one container not load balancing happen there (the same Guid value is returned in all client calls). (Note: stop and start 3 client again then the connections Load Balanced as expected for 3 gRPC service containers)
Is this what we expect Routing mesh to deal with http2 connection? is there anyway to make http2 connections to be balanced to the gRPC service containers without restart the clients!