Accessing ports on individual containers in a swarm

I expose jmx and a custom management port on my services that run on the jvm. This works fine but it means I can only access one container at random in each service.

I’m thinking the solution to access all the containers individually is going to involve a non-docker solution. Probably some proxy service that my containers can register with and then outside the swarm we can access through that.

Before I build this, anyone have any existing solutions or a native docker way to achieve this?