Polling individual container metrics in a Swarm

We would like to scrape a /metrics endpoint on each container in a swarm.
From our understanding, this is not possible due to the Docker Swarm Load-Balancer.

Our understanding is that within a swarm cluster, any ‘service’ is non-deterministically load balanced within the swarm.

So polling m000:12003/metrics might come from any other machine running that service.

Is there a method through which we can either bypass the swarm load balancer, or ask it not to forward a request?