Idealy nginx/haproxy are running in an attachable container overlay network, and the target services are attached to the same network. Btw. nginx supports layer4 traffic by declaring a stream block which could be used for tcp.
What you experience in nginx is caused by dns caching. You can mitigate dns caching like this: NGINX swarm redeploy timeouts - #5 by meyay
Though, I would highly recommend to use Traefik instead of nginx (and haproxy), as it updates the reverse proxy configuration based on (Swarm) service labels or container labels (non swarm services) whenever a task/container is created/deleted. I have never experience any dns caching issue with Traefik.