Are you familiar with setting up reverse proxy rules with nginx? You will basicly need to mount the nginx.conf holding your reverse proxy rules to /etc/nginx/nginx.conf and make sure to use the service names of the target container in the proxy_pass directives. Whetever you do, do NOT use container ips in the proxy_pass directive.
This thead discusses some useful details about nginx as a reverse proxy with docker (it is valid for non swarm deplyoments as well).
You might want to check the ip_hash directive of the upstream block.
Honestly sticky sessions are so 2010… modern applications typicaly store their session state in a key/values store, set the reverse proxy or load balancer to spread traffic evenly amongst the replicas, and let the application fetch existing sessions from the key/value store and push updates back to them.