If you use docker service rm, then the service is removed, all containers, too.
Use docker service update, check the tutorial and the doc, probably use --update-order=stop-first and add --update-delay.
Our services do not expose ports on host, instead Traefik proxy and a Docker Swarm overlay network is used. So no port conflicts.