Well after many days I have finally found the solution.
- Use complete UUIDs
- In /etc/nvidia-container-runtime/config.toml change “DOCKER_RESOURCE_GPU” to “DOCKER_RESOURCE_NVIDIA-GPU”
After making those changes the docker swarm will be able to select the correct GPU on machines with multiple GPUs.