Accessing gpu with container with usernamespace mapping

You could try the documentation of Nvidia for Rootless containers

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#rootless-mode

If it works with rootless Docker, which also uses user namespaces, then maybe it works with only usernamespaces too.