Running OpenGL from container irrespective of nvidia/not nvidia host drivers

System: Ubuntu 18.04

I have a container that runs a GUI app with OpenGL. Normally the sharing of X sockets is done with these parameters:
-e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix

This works, but only if both the container and the host have the same graphics drivers. If, for example, container has only software rendering, but the host has NVidia drivers, the communication doesn’t work.

There is a workaround: I can run X-server (Xephyr) on the host and some window manager (qtile) in the container. Then the setup works on any host, regardless of its drivers. But this workaround complicates the setup for the end user: he now needs to launch Xephyr on the host.

Is there a better way to launch simple OpenGL apps from a container? The rendering is pretty basic so performance is not an issue.