Can I run GUI apps in Docker?

You probably ran the container in privileged mode, at least that is when I had similar issues with systemd in a container. Otherwise containers can’t affect the host system that way.

If you need a web browser only, you can use special images that have RDP servers inside so you can connect to the RDP server through your browser and use the brower in the container too.

I wrote about it here and shared two examples to use that browser for accessing a service in another container which has no internet access.

https://dev.to/rimelek/docker-network-and-network-namespaces-in-practice-5h9#testing-a-webbased-application-without-internet-in-a-container

You can run GUI apps in containers but that is not the main usecase of Docker containers. If you want containerized GUI apps, you can use other kind of containers like snap. In fact, when you install Firefox on Ubuntu, it is installed as a snap package.