Security: Side effects of building a Docker Image?

I have a very unusual requirement and I want to know if it is safe.

I want to provide users of my website with the ability to upload code. My website then builds a Docker Image from that code and runs this Image.

What do I need to do to make this safe?

As far as I can tell, running a Docker Image is already reasonably safe so long as I don’t open any ports on the Container and am careful about the Volumes I mount on it. Is this correct? (I’m aware that you can never have perfect security, so I will also put this inside a VM in addition to this)

But what about building the Docker Image?
Are there any side-effects that can happen on my server if I allow the user to build Docker Images on my server? If I control what can go into the Dockerfile, would that make things safe? Can there be side effects through the requirements.txt file?

(The reason I want to build the code on the server is that (1) I want to make sure I know what code is running inside the Image and (2) uploading just the code and building the Image on the server is much faster than pushing a full Docker Image to the registry running on my server.)

What about keeping the image static and just loading the code into a volume mount?