Docker security issue

On a single VM I have two users user1 and user2.

If as user1 I run : docker run myimage
and map my readonly file to a file in the container, I get a new container running.
My file has 400 permissions, so on the host it is only I who sees the file. Also the file has 400 permissions inside the container.

If user attaches to the container using docker exec -it bash, he is able to see the mapped file.

This means the file that was protected carefully by user1 is exposed to user2.

Is there a way to avoid this?