Can you show how you do it? If you mean you use the “expose” parameter or Dockerfile instruction, that is not enough. You need to forward ports.
So if you can run for example curl http://127.0.0.1:PORTNUMBER to access the service, it means that you either forwarded the port from your localhost only, or you do have some firewall even if you did not install any.
You don’t forward port TO the public IP but FROM the public IP TO the container. Since your container has NET_ADMIN capability and you also use VPN, try an other container without VPN just to make sure that the port forward works with a simple container like nginx so you can open the main page from the browser using the public IP.