Is it possible to access Internet with only one public address for a swarm?

Let’s say there’s a swarm with one manager node and one worker node. Only the manager node has a public address and is able to access the outside world directly. The worker node has only a private subnet address. The whole structure contains a standalone load balancer (outside of Docker system) which makes user requests reach my app services.

In some situation some of my services need to make requests to outside third-party API from inside the containers. But I see always a half of the requests failed with a timeout exception, and those requests come from the worker node. I think the worker node’s network connection is the reason, and attaching public address can solve the problem. But I’m not able to control such things.

So the question: is there a way to make the manager node share the Internet connection to the worker node? If no, what’s the best practice to this kind of use case? (I don’t see the must of more and more public addresses just for the growth of nodes is a good thing.)

Thanks in advance.