Docker Community Forums

Share and learn in the Docker community.

I have a docker container that needs to expose 10,000 ports

I have a container that exposes udp 10000-20000. It had to fire off 10000 docker-proxy processes. This takes about an hour to start and stop. Is there a way around this?

Did you try docker run -p 10000-20000:10000-20000/udp?

1 Like

Yes. That fires off 10,000 user-land docker-proxy processes

Oh that’s horrible. Looks like --network=host might be the only satisfiying option…

The only downside is that I’m prepping this for kubernetes usage. I’m hoping that the k8’s networking routing will help with this as this would limit me to one instance per host.

Yep, one instace per host is nasty,

For what its worth: If you run your service as global swarm service or k8s daemon set, you will have exactly one instance on each node…

I never had a use case that required to publish a range in k8s.

I suppose I can always assign multiple IP addresses and play with the docker networking underneath… Hmm… It would be best if docker-proxy supported a range.

I remembered I came across a blog post that had a deep dive on the docker-proxy.

OK, I tried starting docker-daemon with userland-proxy = false. It does not fire off the docker-proxy processes, but it’s just hanging when I start the container. Trying to figure out how to get some debug info here, as I’m still sort of new at this.

It looks like I may have to be waiting for 10,000 iptables rules to complete, one by one. Both the container and daemon are stuck in futex_wait. Host mode seems to work. I can probably work with that but it will require some changes in my app.

I would recommend to testdrive your image on k8s first. I am not sure if k8s service’s of type NodePort or Loadbalancer (~= published port in docker) even support a range.

I you head the NodePort way, you need to make sure your cluster is configured to support your required port range (+ some additional ports) as node port range. I would strongly recommend to use a Loadbalancer of your cloud provider or if run baremetal MetalLB.

I have a dockerized SIP application that requires a large number of ports for RTP. Since it recently became possible to expose a port range, I decided to move away from --net=host scenario to using those ranges (-p 30000-40000:30000-40000/udp)

However, when a range is large enough, docker eats up all RAM and fails:

ERRO[0230] Handler for POST /containers/{name:.*}/start returned error: Cannot start container 80df70ab22d94408e9a5a2c60590b1b1281e5a59b5531590738739c9f7c7c485: iptables failed: iptables --wait -t nat -A DOCKER -p udp -d 0/0 --dport 38207 ! -i docker0 -j DNAT --to-destination 172.17.0.3:38207: (fork/exec /sbin/iptables: cannot allocate memory)
ERRO[0230] HTTP Error: statusCode=500 Cannot start container 80df70ab22d94408e9a5a2c60590b1b1281e5a59b5531590738739c9f7c7c485: iptables failed: iptables --wait -t nat -A DOCKER -p udp -d 0/0 --dport 38207 ! -i docker0 -j DNAT --to-destination 172.17.0.3:38207: (fork/exec /sbin/iptables: cannot allocate memory)

The implementation right now forks a dockey-proxy process for each port, so you get 10,000 new processes. If you have userland-proxy = false, it still has to run 10,000 iptables processes.

I didn’t see that you responded to lewish95, because I did put him/her/it?! on the ignore list.
Its either an idiot or a terrible bot. The responses usualy are direct quotes from googled pages and rarely contribute to solution.

I have a solution for this.

  1. run with userland-proxy=false
    In /etc/docker/daemon.json {“userland-proxy”: false}
    Don’t use -P or -p on the docker create/run command line.
    Manually put in iptables rules:
    CIP=$(docker inspect --format=’{{.NetworkSettings.IPAddress}}’ container)
    iptables -A DOCKER -t nat -p udp -m udp ! -i docker0 --dport 10000:20000 -j DNA
    T --to-destination $CIP:10000-20000
    iptables -A DOCKER -p udp -m udp -d $CIP/32 ! -i docker0 -o docker0 --dport 100
    00:20000 -j ACCEPT
    iptables -A POSTROUTING -t nat -p udp -m udp -s $CIP/32 -d $CIP/32 --dport 1000
    0:20000 -j MASQUERADE

https://hub.docker.com/r/bettervoice/freeswitch-container/

1 Like