I have a docker container that needs to expose 10,000 ports

General
1

I have a container that exposes udp 10000-20000. It had to fire off 10000 docker-proxy processes. This takes about an hour to start and stop. Is there a way around this?

2

Did you try docker run -p 10000-20000:10000-20000/udp?

1 Like
3

Yes. That fires off 10,000 user-land docker-proxy processes

4

Oh that’s horrible. Looks like --network=host might be the only satisfiying option…

5

The only downside is that I’m prepping this for kubernetes usage. I’m hoping that the k8’s networking routing will help with this as this would limit me to one instance per host.

6

Yep, one instace per host is nasty,

For what its worth: If you run your service as global swarm service or k8s daemon set, you will have exactly one instance on each node…

I never had a use case that required to publish a range in k8s.

7

I suppose I can always assign multiple IP addresses and play with the docker networking underneath… Hmm… It would be best if docker-proxy supported a range.

8

I remembered I came across a blog post that had a deep dive on the docker-proxy.

9

OK, I tried starting docker-daemon with userland-proxy = false. It does not fire off the docker-proxy processes, but it’s just hanging when I start the container. Trying to figure out how to get some debug info here, as I’m still sort of new at this.

10

It looks like I may have to be waiting for 10,000 iptables rules to complete, one by one. Both the container and daemon are stuck in futex_wait. Host mode seems to work. I can probably work with that but it will require some changes in my app.

11

I would recommend to testdrive your image on k8s first. I am not sure if k8s service’s of type NodePort or Loadbalancer (~= published port in docker) even support a range.

I you head the NodePort way, you need to make sure your cluster is configured to support your required port range (+ some additional ports) as node port range. I would strongly recommend to use a Loadbalancer of your cloud provider or if run baremetal MetalLB.

15

I have a solution for this.

  1. run with userland-proxy=false
    In /etc/docker/daemon.json {“userland-proxy”: false}
    Don’t use -P or -p on the docker create/run command line.
    Manually put in iptables rules:
    CIP=$(docker inspect --format=’{{.NetworkSettings.IPAddress}}’ container)
    iptables -A DOCKER -t nat -p udp -m udp ! -i docker0 --dport 10000:20000 -j DNA
    T --to-destination $CIP:10000-20000
    iptables -A DOCKER -p udp -m udp -d $CIP/32 ! -i docker0 -o docker0 --dport 100
    00:20000 -j ACCEPT
    iptables -A POSTROUTING -t nat -p udp -m udp -s $CIP/32 -d $CIP/32 --dport 1000
    0:20000 -j MASQUERADE

https://hub.docker.com/r/bettervoice/freeswitch-container/

1 Like