We want to run a single application in a docker container that simulates multiple devices, each with their own ip address. Docker compose only supports configuring one static ip address to a container. There is a work around to run the ip address add command from within the container which works well (https://stackoverflow.com/questions/74733967/assigning-multiple-ip-addresses-to-docker-container), but requires NET_ADMIN privilege. My understanding is that this is safe since as long as you are using a docker bridge network and not the host network. However, we have a general mandate to not escalate privileges since we deliver this solution to different customers.
My question is whether it would be a heavy lift to add support to docker/compose to set multiple ip address (create multiple interfaces) to a docker container. There are many instances were a device has multiple interfaces on the same subnet, and one might want to use a container to emulate that w/o having to escalate privileges. So it seems like a reasonable feature request.
Given that I’m a total docker noob, is there someone in the community we could contract with to make such a patch and possibly upstream it?
“In this case, you need to designate a physical interface on your Docker host to use for the Macvlan”
Our goal is to avoid any host requirement or privilege escalation
Is it an option to start the container multiple times with each one simulating ONE client instead of starting one container simulating multiple clients?
No, the sims run under a single process and communicate with each other over an internal bus. If we had designed for docker maybe, but there are some benefits to having just one process simulate multiple devices. Again I think real world systems with multiple interfaces are common and would share the file system as well as apps
A container by itself has no network interface as all. Try: docker run -ti --rm --network=none alpine ip a
A container gets a network interface per attached network, with an ip from the subnet of the network. If you are fine with ips from different subnets, then the solution is already there and simple to configure.
Again out goal is to be agnostic of the host and its network and not require any privilege escalation such as NET_ADMIN. We will he running multiple containers communicating over a docker network. We just want one of those containers to have multiple network interfaces.
I don’t see where any capabilities or privilege escalation were recommendeed . I think @meyay meant you can have multiple docker networks attached to a single container. That doesn’t require changing anything on the host. The “network=none” was just an example for a container without network. Not to configure one yourself without Docker.
Yes but doesn’t using multiple networks imply different subnets. We want everything on the same subnet. We also have fixed ip addresses that we use in production and don’t want to change them
Oh, I see. Sorry, I didn’t read the first post before replying.
If you want to ask for a new feature, you can opena ticket in the roadmap
But as you wrote, you did not design the app for containers
So I would not expect that feature to be implemented soon.
If you just want to avoid privilege escalation in the container where your tested app is running, you can create a container that just owns the network namespace and other containers can connect to it, a second container that has the NETWORK_ADMIN capability to run the ip addr add commands and stop, and the third container your app without any special privileges but using the same network namespace as the other two. Here is an example:
services:
network_owner:
image: k8s.gcr.io/pause:3.9
networks:
static:
ipv4_address: 172.17.0.2
network_admin:
image: nicolaka/netshoot:v0.14
command:
- bash
- -c
- |
ip addr add 172.17.0.3/32 dev eth0
ip addr add 172.17.0.4/32 dev eth0
ip addr add 172.17.0.5/32 dev eth0
ip addr
network_mode: service:network_owner
cap_add:
- NET_ADMIN
nginx:
image: nginx
network_mode: service:network_owner
networks:
static:
ipam:
driver: default
config:
- subnet: 172.17.0.0/24
update:
I forgot to mention that this way you could only add port forwarding (if needed at all) to the “network_owner” service as that is the only container that has its own network. It ould still work as all containers would use the same network namespace.
Sounds like that would involve port forwarding. I’ve already tried something similar by creating a “proxy” container that just runs a socat command to serve the ip and forward packets on a specific port to the “test app” docker. The issue with that is that UDP messages arrive with proxy address instead of the sender’s address. It also involves additional containers and redirection - not that those are big issues but overall becomes a messy workaround rather than a solution.
I only menitoned port forwarding because IF that is needed, you need to know where that can be set. If you need communication between containers, there is no need for port forwarding. You wrote previously that you could add IP addresses by running commands in the container, but you didn’t want to give privileged mode or additional capabilities to the container. I just extended that solution. There is only one network namespace for all three containers. It means that in terms of network, there is only one container. You don’t need to forward request from one to another, and you couldn’t even if you wanted to. There is nothing to forward to.