Own bridge for each container

General
1

Hello. How to make that each container uses its own bridge and thus uses its own rndis ethernet? Ubuntu desktop 22.04, a main ethernet is enp2s0 and a bunch of rndis devices (usb dongle), that manages via netplan. When I adjust like in the code below, a container moves traffic through enp2s0 anyway, I guess it is because of route-metric: 10. But I can’t change it to keep the host’s services work as usual. I have a proxy server on the host and it works fine with br87, br88 and etc. Each proxy uses each own bridge adapter@rndis iface. But when I check an external IP from a container that uses br87, I see an external IP that belongs to enp2s0. I need to run a lot of containers.
I’ve tried to use macvlan, but it does not work, because of each macvlan network should use its own subnet. But every of my rndis devices has DHCP server with the same subnet and it can’t be changed because of firmware.
Unfortunatelly, I don’t understand IPTABLES and it’s very hard for me to manage rules. Is there any way to get my aim?

Creating network:

docker network create -d bridge --subnet 10.87.87.0/24 --gateway 10.87.87.1 --opt "com.docker.network.bridge.name"="br87" net87
docker run -it --name id87 --network net87 my_image

Netplan sample:

network:
  version: 2
  renderer: networkd
  ethernets:
    enp2s0:
      dhcp4: true
      dhcp4-overrides:
        route-metric: 10
      dhcp6: false
	id87:
      dhcp4: false
      dhcp6: false
      optional: true	  
  bridges:
    br87:
      dhcp4: yes
      interfaces:
        - id87

ip route | grep br87:

default via 192.168.87.1 dev br87 proto dhcp src 192.168.87.213 metric 100
10.87.87.0/24 dev br87 proto kernel scope link src 10.87.87.1
192.168.87.0/24 dev br87 proto kernel scope link src 192.168.87.213 metric 100
192.168.87.1 dev br87 proto dhcp scope link src 192.168.87.213 metric 100

iptables -t nat -L -v:

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 510K   30M DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
   10   600 DOCKER     all  --  any    any     anywhere            !localhost/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
   46 12504 MASQUERADE  all  --  any    !br87   10.87.87.0/24        anywhere
   14   886 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  br87   any     anywhere             anywhere
    0     0 RETURN     all  --  docker0 any     anywhere             anywhere
2

Hey @zippycrypto,

  1. Create Docker networks for each RNDIS interface.
  2. Configure Netplan to assign each RNDIS interface to a unique bridge.
  3. Run Docker containers on their respective networks.
  4. Set up policy-based routing to route traffic from each subnet through its respective interface.
  5. Adjust IPTables if necessary.

By following these steps, you should be able to route each container’s traffic through its designated RNDIS Ethernet interface.

3

Be kind, show a sample for 4 and 5.