Binding containers to a specific host VLAN (multi interface server)

General Discussions General
1

I have a host which is connected to multiple VLANs that have certain routing rules at my router, they have different properties.

I’ve seen other suggestions about running docker in a VM for each VLAN but that seems ugly and messy too.

/etc/network/interfaces:

# Is routed out a VPN on my router
auto bond0.3
iface bond0.3 inet dhcp
iface bond0.3 inet6 auto

# Is null routed at my router (LAN only)
auto bond0.4
iface bond0.4 inet dhcp
  dns-nameserver 192.168.3.1
iface bond0.4 inet6 auto

# Routed directly to my ISP for containers only
auto bond0.7
iface bond0.7 inet dhcp
iface bond0.7 inet6 auto

I used a macvlan interface on a docker interface called “external_7”:

networks:
  external_v7:
    name: external_v7
    driver: macvlan
    driver_opts:
      parent: bond0.7
    enable_ipv6: false
    ipam:
      config:
      - subnet: 192.168.7.0/24
        gateway: 192.168.7.1
        ip_range: 192.168.7.128/26

Next up I have a Bittorrent client behind a VPN, the main reason for having separate VPN tunnel container and not using the one on my router is because when downloading something it chokes the tunnel, and I can’t use it for anything else, so I have a dedicated VPN tunnel for Bittorrent:

services:
  vpn_bittorrent:
    extends:
      file: ../docker-vpn/docker-compose-vpn.yml
      service: openvpn-client
    container_name: vpn_bittorrent
    networks:
      external_v7:
        ipv4_address: 192.168.7.128
    volumes:
      - /mnt/data/container_data/vpn:/data/vpn
    ports:
      - 192.168.7.128:26129:26129
      - 192.168.7.128:26129:26129/udp

Which takes extension from:

services:
  openvpn-client:
    image: ghcr.io/wfg/openvpn-client
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    restart: unless-stopped

Finally I have qbittorrent running:

  qbittorrent:
    image: ghcr.io/linuxserver/qbittorrent
    container_name: qbittorrent
    network_mode: service:vpn_bittorrent
    environment:
      - PUID=1004
      - PGID=1003
      - WEBUI_PORT=8080
    volumes:
      - /mnt/data/container_data/qbittorrent:/config
      - /mnt/data/shared/incoming:/mnt/shared/incoming
    restart: unless-stopped

Now of course I like to have a reverse proxy in front of this, because I like to use standard ports like :443 or whatever.

  web:
    image: nginx:stable-alpine
    container_name: web
    hostname: web.internal
    networks:
      external_v4:
        ipv4_address: 192.168.4.133
    volumes:
      - /mnt/data/container_data/web/nginx/auth-basic.conf:/etc/nginx/auth-basic.conf:ro
      - /mnt/data/container_data/web/nginx/htpasswd:/etc/nginx/htpasswd:ro
      - /mnt/data/container_data/web/nginx/http.d:/etc/nginx/http.d:ro
      - /mnt/data/container_data/web/nginx/nginx.conf:/etc/nginx/nginx.conf:ro
      - /mnt/data/container_data/web/nginx/proxy-control.conf:/etc/nginx/proxy-control.conf:ro
      - /mnt/data/container_data/web/nginx/ssl.conf:/etc/nginx/ssl.conf:ro
      - /mnt/data/container_data/web/nginx/ssl:/etc/nginx/ssl:ro
      - /mnt/data/container_data/web/nginx/html:/etc/nginx/html:ro
    command: [nginx-debug, '-g', 'daemon off;']
    ports:
      - 8080:8080
      - 9090:9090
      - 8989:8989
      - 7878:7878
    restart: unless-stopped

Each service must require a config:

server {
    listen 443;
    listen [::]:443;
    include /etc/nginx/ssl/qbittorrent.web.internal/qbittorrent.web.internal.conf;

    server_name qbittorrent.web.internal;


    location / {
        proxy_pass http://192.168.7.128:8080/;
        include    /etc/nginx/proxy-control.conf;
        include    /etc/nginx/auth-basic.conf;
    }
}

Now there’s a couple of problems:

  1. If for some reason the VPN container vpn_bittorrent restarts the nginx proxy web will not be able to see the bittorrent container anymore until it too is restarted, the result is a “502 Bad Gateway” error. I suspect this has something to do with the connection between qbittorrent and vpn_bittorrent ie the network_mode: service:vpn_bittorrent . I don’t particularly like this as it makes my whole docker setup incredibly fragile

  2. I want to replace my reverse proxy with something a little more flexible like https://traefik.io/, I don’t want to have to configure nginx configs manually to match a certain IP address eg 192.168.7.128

I’m looking for a simpler way which will allow me to use something like Traefik and resolve my Bad Gateway issues.

2

I know, this is a little old question, but I didn’t have time to answer until know. You might have realized by now, but you don’t need to write your nginx rules manually: GitHub - nginx-proxy/nginx-proxy: Automated nginx proxy for Docker containers using docker-gen
I don’t know if it works with macvlan though.

As for the restart issue, I only have a guess. If the VPN gives you an IP address and bittorrent starts listening on it, if the VPN restarts and changes the IP address, bittorrent doesnt have the previous IP address anymore and it will not use the new one.