Docker container attached to macvlan network can't communicate with the rest of the network

General
, ,
1

Hi all,

I have an issue where I’m trying to assign an unique IP address to a Pi-Hole Docker container on Debian 12 installed inside a VirtualBox VM in order to be able to host both self-hosted services and a DNS server on the same machine. As it’s recommended online, I have decided to configure a macvlan network in order to assign an unique IP address to a container. I am using a host-only network adapter in VirtualBox, given that for now it’s a testing setup before I decide on a solution that’s best for me. The IP address 192.168.92.3 is inaccessible from outside the container, making the Pi-Hole useless.

Pinging to the container doesn’t work, as the outside connectivity fails as well. I’ve spent multiple days trying to troubleshoot an issue, only to be disappointed at the end given that it’s my first time using macvlan.

Here’s my Docker Compose configuration:

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    networks:
      pihole-vlan:
        ipv4_address: 192.168.92.3
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      FTLCONF_dns_listeningMode: 'all'    # Volumes store your data between container upgrades
      ServerIP: 192.168.92.3
    volumes:
      - ${PATH_TO_DATA}/pihole/etc-pihole:/etc/pihole
      - ${PATH_TO_DATA}/pihole/etc-dnsmasq.d:/etc/dnsmasq.d
    restart: unless-stopped

networks:
  pihole-vlan:
    driver: macvlan
    driver_opts:
      parent: enp0s8
    ipam:
      config:
        - subnet: 192.168.92.0/24
          gateway: 192.168.92.1

Here’s the output of pinging from a server:

$ ping 192.168.92.3
PING 192.168.92.3 (192.168.92.3) 56(84) bytes of data.
From 192.168.92.2 icmp_seq=1 Destination Host Unreachable
From 192.168.92.2 icmp_seq=2 Destination Host Unreachable
From 192.168.92.2 icmp_seq=3 Destination Host Unreachable
From 192.168.92.2 icmp_seq=4 Destination Host Unreachable
From 192.168.92.2 icmp_seq=5 Destination Host Unreachable
From 192.168.92.2 icmp_seq=6 Destination Host Unreachable
^C
--- 192.168.92.3 ping statistics ---
7 packets transmitted, 0 received, +6 errors, 100% packet loss, time 6151ms

Here’s how it looks on my PC:

C:\Users\[REDACTED]>ping 192.168.92.3

Pinging 192.168.92.3 with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.92.3:
    Packets: Sent = 3, Received = 0, Lost = 3 (100% loss),
Control-C
^C

Not only that, attempts to ping the host fail:

$ sudo docker exec -it pihole sh
/ # ping 192.168.92.2
PING 192.168.92.2 (192.168.92.2): 56 data bytes
^C
--- 192.168.92.2 ping statistics ---
6 packets transmitted, 0 packets received, 100% packet loss

Here is the IP route of a container:

/ # ip route
default via 192.168.92.1 dev eth0
192.168.92.0/24 dev eth0 proto kernel scope link src 192.168.92.3

Here’s there a relevant line related to the VM’s IP route:

192.168.92.0/24 dev enp0s8 proto kernel scope link src 192.168.92.2
2

I figured it out. It turns out that macvlan requires promiscuous mode to be enabled. In VirtualBox, it can be done through the Virtual Machine Settings > Network > Choose a related network adapter > Advanced > Promiscuous Mode > Allow All (I’ve tried “Allow Virtual Machines”, but it doesn’t work for at least for me).

After changing the VirtualBox settings, it’s also necessary to enable it within a VM:

sudo ip link set enp0sX promisc on

After configuring the promiscuous mode, I am able to connect to a container from my host machine.

Note that due to nature of macvlan, you won’t be able to access the container from within the VM without special configuration as described here: Using Docker macvlan networks :: blog.oddbit.com

3

Note, you don’t need (and probably should not have) those ports defined. That’s for mapping ports on an internal docker bridge network to ports on the host. Once you have your container on a macvlan, those ports only need to be on the container, which is visible to the rest of the LAN (so 192.168.92.3:53, not 192.168.92.2:53). I keep meaning to check what happens when you publish ports on the macvlan container. At least one commenter on StackOverflow says port redirects are ignored, and I think that should be the case).

1 Like
4

Finally found something official from Docker: Port publishing and mapping | Docker Docs
It doesn’t specifically say that port publishing has no effect in a macvlan network, but does say it’s for bridge networks.

1 Like
5

Indeed, the mapped ports only make sense, if a bridge network is additionally attached to the container.

This additional bridge network could be used from the host itself, to access a service inside the container through the published host port. Due to kernel security restrictions the macvlan parent interface (=the host network interface) is not allowed to communicate with macvlan child interfaces (=the container network interfaces).

6

Hi @cyberhead21,

some time ago, I have had similar issues with macvlan. I am running docker not via VirtualBox, but on a little PC in my network. There were 2 key learnings: the ethernet adaptor of the PC needed “promiscuous mode” as well and in my Unifi Switch, i needed to active a “trunk port”, so that all VLAN-IDs are available to the ethernet adapter of the PC.

This might help - or confuse you :slight_smile: : Ipvlan or macvlan in docker-compose.yml

Now, 3 years down the road, I would do my setup differently. I would not use macvlans, but I would use my pihole as DNS and the nginx proxy manager to route the traffic to the docker internal networks of my stacks. This way, SSL is much easier to achieve.