IPv6 working but unreliable for MACVLAN /64 subnet

Help and thank you!! Long read - but, might help you get into IPv6 in your projects, and let you know about a pitfall that I’ve found.

After exploring homelabs I have quite a Homelab setup with CrowdSec / Traefik / Nginx+PHP / BIND / Pihole / etc. But adding IPv6 has provided an interesting learning experience, and I’m pretty sure I’ve found a bug, or space for a feature request.

The BIND is for my local domain, and allows me to configure DNS/FQDN to both IPv4 and IPv6. I can pull the DHCP4/DHCP6 Lease, ARP, and NDP info via API from my network to build hostname/fqdn ↔ IP features in BIND for my inner network. What I noticed is, stuff on IPv6 would stop resolving suddenly. The container is running, what’s going on?

The Docker Network is setup ahead of time because I have multiple Stacks that I run on my multiple Zones (multiple VMs), and have found this to be ideal, especially w/IPv4.

If you attempt to use the same subnet and deliver the IPAM network setup from a Docker Compose, you will not be able to join the network, and/or, the network is delivered by one stack, and used by many, not an ideal state.

Here’s the script that sets up the Docker MACVLAN Network, this secondary NIC does not have an IP of its own as far as the VM is concerned (netplan - DHCPv4/DHCPv6 off, optional: true), but the Docker Engine on that host has the network connect to it:

docker network create \
    --driver=macvlan \
    --subnet=10.50.40.0/24 --gateway=10.50.40.1 --ip-range=10.50.40.64/27 \
    --ipv6 --subnet=2104:120a:3478:b207::/64 --gateway=2104:120a:3478:b207:237:12ab:3412:90cd \
    --attachable \
    -o parent=eth1 -o macvlan_mode=bridge \
    docker50network

MACVLAN means the Container has the IP, so I don’t have to Port forward, and have no use for Ports and can just Expose, but likely do not need to. Also, I am using a unique MAC address pool that is set in Docker for Docker Container assignments - this was a recent enhancement to attempt to limit the space the extra IPv6 addresses showed up in, as usually it was a random MAC address.

Now, here is one of my stacks that uses an IPv6 Address:

services:
  webserver:
    container_name: hosting
    image: nginx:latest
    hostname: hosting
    domainname: novalabs.home
    extra_hosts:
      - "hosting:127.0.0.1"
      - "hosting.homelab.home hosting:10.10.100.85"
    networks:
      docker50network:
        mac_address: 02:42:ac:34:42:11
        ipv4_address: 10.50.40.67
        ipv6_address: 2104:120a:3478:b207::6
        aliases:
          - hosting.homelab.home
    expose:
      - '80'
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - ./nginx/snippets/realip.conf:/etc/nginx/snippets/realip.conf
      - ./nginx/snippets/redirect.conf:/etc/nginx/snippets/redirect.conf
      - ./www:/var/www/html:ro
      - /etc/localtime:/etc/localtime:ro
    environment:
      UID: 1000
      GID: 1000
    entrypoint:
      - /bin/sh
      - -c
      - |
        usermod -u $${UID} nginx
        groupmod -g $${GID} nginx
        /docker-entrypoint.sh nginx -g 'daemon off;'
    healthcheck:
      test: ["CMD", "nginx", "-t"]
      interval: 20s
      timeout: 5s
      retries: 10
    restart: unless-stopped

networks:
  docker50network:
    external: true

What will happen is after a reboot of the container, it will soon show up with a new IPv6 address, and what’s possibly worse is, you cannot find it in Portainer/Docker, just your network stack. Usually this is with a new MAC that you did not specify. Further, as briefly mentioned above, there will likely also show up an extra IPv6 in addition to the one you requested, that is flavored at the end similar to the MAC address specified.

Looking for any tips or suggestions or how I could float this up as a bug, it is great we get all this surface IAC (infrastructure as code <3) to build but when the objects deviate from that, then firewalls don’t allow IPs/etc.

Hope the story/read helps someone else, definitely looking for help because having to stop and remove all containers on a reboot/shutdown-turn-on or no more container-restart, container redeploy - as on an initial deploy, it seems to “keep to” its IPv6 address(es). Read notes that this might be a Docker Compose bug more than a Docker bug, and that it might be due to not having the Network IPAM being part of the Docker Compose. As in, if you ‘attach’ to a Network, you might keep the initial MAC+IPv6, but expect it to change on container restarts. If you only have one stack on a VM, you could deploy to a subnet IPv4+IPv6 via the IPAM Network options in Docker Compose, but you will be absorbing a whole subnet, and you will have to do that in all your stacks… I have 40+ stacks, not an easy lift. Plus I would have to Route those and each VLAN via the Router, again, quite a lot of setup to run multiple stacks that would otherwise sit very easily next to each other in the same subnet/zone.

Will repeat it, my method of network with Docker Compose has certainly lead to the thought that pre-deploying the network and attaching to it is best in the homelab and likely many smaller business cases. We are not trying to swarm and serve billions of packets, just a small service that in some cases does a great thing.

If there is a better way to do the IPv4 and IPv6 deployment to in some cases 20+ containers per VM (about 10 stacks per VM) and have them all with an actual IP on them that I specify that does not change/get-added, I’m all ears.

Maybe it is just my brain not working, but it was hard to follow your “story” as you called it. Can you summarize what the main problem is? Is it that the ipv6 address changes when you restart a container? Or when you reboot the host machine?

There is no such thing as rebooting a container. You don’t reboot a process and a container is basically a process on the host, not a virtual machine.

The containers will more or less only keep the specified IP in the docker-compose.yml on initial deploy.

If containers are restarted, or host running containers is rebooted (containers are effectively restarted, not an initial deploy) then the container(s) will get a new IPv6 address. They might also keep the old one, but, they will generally get a new one.

To get around this I’ve found a way to set the MAC address, and re-set/use the IPv6 it gets on initial deploy on the container so it does not change and this seems to have resolved most of the issue, the rest of it was a start up service that deploys all stacks/containers and a shut down service that stop && removes all containers. This ensures all containers are deployed new and with their IPv6/MAC cohesion intact.

Using the MAC address space that I have seems to have a specific pattern affect on the IPv6 address, after the subnet mask, and seems to allow me to forecast what IPv6 address the host will get but generally I go back through and update according to what they get when they deploy, and as they re-deploy they keep it.

I’m realizing I did not illustrate this on my example IPv6 address in the example, this pattern of MAC ↔ IPv6 address, so my appologies, but it would probably look a little like this instead:

2104:120a:3478:b207:42ac:3442:111:c6a8

I could not reproduce the issue. I restarted the container, rebooted the host VM, and I still have the same IP address. Even without setting the mac address. My Docker CE version is 28.1.1 in the test VM. What is your version? And how did you install Docker?