Help and thank you!! Long read - but, might help you get into IPv6 in your projects, and let you know about a pitfall that I’ve found.
After exploring homelabs I have quite a Homelab setup with CrowdSec / Traefik / Nginx+PHP / BIND / Pihole / etc. But adding IPv6 has provided an interesting learning experience, and I’m pretty sure I’ve found a bug, or space for a feature request.
The BIND is for my local domain, and allows me to configure DNS/FQDN to both IPv4 and IPv6. I can pull the DHCP4/DHCP6 Lease, ARP, and NDP info via API from my network to build hostname/fqdn ↔ IP features in BIND for my inner network. What I noticed is, stuff on IPv6 would stop resolving suddenly. The container is running, what’s going on?
The Docker Network is setup ahead of time because I have multiple Stacks that I run on my multiple Zones (multiple VMs), and have found this to be ideal, especially w/IPv4.
If you attempt to use the same subnet and deliver the IPAM network setup from a Docker Compose, you will not be able to join the network, and/or, the network is delivered by one stack, and used by many, not an ideal state.
Here’s the script that sets up the Docker MACVLAN Network, this secondary NIC does not have an IP of its own as far as the VM is concerned (netplan - DHCPv4/DHCPv6 off, optional: true), but the Docker Engine on that host has the network connect to it:
docker network create \
--driver=macvlan \
--subnet=10.50.40.0/24 --gateway=10.50.40.1 --ip-range=10.50.40.64/27 \
--ipv6 --subnet=2104:120a:3478:b207::/64 --gateway=2104:120a:3478:b207:237:12ab:3412:90cd \
--attachable \
-o parent=eth1 -o macvlan_mode=bridge \
docker50network
MACVLAN means the Container has the IP, so I don’t have to Port forward, and have no use for Ports and can just Expose, but likely do not need to. Also, I am using a unique MAC address pool that is set in Docker for Docker Container assignments - this was a recent enhancement to attempt to limit the space the extra IPv6 addresses showed up in, as usually it was a random MAC address.
Now, here is one of my stacks that uses an IPv6 Address:
services:
webserver:
container_name: hosting
image: nginx:latest
hostname: hosting
domainname: novalabs.home
extra_hosts:
- "hosting:127.0.0.1"
- "hosting.homelab.home hosting:10.10.100.85"
networks:
docker50network:
mac_address: 02:42:ac:34:42:11
ipv4_address: 10.50.40.67
ipv6_address: 2104:120a:3478:b207::6
aliases:
- hosting.homelab.home
expose:
- '80'
volumes:
- ./nginx/conf.d:/etc/nginx/conf.d:ro
- ./nginx/snippets/realip.conf:/etc/nginx/snippets/realip.conf
- ./nginx/snippets/redirect.conf:/etc/nginx/snippets/redirect.conf
- ./www:/var/www/html:ro
- /etc/localtime:/etc/localtime:ro
environment:
UID: 1000
GID: 1000
entrypoint:
- /bin/sh
- -c
- |
usermod -u $${UID} nginx
groupmod -g $${GID} nginx
/docker-entrypoint.sh nginx -g 'daemon off;'
healthcheck:
test: ["CMD", "nginx", "-t"]
interval: 20s
timeout: 5s
retries: 10
restart: unless-stopped
networks:
docker50network:
external: true
What will happen is after a reboot of the container, it will soon show up with a new IPv6 address, and what’s possibly worse is, you cannot find it in Portainer/Docker, just your network stack. Usually this is with a new MAC that you did not specify. Further, as briefly mentioned above, there will likely also show up an extra IPv6 in addition to the one you requested, that is flavored at the end similar to the MAC address specified.
Looking for any tips or suggestions or how I could float this up as a bug, it is great we get all this surface IAC (infrastructure as code <3) to build but when the objects deviate from that, then firewalls don’t allow IPs/etc.
Hope the story/read helps someone else, definitely looking for help because having to stop and remove all containers on a reboot/shutdown-turn-on or no more container-restart, container redeploy - as on an initial deploy, it seems to “keep to” its IPv6 address(es). Read notes that this might be a Docker Compose bug more than a Docker bug, and that it might be due to not having the Network IPAM being part of the Docker Compose. As in, if you ‘attach’ to a Network, you might keep the initial MAC+IPv6, but expect it to change on container restarts. If you only have one stack on a VM, you could deploy to a subnet IPv4+IPv6 via the IPAM Network options in Docker Compose, but you will be absorbing a whole subnet, and you will have to do that in all your stacks… I have 40+ stacks, not an easy lift. Plus I would have to Route those and each VLAN via the Router, again, quite a lot of setup to run multiple stacks that would otherwise sit very easily next to each other in the same subnet/zone.
Will repeat it, my method of network with Docker Compose has certainly lead to the thought that pre-deploying the network and attaching to it is best in the homelab and likely many smaller business cases. We are not trying to swarm and serve billions of packets, just a small service that in some cases does a great thing.
If there is a better way to do the IPv4 and IPv6 deployment to in some cases 20+ containers per VM (about 10 stacks per VM) and have them all with an actual IP on them that I specify that does not change/get-added, I’m all ears.