Docker Community Forums

Share and learn in the Docker community.

UDP IPv6 return traffic coming from different IP address

I’ve found an odd behavior with an UDP ports over IPv6 in docker. I’m not entirely sure where to start, so I’ll share my observations, a test setup to recreate the issue.

Any help is appreciated, thank you.

  • I’m running a pihole image, but I believe that this issue is likely not specific to this image.
  • The docker host has multiple IPv6 addresses. This is very common in IPv6. One static well-known, but also a privacy address.
  • When observing with tcpdump on the host NIC, I see:
    • DNS queries (being served by the container) show src=Laptop, dst=Address1.
    • DNS responses (being generated by the container) show src=Address2, dst:Laptop.
    • The reply source does not match the request destination.
  • Using host networking fixes the problem.

My expectation is that DNS responses would come back from the address where the query originated. I suspect something is going on with iptables/nftables where the session state isn’t getting tracked quite right.

Currently my work-around is to use host networking, which is OK since this device is single-purpose.

Is anyone else experiencing this? Is this a bug? has it been reported / where would I report such a thing?

Any other information I need to provide.

The closest thing I’ve found:

Test Setup

docker-compose.yml:

version: "3"

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    #network_mode: host
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      TZ: 'UTC'
    volumes:
       - './etc-pihole/:/etc/pihole/'
       - './etc-dnsmasq.d/:/etc/dnsmasq.d/'
    dns:
      - 127.0.0.1
      - 1.1.1.1
      - 8.8.8.8
#    cap_add:
#      - NET_ADMIN
    restart: unless-stopped

Testing Commands

Docker Host

cat /etc/debian_version
uname -a
docker --version
cat /etc/network/interfaces
docker-compose up -d
ip -6 addr
tcpdump -ni eth0 -s0 -w pihole.pcap 'udp port 53 and ip6'
[CTRL-C after test is done]
tcpdump -rn pihole.pcap

Laptop:

dig google.com @REDACTED_PREFIX::2

Output

Docker Host

root@pihole:~/pihole# cat /etc/debian_version
10.11

root@pihole:~/pihole# uname -a
Linux pihole 5.4.157-1-pve #1 SMP PVE 5.4.157-1 (Mon, 29 Nov 2021 12:01:44 +0100) x86_64 GNU/Linux

root@pihole:~/pihole# docker --version
Docker version 20.10.12, build e91ed57

root@pihole:~/pihole# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
	address 192.168.42.2/24
	gateway 192.168.42.1

iface eth0 inet6 static
	address REDACTED_PREFIX::2/64

	autoconf 1
	accept_ra 2

root@pihole:~/pihole# docker-compose up -d
Recreating pihole ... done

root@pihole:~/pihole# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 REDACTED_PREFIX:e44d:17ff:fee2:fe14/64 scope global dynamic mngtmpaddr
       valid_lft 4294967218sec preferred_lft 4294967218sec
    inet6 REDACTED_PREFIX::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::e44d:17ff:fee2:fe14/64 scope link
       valid_lft forever preferred_lft forever
3: br-1b31b053eca8: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 state DOWN
    inet6 fe80::42:77ff:fe99:a4cf/64 scope link
       valid_lft forever preferred_lft forever

root@pihole:~/pihole# tcpdump -nr pihole.pcap
14:06:30.193288 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.61003 > REDACTED_PREFIX::2.53: 57950+ [1au] A? google.com. (39)
14:06:30.193680 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.61003: 57950 1/0/1 A 142.251.32.14 (55)
14:06:35.208945 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.61003 > REDACTED_PREFIX::2.53: 57950+ [1au] A? google.com. (39)
14:06:35.209213 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.61003: 57950 1/0/1 A 142.251.32.14 (55)
14:06:40.196996 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.61003 > REDACTED_PREFIX::2.53: 57950+ [1au] A? google.com. (39)
14:06:40.197190 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.61003: 57950 1/0/1 A 142.251.32.14 (55)
14:06:48.995986 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.60422 > REDACTED_PREFIX::2.53: 10923+ [1au] AAAA? google.com. (39)
14:06:48.996965 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.60422: 10923 1/0/1 AAAA 2607:f8b0:4009:817::200e (67)
14:06:53.999137 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.60422 > REDACTED_PREFIX::2.53: 10923+ [1au] AAAA? google.com. (39)
14:06:53.999412 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.60422: 10923 1/0/1 AAAA 2607:f8b0:4009:817::200e (67)
14:06:58.996236 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.60422 > REDACTED_PREFIX::2.53: 10923+ [1au] AAAA? google.com. (39)
14:06:58.996510 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.60422: 10923 1/0/1 AAAA 2607:f8b0:4009:817::200e (67)
14:06:59.519148 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57473 > REDACTED_PREFIX::2.53: 9811+ A? fonts.gstatic.com. (35)
14:06:59.519172 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.60230 > REDACTED_PREFIX::2.53: 63866+ AAAA? fonts.gstatic.com. (35)
14:06:59.542746 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57473: 9811 2/0/0 CNAME gstaticadssl.l.google.com., A 172.217.4.35 (87)
14:06:59.544465 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.60230: 63866 2/0/0 CNAME gstaticadssl.l.google.com., AAAA 2607:f8b0:4009:806::2003 (99)
14:07:00.084832 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.59014 > REDACTED_PREFIX::2.53: 27305+ A? www.gstatic.com. (33)
14:07:00.085266 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57307 > REDACTED_PREFIX::2.53: 33187+ AAAA? www.gstatic.com. (33)
14:07:00.091946 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57937 > REDACTED_PREFIX::2.53: 42501+ A? lh3.googleusercontent.com. (43)
14:07:00.092277 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57961 > REDACTED_PREFIX::2.53: 39273+ AAAA? lh3.googleusercontent.com. (43)
14:07:00.104260 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57937: 42501 2/0/0 CNAME googlehosted.l.googleusercontent.com., A 142.250.191.225 (88)
14:07:00.105042 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57961: 39273 2/0/0 CNAME googlehosted.l.googleusercontent.com., AAAA 2607:f8b0:4009:808::2001 (100)
14:07:00.107953 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.59014: 27305 1/0/0 A 142.250.190.99 (49)
14:07:00.107973 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57307: 33187 1/0/0 AAAA 2607:f8b0:4009:819::2003 (61)
14:07:00.512608 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57473 > REDACTED_PREFIX::2.53: 9811+ A? fonts.gstatic.com. (35)
14:07:00.512839 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57473: 9811 2/0/0 CNAME gstaticadssl.l.google.com., A 172.217.4.35 (90)
14:07:00.546750 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.60230 > REDACTED_PREFIX::2.53: 63866+ AAAA? fonts.gstatic.com. (35)
14:07:00.546950 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.60230: 63866 2/0/0 CNAME gstaticadssl.l.google.com., AAAA 2607:f8b0:4009:806::2003 (102)
14:07:01.135683 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.59014 > REDACTED_PREFIX::2.53: 27305+ A? www.gstatic.com. (33)
14:07:01.135685 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57307 > REDACTED_PREFIX::2.53: 33187+ AAAA? www.gstatic.com. (33)
14:07:01.135918 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57937 > REDACTED_PREFIX::2.53: 42501+ A? lh3.googleusercontent.com. (43)
14:07:01.135919 IP6 REDACTED_PREFIX:f1fb:58f:633f:beed.57961 > REDACTED_PREFIX::2.53: 39273+ AAAA? lh3.googleusercontent.com. (43)
14:07:01.136016 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57307: 33187 1/0/0 AAAA 2607:f8b0:4009:819::2003 (61)
14:07:01.136036 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.59014: 27305 1/0/0 A 142.250.190.99 (49)
14:07:01.136060 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57937: 42501 2/0/0 CNAME googlehosted.l.googleusercontent.com., A 142.250.191.225 (109)
14:07:01.136099 IP6 REDACTED_PREFIX:e44d:17ff:fee2:fe14.53 > REDACTED_PREFIX:f1fb:58f:633f:beed.57961: 39273 2/0/0 CNAME googlehosted.l.googleusercontent.com., AAAA 2607:f8b0:4009:808::2001 (121)

Laptop:

% dig google.com @REDACTED_PREFIX::2

; <<>> DiG 9.10.6 <<>> google.com @REDACTED_PREFIX::2
;; global options: +cmd
;; connection timed out; no servers could be reached