Unblocking Unbound, docker networks and bridges, (also pihole)

I have unbound container mvance/unbound, and am not able to get it to reach out of the container to query root servers.

It starts and reaches ‘healthy’. The docker compose also starts a pihole container (pihole/pihole, version 6+), and a network between them. The aim is to receive home queries, filter at pihole, then recurse privately with unbound.

Host port 53 is open on ufw, and resolved is stopped. Port 53 is passed through to pihole, and it shown receive local and network traffic. Docker network ipam is configured, and pihole dns is set to unbound_ip@port. I set unbound logs to docker logs and I can see it does get pihole traffic. Reaching out to root servers, it fails all attempts (default cloudflare or root servers) pretty instantly. Does the private docker network need some means of getting to the default bridge? I tried some options but all rejected by docker.

I have been running docker about a year, and using searxng and paperlessngx. It is on a ubuntu server. I have worked (a tiny way) through Udemy Docker course by Bret Fisher.

Been banging my head on this about 3 hours a day for a week. Helpful though they are, the chatbots are talking me in a loop.

I would appreciate a little direction, what would you like me to share?

Did you try execing into the unbound container, and try to run nslookup for arbitrary domains?

If the unbound image comes without nslookup, you can use a troubleshooting container, like netshoot, and start it joined to the network namespace of the unbound container.

Furthermore , it wouldn’t hurt to share the compose file as is, of course with redacted password (if any exist in the compose file). This allows us to see whether something is configured, but needs to be configured differently, or something is missing and needs to be configured additionally.

Thanks for the quick reply and the pointers.

Yes unbound container is so stripped down (5mb alpine linux?) it does not even have ping. But I can ping the unbound container from the pihole container.

Below is my docker-compose.yml and also the unbound.conf. I am fairly happy with the docker-compose.yml, but I will admit to a some cargo-culting for the unbound.conf. I also added the output of docker network inspect. I put in the (concatenated) tail of the docker logs, (verbosity = 5), and added the output of some docker exec.

docker-compose.yml

services:
  unbound:
    container_name: unbound
    image: mvance/unbound:latest
    restart: unless-stopped
    volumes:
      - "./unbound/unbound.conf:/opt/unbound/etc/unbound/unbound.conf"
      - "./unbound/hosts:/opt/unbound/etc/unbound/hosts"
      - "./unbound/root.hints:/opt/unbound/etc/unbound/root.hints"
    networks:
      dns_network:
        ipv4_address: 172.20.0.2 # Optional static IP (if you define subnet below)

  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    restart: unless-stopped
    ports:
      - "53:53/tcp"
      - "53:53/udp"
        #- "67:67/udp" for dhcp
      - "8081:80/tcp"
        #- "443:443/tcp"
    environment:
      TZ: "Europe/London"
      WEBPASSWORD:
        "change_this_password"
        #FTLCONF_webserver_api_password: 'temptemp'
      PIHOLE_DNS_:
        "172.20.0.2#53"

        # VIRTUAL_HOST: "192.168.178.52" needed if using an additional reverse proxy
    volumes:
      - "./pihole/etc-pihole:/etc/pihole"
      - "./pihole/etc-dnsmasq.d:/etc/dnsmasq.d"
      - "./pihole/etc-pihole/pihole-FTL.conf:/etc/pihole/pihole-FTL.conf"
        #    cap_add:
        #      - NET_ADMIN   needed if using dhcp.
    networks:
      dns_network:
        ipv4_address: 172.20.0.3

networks:
  dns_network:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/24

I think this is ok (bit messy). I get that you do not need to publish a port, if you are not expecting traffic ingress. Traffic ingress to this is LAN to host to pihole to unbound to external root servers. Am I wrong?

unbound.conf

server:
  # General settings
  directory: "/opt/unbound/etc/unbound"
  interface: 0.0.0.0@53
  port: 53
  do-ip4: yes
  do-ip6: no
  do-udp: yes
  do-tcp: yes

  # Access control
  access-control: fc00::/7 allow
  access-control: ::1/128 allow
  access-control: 127.0.0.1/32 allow
  access-control: 172.20.0.0/24 allow

  # Logging
  verbosity: 5
  logfile: "" # /var/log/syslog/unbound.log
  log-queries: yes
  log-replies: yes


  # Other
  root-hints: "/opt/unbound/etc/unbound/root.hints"
  # rrset-roundrobin: yes # not needed for home setup

# Forwarding
#forward-zone:
#  name: "."
#  forward-addr: 8.8.8.8
  #forward-addr: 1.1.1.1

I am much vaguer on my understanding here, I feel the problem could be in this file but I can’t put my finger on it.

docker network inspect

infra@nas04:~/dns-config$ docker network ls
NETWORK ID     NAME                     DRIVER    SCOPE
e58fd2e33d4f   bridge                   bridge    local
e179de44b850   dns-config_dns_network   bridge    local
63a887e57d36   host                     host      local
434d4fdcb1e2   none                     null      local
d7a65ba9e3b6   paperless_default        bridge    local
infra@nas04:~/dns-config$ docker network inspect dns-config_dns_network
[
    {
        "Name": "dns-config_dns_network",
        "Id": "e179de44b850261039f50ebda824c3dc3b9a599e5b49d09866b569bd6dcdac92",
        "Created": "2025-04-07T06:20:33.549305185Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.20.0.0/24"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "3ce68e33cc75dae6bde054632d1018f078467b7005a4353b80794f2aa96fe50c": {
                "Name": "pihole",
                "EndpointID": "9314ae72b9fae12eda5b7568992e4066be41b56a3e61d0bfffac5f954dad827a",
                "MacAddress": "02:42:ac:14:00:03",
                "IPv4Address": "172.20.0.3/24",
                "IPv6Address": ""
            },
            "82b3669077c14fc1a51ff65bb18c4d8ba43e907d8ccf0b945148946d3a7695e8": {
                "Name": "unbound",
                "EndpointID": "cd7d9a9bc45a576b40e3127fc4707198b324377706ae50e03088f98f2ded8d14",
                "MacAddress": "02:42:ac:14:00:02",
                "IPv4Address": "172.20.0.2/24",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "dns_network",
            "com.docker.compose.project": "dns-config",
            "com.docker.compose.version": "2.27.1"
        }
    }
]
infra@nas04:~/dns-config$ 

I think this matches the docker-compose.yml file

some errors from docker exec

infra@nas04:~/dns-config$ docker exec -it unbound ping pihole
OCI runtime exec failed: exec failed: unable to start container process: exec: "ping": executable file not found in $PATH: unknown
infra@nas04:~/dns-config$ docker exec -it pihole ping unbound
PING unbound (172.20.0.2): 56 data bytes
64 bytes from 172.20.0.2: seq=0 ttl=64 time=0.237 ms
64 bytes from 172.20.0.2: seq=1 ttl=64 time=0.236 ms
64 bytes from 172.20.0.2: seq=2 ttl=64 time=0.249 ms
64 bytes from 172.20.0.2: seq=3 ttl=64 time=0.257 ms
^C
--- unbound ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.236/0.244/0.257 ms
infra@nas04:~/dns-config$ docker exec -it pihole dig @unbound google.com
;; communications error to 172.20.0.2#53: timed out
;; communications error to 172.20.0.2#53: timed out
;; communications error to 172.20.0.2#53: timed out

; <<>> DiG 9.18.35 <<>> @unbound google.com
; (1 server found)
;; global options: +cmd
;; no servers could be reached
infra@nas04:~/dns-config$ 

The bit of ‘docker logs unbound’ that cursor.ai thinks is relevant

The log is huge and repeating, I wasn’t going to past it here

[1744009012] unbound[1:0] debug: query response was timeout
[1744009024] unbound[1:0] debug: query response was timeout
[1744009048] unbound[1:0] debug: query response was timeout

And using netshoot (thanks for this pointer)

infra@nas04:~/dns-config$ docker run -it --net container:unbound nicolaka/netshoot
...
 82b3669077c1  ~  ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
1814: eth0@if1815: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:ac:14:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.20.0.2/24 brd 172.20.0.255 scope global eth0
       valid_lft forever preferred_lft forever

 82b3669077c1  ~  ip route
default via 172.20.0.1 dev eth0 
172.20.0.0/24 dev eth0 proto kernel scope link src 172.20.0.2 

 82b3669077c1  ~  drill google.com
Error: error sending query: Could not send or receive, because of network error

 82b3669077c1  ~  ping 198.41.0.4
PING 198.41.0.4 (198.41.0.4) 56(84) bytes of data.
64 bytes from 198.41.0.4: icmp_seq=1 ttl=58 time=13.3 ms
64 bytes from 198.41.0.4: icmp_seq=2 ttl=58 time=13.6 ms
64 bytes from 198.41.0.4: icmp_seq=3 ttl=58 time=13.9 ms

^C
--- 198.41.0.4 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6010ms
rtt min/avg/max/mdev = 12.656/13.686/14.317/0.522 ms

 82b3669077c1  ~  drill @198.41.0.4 google.com        
Error: error sending query: Could not send or receive, because of network error

 82b3669077c1  ~  netstat -tulpn | grep 53
tcp        0      0 0.0.0.0:53              0.0.0.0:*               LISTEN      -
udp        0      0 0.0.0.0:53              0.0.0.0:*                           -

 82b3669077c1  ~  ping 198.41.0.4
PING 198.41.0.4 (198.41.0.4) 56(84) bytes of data.
64 bytes from 198.41.0.4: icmp_seq=1 ttl=58 time=14.6 ms
64 bytes from 198.41.0.4: icmp_seq=2 ttl=58 time=12.3 ms
64 bytes from 198.41.0.4: icmp_seq=3 ttl=58 time=13.6 ms
64 bytes from 198.41.0.4: icmp_seq=4 ttl=58 time=12.8 ms
^C
--- 198.41.0.4 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 12.317/13.324/14.586/0.868 ms

 82b3669077c1  ~  ip route        
default via 172.20.0.1 dev eth0 
172.20.0.0/24 dev eth0 proto kernel scope link src 172.20.0.2 

 82b3669077c1  ~  tcptraceroute 198.41.0.4 53
Selected device eth0, address 172.20.0.2, port 50567 for outgoing packets
Tracing the path to 198.41.0.4 on TCP port 53 (domain), 30 hops max
 1  * * *
 2  * * *
 3  * * * 

28  * * *
29  * * *
30  * * *
Destination not reached

 82b3669077c1  ~  

ICPM (ping) works
DNS (drill) fails
Traceroute fails.

Did we have the ports right?

My pet working theory [theory_serial_num: TK421 :slight_smile: ]:

Unbound is listening on port 53. Which port 53? We know it is on port 53 in the container, because that is default. What port does it use to DNS query the root servers? Since we have not published the ports, does it default to “host_port_N:container_port_N”, ie - the same number? If so then it is also listening on host port 53? This will be blocked, since pihole is already on port 53?

When we used ‘ports: - 5353:53/tcp’ and udp, for the unbound service, I seem to recall this did not fix this issue. Can try again though.

I found I had set some firewall forwarding rules, previously while experimenting, to forward for 53 to 5353. No traffic was reaching pihole. I only found this on sudo ufw show raw, which is giant volume of output. pihole and tcpdump got much busier.

I also did not have a rule that explicitly allowed a high random port out from unbound. I added an allow out rule.

Further pihole was only configured for A records and not SRV records and other DNS records, like MX. I modified setupVars.conf,
REV_SERVER=false
DNSMASQ_LISTENING=all
CACHE_SIZE=10000

This was much better for tcpdump and pihole traffic.

What I still cannot do is serve traffic to anything not on the host.

If I set a laptop to dirext DNS to this server, then using tcpdump, I can see it goes from host, to pihole container, to unbound container, then unbound sends out but nothing. Service on the host will get a root server reply through unbound. 2 different boxes on the network using pihole, despite their dns requests reaching unbound, still do not get root server DNS responses. Any thoughts?

Might my ISP provided router (Fritzbox 7530, from provider Zen in the UK) be blocking DNS traffic for a client which does not originate from the client?

Is DNS traffic from a third party considered suspicious? Might there be some ‘security feature’ which inspects the DNS packets?

I am just trying to learn here so the router is the main DNS for the home, I have 2 test machines trying to use the DNS service we set up here on a server (not the router) in docker. If I switch the DNS service off the router, might it work?

Will also annoy everyone in the house. Will try it in the middle of the night.

===

Had another go. My ISP was not blocking anything.

My post above (“Pihole IS getting LAN traffic”) was not fully accurate.

  • Using tcpdump I can see network client DNS requests DO come into the server, port 53.
  • Looking at the UFW rules, incoming DNS requests ARE allowed.
  • Reviewing docker-compose.yml port 53 IS mapped to pihole 53:53
  • Reviewing docker network inspect dns_network shows pihole IS mapped to port 53, tcp and udp.
  • Watching Pihole live logs from inside and outside the container while using dig and browsing on a LAN client, show only host traffic* going on in the pihole container (*tailscale, pihole, syncthing).
  • Host traffic and container traffic DOES go through pihole.
  • LAN client traffic DOES NOT get to pihole.
  • Using lsof we can see nothing else is bound to port 53,
  • Reviewing and adding more firewall rules to allow 53, does not allow LAN clients to dig or browse through pihole/unbound container stack.

This looks like a firewall rule issue but I don’t think so because:

  • Mapping unbound container to a separate external port (5335:53) (with ufw allow rules) DOES allow working dig from LAN clients.
  • Mapping pihole container to another port (5353:53) (with same ufw allow rules) DOES allow LAN client dig requests through pihole, and through unbound containers.

Up to this point I have not made it work.

So what else?

Docker is also running the paperlessngx container stack including redis, gotenburg, tika, broker and the webserver. The internal network of docker has 5 parts, but should it? It has bridge, dns_network, host, none, paperless_default. Is docker sending host port 53 to paperlessngx containers?

Googling about I get a sense there is something about multiple bridges colliding in docker. I don’t really understand the that issue, so I had a look through the output of docker network inspect for the bridge, paperless_default and dns_config. There was no subnet overlap. Comparing paperless_default (which works) to dns_network (which doesn’t), I could not see anything standout different.

Pihole in host mode.

So I set it to use pihole in host mode. And it works. Dig from LAN clients works, and browsing works. Tailing the logs in pihole, yes the LAN clients are not falling back to a secondary nameserver.

So this is a work around, not a fix. Pihole sends traffic to unbound by ?‘being outside the docker network’? and pouring traffic into host port 5335, and unbound is listening on that port (5335:53). Pihole has setting cap_add: - NET_ADMIN which means (I think) it is now trying to be a dhcp server - probably cause other issues later. I understand network_mode: "host" invites more problems too.

Up to this point I have not made it work the way it is supposed to. What am I not seeing? Thanks