I’m developer and use some project-stacks. To be able to reach all projects using wildcard-domain (foo.bar.docker) I’ve used nginx with stream-proxy to docker-dns (127.0.0.11) for a while now. But it seems to behave laggy as nginx was not very good in forwarding udp to dns-upstream of docker.
My setup is as follows:
I have a general docker-compose stack that f.e. includes dnsmasq-service. It has an external network I’ve named “dns-tier”. I assign this network to other stacks where I need to reach a service. For the hostnames I use docker-compose “hostname” and network-aliases.
This works fine so far by using the docker internal dns! But the docker-dns sometimes is laggy and it’s not simply possible to expose it to the host-computer running docker. There came nginx in place forwarding 127.0.0.11:53 from host into the docker world.
So I decided to give dnsmasq a try. Usually I remember it should have worked fine. I’ve drilled down everything to this small config:
no-daemon
quiet-dhcp6
domain-needed
bogus-priv
#no-resolv
no-hosts
#dns-loop-detect
#user=root
# Set size of DNS cache
# Note: Large cache size can affect performance
cache-size=1500
neg-ttl=10
local-ttl=10
max-cache-ttl=10
#no-negcache
# Expand hosts using 'domain' set above.
expand-hosts
# Log to /dev/stdout
log-facility=-
# !!! Debugging only !!!
log-queries
strict-order
addn-hosts=/etc/dnsmasq.misc/resolvers.conf
local=/docker/
server=/.docker/127.0.0.11
server=/.projects.docker/127.0.0.11
server=8.8.8.8
server=8.8.4.4
server=2001:4860:4860::8888
server=2001:4860:4860::8844
My docker-service is started with this Command:
dnsmasq -kd -C /etc/dnsmasq/main.conf
So far so good! At first it looked everything fine. But today I’ve noticed some wildcard-domains are NOT being resolved anymore:
I have a docker-compose stack for a web project with this hostname/aliases for the service:
hostname: website.projects.docker
aliases:
lp1.website.projects.docker
lp2.website.projects.docker
It worked yesterday like this:
$ host website.projects.docker → returns correct IP
$ host lp1.website.projects.docker → returns correct IP
$ dig website.projects.docker @127.0.0.11 → returns correct IP
$ dig lp1.website.projects.docker @127.0.0.11 → returns correct IP
But today… only resolving “website.projects.docker”. I’ve been restarting the dnsmasq-service, the whole stack containing it and same for the website-project docker-stack!
The interesting is… I’ve enabled logging to stdout and the logs of dnsmasq queries/resolving indicate that dnsmasq knows the correct IP of lp1.website.projects.docker but the dns-response says it doesn’t.
Hi thanks for trying to help. In the meantime the dns-resolution against docker doesn’t work anymore for any names. I fiddlet around the config testing if there is something wrong but even after changing everything back (and restarting docker and containers) it’s not working.
I’m using normal docker without swarm (only local development).
I’ve commented out the config for addn-hosts in the meantime as for testing I’m using only the config here and even commented out google-dns in hope that may fix it.
I want just to simply forward everything (or all *.docker if possible) to the docker internal dns. Cross-Testing in other containers they are able to resolv the hostnames correctly using dig or host command.
This way I was able to cross-link any service/container using the dns-tier network into my docker-compose stack where my “docker-dns” (dnsmasq and before that nginx stream-proxy) was running. Both of them are configured to forward to 127.0.0.11 which is the docker internal dns (same as exposed by /etc/resolv.conf).
I’ve been using the nginx dns-proxy solution for about 2 month now! So it’s ridiculous this stops working from now to then without any obvious reason.
Testing on CLI of the docker-dns service it still can resolve the hostname of my httpd container:
dig website.projects.docker (optionally with @127.0.0.11)
But same for network-alias (bound to dns-tier network) doesn’t work anymore!
lp1.website.projects.docker
Just for notice … as this setup worked (before docker-/compose updates) for weeks now, I believe there must be any bug?!
I’ve opened an issue on github docker/compose hoping there anyone might have a solution or can confirm this worked once and what could be the reason for this:
accidently in one of the docker-compose networks I’ve had the mistake to use a different name-attribute for the “dns-tier” network. This obviously created two different networks so that the stack with my dns and the project not had the same network. But for some reason I was able to resolve the service.hostname of the other stack services anyway.
I’ve added these config changes to my docker-dns service for docker-compose.yml. Not shure one of these made the rest working, but I’m not going to touch it again now!:
networks:
dns-tier:
name: dns-tier #<--- keep an eye this must match in any stuck the network is being used?!
external: true
driver_opts:
"com.docker.network.driver.mtu": 1350 #<--- added this as mentioned in one of gitlab issues for docker-compose for similar issue
services:
docker-dns:
restart: unless-stopped
hostname: dns.docker
cap_add:
- NET_ADMIN #<--- added this
extends:
service: docker-dns
file: ./dnsmasq-proxy/docker-compose.dnsmasq.yaml
ports:
- "${DOCKER_HOST_IP:-127.0.0.11}:${HOST_DNS_PORT:-53}:53/udp"
- "${DOCKER_HOST_IP:-127.0.0.11}:${HOST_DNS_PORT:-53}:53/tcp"
networks:
dns-tier:
and for the project service-container I’ve added this ``dns: [“dns.docker”]```. Even there not shure its optional. But never touch a running system.
In the meantime I’ve tested some other dockerized dns-services (f.e. ageddo/dns-proxy-server) which also looks promising. I’ve tested this one too, after it obviously worked again. Surprisingly… this one was not able to resolve only hostnames and not the docker network-alias names for some reason. I don’t know why!
May be it helps someone else getting stuck on this … for me … the issue is ready to be closed now, I think.