CentOS container DNS problem

Greetings!
I have a trouble with DNS system at a private docker network after installing some third-party bash-script (into container).

Entry:
My host system is CentOS 7.6, docker version - 18.09.6, build 481bc77156
The basic image, used to deploy container, is https://hub.docker.com/_/centos/
(some extra Dockerfile instructions were applyed also to enable systemd:

FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]

)

The container run with such parameters:

docker run -dit --name="mycontainer" --restart always --privileged --net mynetwork -p 22004:22 --expose 80 --expose 443 local/centos-systemd:my_tag

After this dns works correctly (at a container-level):

[root@3e9a75147f38 /]# ping google.com
PING google.com (108.177.14.101) 56(84) bytes of data.
64 bytes from lt-in-f101.1e100.net (108.177.14.101): icmp_seq=1 ttl=47 time=4.71 ms
64 bytes from lt-in-f101.1e100.net (108.177.14.101): icmp_seq=2 ttl=47 time=5.12 ms
64 bytes from lt-in-f101.1e100.net (108.177.14.101): icmp_seq=3 ttl=47 time=5.10 ms
64 bytes from lt-in-f101.1e100.net (108.177.14.101): icmp_seq=4 ttl=47 time=5.21 ms

But once I have installed some extra web environment by means of third-party bash script, the domain resolving into container fails:

[root@docker /]# ping google.com
ping: google.com: Name or service not known

At the file /etc/resolv.conf nothing changed:

[root@docker /]# cat /etc/resolv.conf
nameserver 127.0.0.11
options ndots:0

By ip-addresses ping works correctly:

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=46 time=4.59 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=46 time=4.66 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=46 time=4.62 ms

Some additional data:

# docker inspect network mynetwork
<...>
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
<...>

Before bash script installed:

[root@3e9a75147f38 /]# traceroute google.com
traceroute to google.com (173.194.73.100), 30 hops max, 60 byte packets
 1  gateway (172.18.0.1)  0.038 ms  0.021 ms  0.020 ms
 2  10.112.119.1 (10.112.119.1)  0.451 ms  0.572 ms  0.660 ms
 3  121.spb.net.selectel.ru (188.93.17.121)  1.069 ms  1.033 ms  1.018 ms
 4  62.spb.net.selectel.ru (188.93.17.62)  1.308 ms  1.286 ms  1.127 ms
 5  188.93.16.145 (188.93.16.145)  6.547 ms  6.527 ms  6.543 ms
 6  74.125.244.181 (74.125.244.181)  1.342 ms 74.125.244.133 (74.125.244.133)  1.217 ms 74.125.244.180 (74.125.244.180)  1.173 ms
 7  216.239.42.53 (216.239.42.53)  5.326 ms  5.940 ms 216.239.42.85 (216.239.42.85)  4.915 ms
 8  209.85.249.173 (209.85.249.173)  5.050 ms  5.021 ms  4.799 ms
 9  64.233.175.175 (64.233.175.175)  4.784 ms 72.14.236.115 (72.14.236.115)  4.941 ms  4.754 ms
[root@3e9a75147f38 /]# dig google.com
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> google.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1288
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;google.com.                    IN      A

;; ANSWER SECTION:
google.com.             197     IN      A       173.194.73.100
google.com.             197     IN      A       173.194.73.139
google.com.             197     IN      A       173.194.73.138
google.com.             197     IN      A       173.194.73.102
google.com.             197     IN      A       173.194.73.113
google.com.             197     IN      A       173.194.73.101

;; Query time: 2 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Tue Jun 11 02:49:37 UTC 2019
;; MSG SIZE  rcvd: 135

After bash script installed:

[root@docker /]# traceroute google.com
google.com: Name or service not known
Cannot handle "host" cmdline arg `google.com' on position 1 (argc 1)
[root@docker /]# dig google.com

; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.1 <<>> google.com
;; global options: +cmd
;; connection timed out; no servers could be reached

Other containers, placed into this private network, works with DNS correctly until this bash script is not installed.
This script installs a lot of data and services (web environment with nginx, httpd, mysql, nodejs etc), I can’t debug it and find out a concrete service, that break the DNS.

Any ideas of how to fix this dns issue?

1 Like

I’m facing a similar issue where my first dig call works but then subsequent dig calls tells me no servers could be reached.

Did you resole this?