DNS and Internet in Container Can Only be Accessed when Container Run -u root

Docker Engine General
1

Hi, recently I cleaned out and my docker home directory to free up some space, upgraded docker to 20.10.17-ce ran docker and re-installed some containers (sonatype/nexus3, nginx). Before I reinstalled and upgraded, the containers had no issues communicating with the internet / resolving external DNS. Now the containers will only resolve DNS / communicate to the Internet if I run the container -u root (as root user). The other related related symptom is that /etc/resolv.conf can only be read when you enter the containers as -u root - otherwise regardless of file permissions / ownership the container says file not readable. I tried explicitly setting --dns on container startup, changing between host and bridge network and other tricks, but only running as -u root would cause Internet connectivity / DNS to start working - and this is consistent across varied containers (a base Centos container, Nexus, …).

I am running Docker 20.10.17-ce on SLES 12 SP 5 using devicemapper with a mapped file (that I upsized using dd). This basic configuration has worked for years through multiple updates / rebuilds of docker home, and still works on other hosts.

Here is my Docker info below:

Server Version: 20.10.17-ce
 Storage Driver: devicemapper
  Pool Name: docker-0:46-885781-pool
  Pool Blocksize: 65.54kB
  Base Device Size: 21.47GB
  Backing Filesystem: xfs
  Udev Sync Supported: true
  Data file: /dev/loop0
  Metadata file: /dev/loop1
  Data loop file: /opt/itds/cicd/docker/devicemapper/devicemapper/data
  Metadata loop file: /opt/itds/cicd/docker/devicemapper/devicemapper/metadata
  Data Space Used: 25.22GB
  Data Space Total: 187.9GB
  Data Space Available: 162.7GB
  Metadata Space Used: 51.68MB
  Metadata Space Total: 2.147GB
  Metadata Space Available: 2.096GB
  Thin Pool Minimum Free Space: 18.79GB
  Deferred Removal Enabled: true
  Deferred Deletion Enabled: true
  Deferred Deleted Device Count: 0
  Library Version: 1.03.01 (2021-05-07)
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux oci runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 31aa4358a36870b21a992d3ad2bef29e1d693bec
 runc version: v1.1.4-0-ga916309fff0f
 init version:
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 4.12.14-122.113-default
 Operating System: SUSE Linux Enterprise Server 12 SP5
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 15.49GiB
 Docker Root Dir: /opt/itds/cicd/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
2

I edited your post, please, use code blocks for sharing codes and terminal outputs.

I found this for you

OS requirements

To install Docker Engine, you need a maintained version of SLES 15-SP3 or SLES 15-SP4 on s390x (IBM Z). Archived versions aren’t supported or tested.

It is possible that your OS is too old for the latest available Docker, although I don’t know why such a non-supported version would be installable.

resolv.conf is mounted from the host from special folders. docker container inspect can tell you the source path if you want to investigate why that file is not readable for the container without root user.

3

Thanks rimelek, did you intend to include a link about Docker Hub Two Factor Authentication? I am unclear how that relates to the container DNS issue. I checked the /etc/resolv.conf file as suggested in the host filesystem, but no real clues there - it is permission 777 owned by nobody, but still not readable by non-root in the container. As far as the versioning considerations, I have the same and newer versions of Docker from SLES 12 standard package distributions on other hosts and do not see this issue (I can both read the resolv.conf and resolve outside addresses as non-root user). Any other thoughts would be appreciated.

Thanks,
Phil

4

No, I am sorry, I must have pasted the wrong link when I came back to this topic after I posted in another. I fixed the link, thanks.

I know nothing about SLES, so I have no idea what could cause this. A file inside the container is owned by “nobody” user and “nogroup” when it is mounted from the host and usernamespaces are enabled or Docker is actually rootless docker. However, I don’t know if it is normal on OpenSUSE or not.

Check the output of

docker context ls

Did you install Docker on these different machines the same way?