No Such Host: Docker daemon can't access kubernetes registry but wget on the same node can connect to the registry

Docker Engine General
1

Hi,

I have an Alpine Linux based container on a single node kubernetes cluster(for testing). I have a private docker registry installed within my cluster at docker-registry.default:5000 . I can login to the alpine node and use wget and access my private docker registry.

kubectl exec -it pod/nuclio-dashboard-5c5c48947b-lpgx8 -- /bin/sh
/ # wget -qO- https://docker:mypassword@docker-registry.default:5000/v2/_catalog
{"repositories":["nuclio/processor-helloworld3"]}

But I can’t seem to access it using docker on the same pod. Both Client and Server are 2019 builds

kubectl exec -it pod/nuclio-dashboard-5c5c48947b-lpgx8 -- /bin/sh

/ # which docker
/usr/local/bin/docker
/ # docker login -u docker -p mypassword docker-registry.default:5000
Error response from daemon: Get https://docker-registry.default:5000/v2/: dial tcp: lookup docker-registry.default on 169.254.169.254:53: no such host

I can logon to the Docker Hub registry.

docker login -u my_hub_user  -p my_hub_password
Login Succeeded

I have tried to simplify the problem but the original problem is that kubernetes cannot pull an image from my private kubernetes registry and it gives me the same error.

Normal   Scheduled  7m52s                   default-scheduler                                       Successfully assigned nuclio/helloworld1-96876fcf9-7c87w to gke-your-first-cluster-1-pool-1-fe915942-gnkx
  Normal   Pulling    6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Failed to pull image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest": rpc error: code = Unknown desc = Error response from daemon: Get https://docker-registry.default:5000/v2/: dial tcp: lookup docker-registry.default on 169.254.169.254:53: no such host
  Warning  Failed     6m14s (x4 over 7m51s)   kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ErrImagePull
  Normal   BackOff    6m2s (x6 over 7m51s)    kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Back-off pulling image "docker-registry.default:5000/docker/nuclio/processor-helloworld1:latest"
  Warning  Failed     2m38s (x21 over 7m51s)  kubelet, gke-your-first-cluster-1-pool-1-fe915942-gnkx  Error: ImagePullBackOff

If I port forward to the private kubernetes docker registry from my machine and I do a pull it works.

Thank you for any pointers you can give me.

2

This helped me: https://stackoverflow.com/questions/59521002/no-such-host-docker-daemon-cant-access-kubernetes-registry-but-wget-on-the-sam solve the problem.