Cannot access index.docker.io

Hi,
‘docker pull hello-world’ fails with:
Pulling repository docker.io/library/hello-world
Network timed out while trying to connect to https://index.docker.io/v1/repositories/library/hello-world/images. You may want to check your internet connection or if you are behind a proxy.

However, I have configured http_proxy and https_proxy as environment variables in the shell and also in /etc/sysconfig/docker. Further, wget of the above URL completes but returns just two characters: [] .

I also tried: ‘docker pull quay.io/aptible/hello-world’. Same issue. However, wget of that URL succeeds and returns valid content.

Please let me know what could be going wrong here.

Thanks,
Sundar

If you are on a systemd system, then /etc/sysconfig/docker may not be the correct place to configure an environment variable.

Here’s a tip to see what environment variables any given process on a system has:

cat /proc/<pid>/environ | sed 's/\x0/\n/g'

You can use that to validate that your init system is setting the variables you expect it to.

Usually in a systemd system, you’ll need to set the environment variables in the systemd unit or systemd unit override file. Details on doing that can be found here.

The official registry actually has nothing to do with index.docker.io. The actual registry address is going to be registry-2.docker.io instead. The official registry no longer runs the v1 registry, so any test to the v1 registry will definitely fail. The reason you’re seeing the v1 output in the error you’re getting is that it falls back to v1 after a v2 failure. This exact behavior can vary depending on the actual version of docker you’re running.

What’s the full output of docker version and docker info?

Thank you very much, Jeff.

With your pointers, I could verify that the http_proxy is set in the
dockerd process environment. Now, ‘docker run hello-world’ is able to pull
the hello-world image and proceed one step further.

Unfortunately, it is only one step further: it still fails with an error. I
should mention that, to fix error messages that docker-runc is not defined,
I have symlink’d: “/usr/bin/docker-runc -> /usr/bin/runc”, where runc
version is as below:

runc version 1.0.0-rc3
commit: b3fc8307fbabf7c95a1bed604c751e901b7553fe-dirty
spec: 1.0.0-rc5

The error output from ‘docker run hello-world’ is this:
<error_output>
Incorrect Usage.

NAME:
docker-runc create - create a container
<<… docker-runc man page … >>
flag provided but not defined: -console
/usr/bin/docker-current: Error response from daemon: oci runtime error:
flag provided but not defined: -console.
</error_output>

Here’s more information:
# docker version
Client:
Version: 1.12.6
API version: 1.24
Package version: docker-common-1.12.6-16.el7.centos.x86_64
Go version: go1.7.4
Git commit: 3a094bd/1.12.6
Built: Fri Apr 14 13:46:13 2017
OS/Arch: linux/amd64

Server:
Version: 1.12.6
API version: 1.24
Package version: docker-common-1.12.6-16.el7.centos.x86_64
Go version: go1.7.4
Git commit: 3a094bd/1.12.6
Built: Fri Apr 14 13:46:13 2017
OS/Arch: linux/amd64

# docker info
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 1
Server Version: 1.12.6
Storage Driver: overlay2
Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge overlay host null
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor seccomp
Kernel Version: 4.4.50-rt62-1704170124nfv
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 36
Total Memory: 62.88 GiB
Name: <>
ID: DE7S:L7PV:GWKR:L5A7:TOVK:RQ7E:NIIL:ABEA:Q3WH:MH2K:AUMT:NVWF
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 14
Goroutines: 23
System Time: 2017-04-24T02:04:28.871843722-07:00
EventsListeners: 0
Http Proxy: <>
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Insecure Registries:
127.0.0.0/8
Registries: docker.io (secure)

Here are the dockerd and containerd command lines:
/usr/bin/dockerd-current -H fd:// --storage-driver=overlay2 -D
/usr/bin/docker-containerd-current -l
unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
docker-containerd-shim --metrics-interval=0 --start-timeout 2m --state-dir
/var/run/docker/libcontainerd/containerd --runtime docker-runc --debug

Thank you & Regards,
Sundar

Though the ‘docker run hello-world’ command fails, the container is created but not running. The ‘docker ps -a’ shows multiple containers in Created state.

Looking at /var/log/messages, I don’t see where the ‘-console’ argument is passed. Here is the /var/log/messages file after running the command ( I cleared all previous entries):

Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.157501971-07:00” level=debug msg=“Calling POST /v1.24/containers/create"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.157814531-07:00” level=debug msg=“form data: {“AttachStderr”:true,“AttachStdin”:false,“AttachStdout”:true,“Cmd”:null,“Domainname”:””,“Entrypoint”:null,“Env”:[],“HostConfig”:{“AutoRemove”:false,“Binds”:null,“BlkioDeviceReadBps”:null,“BlkioDeviceReadIOps”:null,“BlkioDeviceWriteBps”:null,“BlkioDeviceWriteIOps”:null,“BlkioWeight”:0,“BlkioWeightDevice”:null,“CapAdd”:null,“CapDrop”:null,“Cgroup”:"",“CgroupParent”:"",“ConsoleSize”:[0,0],“ContainerIDFile”:"",“CpuCount”:0,“CpuPercent”:0,“CpuPeriod”:0,“CpuQuota”:0,“CpuShares”:0,“CpusetCpus”:"",“CpusetMems”:"",“Devices”:[],“DiskQuota”:0,“Dns”:[],“DnsOptions”:[],“DnsSearch”:[],“ExtraHosts”:null,“GroupAdd”:null,“IOMaximumBandwidth”:0,“IOMaximumIOps”:0,“IpcMode”:"",“Isolation”:"",“KernelMemory”:0,“Links”:null,“LogConfig”:{“Config”:{},“Type”:""},“Memory”:0,“MemoryReservation”:0,“MemorySwap”:0,“MemorySwappiness”:-1,“NetworkMode”:“default”,“OomKillDisable”:false,“OomScoreAdj”:0,“PidMode”:"",“PidsLimit”:0,“PortBindings”:{},“Privileged”:false,“PublishAllPorts”:false,“ReadonlyRootfs”:false,“RestartPolicy”:{“MaximumRetryCount”:0,“Name”:“no”},“SecurityOpt”:null,“ShmSize”:0,“UTSMode”:"",“Ulimits”:null,“UsernsMode”:"",“VolumeDriver”:"",“VolumesFrom”:null},“Hostname”:"",“Image”:“hello-world”,“Labels”:{},“NetworkingConfig”:{“EndpointsConfig”:{}},“OnBuild”:null,“OpenStdin”:false,“StdinOnce”:false,“Tty”:false,“User”:"",“Volumes”:{},“WorkingDir”:""}“
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.157973300-07:00” level=info msg=”{Action=create, Username=root, LoginUID=0, PID=16984}"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.301127090-07:00” level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/b5b5246c259d714f642aaea89d599bf9817456ca43a50b53aacdba40b69067b5/merged"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.448863973-07:00” level=debug msg=“Calling POST /v1.24/containers/7669844e778db323c3c470785fdac81f4bd655caf853873456d3fb14ef16211a/attach?stderr=1&stdout=1&stream=1"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.449040869-07:00” level=info msg=”{Action=attach, Username=root, LoginUID=0, PID=16984}"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.449371502-07:00” level=debug msg="attach: stdout: begin"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.449364122-07:00” level=debug msg="attach: stderr: begin"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.449710497-07:00” level=debug msg=“Calling POST /v1.24/containers/7669844e778db323c3c470785fdac81f4bd655caf853873456d3fb14ef16211a/start"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.449850870-07:00” level=info msg=”{Action=start, Username=root, LoginUID=0, PID=16984}"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.470937016-07:00” level=debug msg="container mounted via layerStore: /var/lib/docker/overlay2/b5b5246c259d714f642aaea89d599bf9817456ca43a50b53aacdba40b69067b5/merged"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.471131412-07:00” level=debug msg="Assigning addresses for endpoint tiny_ptolemy’s interface on network bridge"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.471163484-07:00” level=debug msg="RequestAddress(LocalDefault/172.17.0.0/16, , map[])"
Apr 24 16:09:31 otcnfv03 kernel: device vetheaf0fb6 entered promiscuous mode
Apr 24 16:09:31 otcnfv03 kernel: IPv6: ADDRCONF(NETDEV_UP): vetheaf0fb6: link is not ready
Apr 24 16:09:31 otcnfv03 NetworkManager[7677]: [1493075371.4734] manager: (veth0a125cc): new Veth device (/org/freedesktop/NetworkManager/Devices/32)
Apr 24 16:09:31 otcnfv03 NetworkManager[7677]: [1493075371.4761] manager: (vetheaf0fb6): new Veth device (/org/freedesktop/NetworkManager/Devices/33)
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.507732194-07:00” level=debug msg="Assigning addresses for endpoint tiny_ptolemy’s interface on network bridge"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.657701774-07:00” level=debug msg="Programming external connectivity on endpoint tiny_ptolemy (bdbd890d21bbe78f1a7c01e5088da0c06d69265373f38557a16ba272548cd72f)"
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.741199216-07:00” level=error msg=“containerd: start container” error=“oci runtime error: flag provided but not defined: -console” id=7669844e778db323c3c470785fdac81f4bd655caf853873456d3fb14ef16211a
Apr 24 16:09:31 otcnfv03 dockerd: time=“2017-04-24T16:09:31.742011537-07:00” level=error msg="Create container failed with error: oci runtime error: flag provided but not defined: -console

BTW, I see that this issue is previously reported: https://github.com/moby/moby/issues/30198

However, the suggested workaround of downgrading to runc-0.1.1 did not work for me. I still get an error:

docker run hello-world

No help topic for ‘create’
/usr/bin/docker-current: Error response from daemon: shim error: open pid: no such file or directory.

Resolved the issue myself. Reinstalled all docker-related components, removed a spurious docker.service in /etc/systemd/system and used the default docker.service in /usr/lib/systemd/system . For good measure, I also added overlay2 as storage driver.

Thank you,
Sundar