EOF in /var/run/docker.sock

mariusgundersen · February 1, 2015, 2:48pm

The docker daemon refuses to start on my system. I’ve attempted to kill it, but it restarts again right away, and then stops instantly. I know it stops instantly, because of the output in dmesg:

init: docker main process (25641) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25675) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25709) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25743) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25777) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25811) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25845) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25879) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25913) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25947) terminated with status 1
init: docker main process ended, respawning
init: docker main process (25981) terminated with status 1
init: docker respawning too fast, stopped

The error message I get in /var/logs/upstart/docker.log is

INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (var/run/docker.sock)
FATA[0000] EOF
INFO[0000] +job serveapi(unix://var/run/docker.sock)
FATA[0000] EOF

I get the same error message when I try sudo docker -d. The result is that the docker client is unable to connect to the docker server:

$ sudo docker info
FATA[0000] Cannot connect to the Docker daemon. Is 'docker -d' running on this host?

Any idea what might be wrong here?

svendowideit · February 2, 2015, 12:56pm

no idea at all :(. we need to know a lot more about your system to suggest something.

however - maybe you could run https://raw.githubusercontent.com/docker/docker/master/contrib/check-config.sh on your box, and it might show up what the incompatibility is

feel free to paste the output here (and the output of docker version)

mariusgundersen · February 2, 2015, 3:05pm

Rebooting my system fixed it. Really unfortunate that I have to take down the entire server for this.

svendowideit · February 3, 2015, 1:37am

I presume that the installation process upgraded your kernel?

dtrush · February 6, 2015, 4:55am

I’m experiencing the exact same issue.

Ubuntu 14.04.1 LTS

Client version: 1.3.2
Client API version: 1.15
Go version (client): go1.3.3
Git commit (client): 39fa2fa
OS/Arch (client): linux/amd64

check-config reported everything enabled except for optional CONFIG_OVERLAY_FS.

When I run “sudo service docker start” I get:

docker start/running, process 9683

…but ps aux | grep docker turns up nothing.

When I inspect the logs, it’s exactly the same kind of logs as OP reported.

and yes I’ve rebooted

svendowideit · February 10, 2015, 2:18am

what happens if you run sudo docker -D -d ? (the -D means debug output)

dtrush · February 28, 2015, 4:36am

INFO[0000] +job serveapi(unix:///var/run/docker.sock)
INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
DEBU[0000] Registering POST, /containers/{name:.}/wait
DEBU[0000] Registering POST, /exec/{name:.}/start
DEBU[0000] Registering POST, /auth
DEBU[0000] Registering POST, /commit
DEBU[0000] Registering POST, /images/{name:.}/tag
DEBU[0000] Registering POST, /containers/{name:.}/restart
DEBU[0000] Registering POST, /containers/{name:.}/stop
DEBU[0000] Registering POST, /images/{name:.}/push
DEBU[0000] Registering POST, /containers/{name:.}/kill
DEBU[0000] Registering POST, /containers/{name:.}/unpause
DEBU[0000] Registering POST, /containers/{name:.}/start
DEBU[0000] Registering POST, /containers/{name:.}/exec
DEBU[0000] Registering POST, /images/create
DEBU[0000] Registering POST, /containers/{name:.}/copy
DEBU[0000] Registering POST, /exec/{name:.}/resize
DEBU[0000] Registering POST, /containers/{name:.}/attach
DEBU[0000] Registering POST, /build
DEBU[0000] Registering POST, /images/load
DEBU[0000] Registering POST, /containers/create
DEBU[0000] Registering POST, /containers/{name:.}/pause
DEBU[0000] Registering POST, /containers/{name:.}/resize
DEBU[0000] Registering DELETE, /containers/{name:.}
DEBU[0000] Registering DELETE, /images/{name:.}
DEBU[0000] Registering OPTIONS,
DEBU[0000] Registering GET, /images/viz
DEBU[0000] Registering GET, /containers/{name:.}/top
DEBU[0000] Registering GET, /events
DEBU[0000] Registering GET, /info
DEBU[0000] Registering GET, /images/{name:.}/json
DEBU[0000] Registering GET, /containers/{name:.}/export
DEBU[0000] Registering GET, /containers/ps
DEBU[0000] Registering GET, /containers/json
DEBU[0000] Registering GET, /containers/{name:.}/json
DEBU[0000] Registering GET, /containers/{name:.}/logs
DEBU[0000] Registering GET, /_ping
DEBU[0000] Registering GET, /images/json
DEBU[0000] Registering GET, /images/get
DEBU[0000] Registering GET, /images/{name:.}/history
DEBU[0000] Registering GET, /containers/{name:.}/attach/ws
DEBU[0000] Registering GET, /exec/{id:.}/json
DEBU[0000] Registering GET, /version
DEBU[0000] Registering GET, /images/search
DEBU[0000] Registering GET, /images/{name:.}/get
DEBU[0000] Registering GET, /containers/{name:.*}/changes
DEBU[0000] docker group found. gid: 999
DEBU[0000] Generated prefix: docker-8:1-277334
DEBU[0000] Checking for existence of the pool 'docker-8:1-277334-pool’
FATA[0000] EOF

dtrush · February 28, 2015, 4:39am

That was the output for docker -D -d. Note that I’m usually trying to run docker as a service with sudo docker service start. At that time, I have DOCKER_OPTS as defined in /etc/default/docker as:

DOCKER_OPTS=“-H unix:///var/run/docker.sock -H tcp://192.168.0.1:2375”

But note that I’ve got a loopback interface for 192.168.0.1:

lo:0 Link encap:Local Loopback
inet addr:192.168.0.1 Mask:255.255.255.0
UP LOOPBACK RUNNING MTU:65536 Metric:1

Thanks for looking into this… the issue seems to come and go. I had left this thread since it stopped happening mysteriously It’s happening again now, though…

dtrush · April 27, 2015, 7:32pm

Figured I would update this thread for anybody out there experiencing the same issue. The reason I’m running docker bare like this anyways is because I’m running in an environment that doesn’t support virtualization, so boot2docker (basically runs a docker-linux box inside of VirtualBox) is off the table.

Anyways, given the final lines of the error output any time I would attempt to start docker:

DEBU[0000] Checking for existence of the pool ‘docker-8:1-277334-pool’
FATA[0000] EOF

… I was able to figure out that probably my docker install was remembering old data (possibly from another version, as I had installed 1.6.0 and then reverted to 1.4.1, since I need 1.4.1 as a dependency in my specific environment, at least for the time being), and so I apt-get purge’d my installation of docker and had to manually run “sudo rm -r /var/lib/docker”. Upon reinstalling and setting up my /etc/default/docker file again, I was able to start docker as an upstart service on my Ubuntu 14.04 box.

I’m happy to be able to post that I fixed it, but sad that it was such a freaky ride. Oh well, bleeding edge, right? Perhaps consider improving error messages when crashing due to EOF, as I still don’t quite understand what was happenign with the “pool”…

=== EDIT ONE since I’m arbitrarily restricted to three posts… dear god:

Well… it’s back. Now the docker pool has a slightly different name, but the exact same issue persists. I’m starting to think this has to do with errors bubbling up from inside the container. I’m running the Google Cloud SDK (“gcloud” command) Managed VMs operations (“gcloud preview app run” / “gcloud preview app deploy”). I can fix it like so:

$ sudo mv /var/lib/docker/ /var/lib/docker_bak

$ sudo service docker restart

stop: Unknown instance:
docker start/running, process 5229

$ sudo netstat -tunlp | grep docker

tcp 0 0 192.168.0.1:2375 0.0.0.0:* LISTEN 5229/docker

I’m still unsure why this is happening, although it seems related to terminating “gcloud preview app run”, and the existence of this “pool” that keeps throwing EOFs

=== EDIT TWO (it’s really whack that I’m limited to 3 posts… but I’ll live with it)

I’m continuing to try to understand this bug better…

So the issue seems to come when attempting to "check for existence of the pool ‘docker-8:1-277334-pool’ (or similar names), dying with “EOF”.

I noticed when running fdisk -l, the output as follows:

user @host:~$ sudo fdisk -l

Disk /dev/sda: 10.7 GB, 10737418240 bytes
4 heads, 32 sectors/track, 163840 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00019d2d

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 20971519 10484736 83 Linux

Disk /dev/mapper/docker-8:1-2442-pool: 107.4 GB, 107374182400 bytes
255 heads, 63 sectors/track, 13054 cylinders, total 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 65536 bytes
Disk identifier: 0x00000000

Disk /dev/mapper/docker-8:1-2442-pool doesn’t contain a valid partition table

This seems to indicate that the “docker pool” is a virtual device which gets somehow unmounted/has no contents, and docker has a stale reference to it when attempting to start up.

I hope all the information I’ve provided will be helpful to the devs as they look into this issue. I’m continuing to delete /var/lib/docker

Topic		Replies	Views
Need help with stopping containers and starting Docker daemon General	2	9130	September 23, 2019
Container dies instantly after start Docker Hub	0	4510	December 6, 2017
How to kill a docker process that is in docker run -v /var/run/docker.sock:/var/run/docker.sock General	0	491	July 26, 2021
Killing the daemon restarts it \|\| Ubuntu General	2	512	March 2, 2019
Docker compose, error EOF from daemon General docker-compose	0	988	June 15, 2022

EOF in /var/run/docker.sock

Related topics