Docker: Error response from daemon: connection error: desc = "transport: dial unix /var/run/docker/containerd/docker-containerd.sock: connect: connection refused": unknown

I am getting the error “docker: Error response from daemon: connection error: desc = “transport: dial unix /var/run/docker/containerd/docker-containerd.sock: connect: connection refused”: unknown.” when trying to restart existing containers, run new containers, docker top, docker exec, and probably more commands I have not listed. I also only dashes when I do docker stats. I can list containers and see they are running and I am able to use the apps on the containers as expected. Any thoughts?

docker info
Containers: 11
Running: 10
Paused: 0
Stopped: 1
Images: 12
Server Version: 17.11.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: N/A (expected: 992280e8e265f491f7a624ab82f3e238be086e49)
runc version: 0351df1c5a66838d0c392b4ac4cf9450de844e2d
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-104-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 120.7GiB
Name: myserver
ID: my:server:id
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
x.x.x.x/8
Live Restore Enabled: false

WARNING: No swap limit support

1 Like

I am also having this issue.

I appear to have started also also not having docker be able to delete some containers after they finished running.

Log messages like
Jan 5 14:59:53 dockerd: time=“2018-01-05T14:59:53.067411517-05:00” level=error msg=“Failed to remove container fc78917e0612f0a81be9210fac5cdc21e3534ea13af468ab4f1b214ceead1afb: N
o such container: fc78917e0612f0a81be9210fac5cdc21e3534ea13af468ab4f1b214ceead1afb”

Do you have your sock mounted to any other cotnainer such as Portainer? Or are you using any webproxy?

I have this sumologic container with /var/run/docker.sock:/var/run/docker.sock as a volume. That container had been running for about a week before these problems started.

I am getting the same error.
The problem is the /var/run/docker is not accessible from the current login user in ubuntu. That’s why it could not connect to the container. And throwing the connection refused error.

Also getting errors with latest docker-ce: v17-12

Above all the issues the best way is to restart docker and then perform commands in it. for ubuntu service docker restart
This fix my problems…:grinning:

2 Likes

I am getting the same error, docker for mac.

Restarting docker of course clears the problem, but only temporarily. I am several restarts deep and hoping not to have to do it anymore. The nature of docker for mac means restarts probably take much longer than the pure linux experience.

Hello. I have the same error on Windows after the latest update:

Docker version 17.12.0-ce, build c97c6d6

Yes, if I restart Docker, everything works but doing it several times during a day is not convenient. Do you plan to fix it in the nearest future?

Hello I am getting similar error.

Error response from daemon: Could not kill running container 5a7e6e7bab09b8987a1757490dea6dd3626763acb17e3eadacd4d9641ddce511, cannot remove - Cannot kill container 5a7e6e7bab09b8987a1757490dea6dd3626763acb17e3eadacd4d9641ddce511: connection error: desc = “transport: dial unix /var/run/docker/containerd/docker-containerd.sock: connect: connection refused”: unknown

Server:
Engine:
Version: 17.12.0-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.2
Git commit: c97c6d6
Built: Wed Dec 27 20:09:53 2017
OS/Arch: linux/amd64
Experimental: false

Is there any update on this? Thanks

Has anyone found a solution for this?? My company uses docker on the mac for local testing of container networking before they are pushed to AWS for hosting. The problem with local testing is, there is absolutely no excuse for my cpu usage to skyrocket to over 410% and sound like my mac is about to take off through the window…I am simply trying to test my oplog-monitor but each time I spin up my elasticsearch container, everything gets loud, fails, and now I’m 6 restarts deep with no progress and no solution.

Specs:

I also get this error.
$ sudo docker kill 123456
Error response from daemon: Cannot kill container: 123456: Cannot kill container 123456: connection error: desc = “transport: dial unix /var/run/docker/containerd/docker-containerd.sock: connect: connection refused”: unknown

$ docker version
Client:
Version: 17.12.1-ce
API version: 1.35
Go version: go1.9.4
Git commit: 7390fc6
Built: Tue Feb 27 22:17:40 2018
OS/Arch: linux/amd64

Server:
Engine:
Version: 17.12.1-ce
API version: 1.35 (minimum version 1.12)
Go version: go1.9.4
Git commit: 7390fc6
Built: Tue Feb 27 22:16:13 2018
OS/Arch: linux/amd64
Experimental: false

I was getting this error on my Mac, 10.12.6, Docker 17.?.? (upgraded to 18.03.0-ce-mac59 (23608) in attempt to fix, with no luck). Also was getting CPU bound when this happened…

Figured out I didn’t have enough memory allocated to Docker (Preferences->Advanced->Memory).

Details: The condition arose over a month after I had granted a DIND container max of 2G, and Docker overall was set to max of 2G. When the container finally started needing somewhere above 700M, the disconnect/CPU-bound condition would arise. Nothing but restarting Docker would fix it. I haven’t seen the condition since I allocated 8G memory to Docker.

It seems to have bug in containerd, see below:


Reported workaround is to killall -9 dockerd or reboot the system, but it’s better update docker version to 17.12.1 or 18.03 .

2 Likes

UPDATE:
I found this happening again after installing an update to my Mac OS (assuming it’s similar cause for others).

My Best assumption is bad sockets, and connections for docker images. So updating docker is not the issue, it’s the connections and sockets on your OS.

The solution that worked for me everytime was the following (mac):

~ docker ps --all
//shows all background images still processing
~ docker-compose down
//shut down any compose safely
~ docker system prune -a
~ docker ps --all
//should be empty
~ docker images
~ docker rmi -f { container_id_1 container_id_2 container_id_3 }
// all containers listed with a space between them
~ docker images
//should be empty

restart docker

Now you should be able to rebuild your images with no problem (since all sockets are new).

Hope this helps someone…

Cheers,

I am also having this issue.

I appear to have started also also not having docker be able to delete some containers after they finished running.

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 4
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1062.4.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 1.795GiB
 Name: Rain
 ID: 256T:4W3A:PUPX:TG3L:R4YB:Z67S:D23R:YSJN:J3IQ:KX3I:4ZUN:OLD6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors:

 Live Restore Enabled: false
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since 日 2021-07-11 23:59:03 CST; 10min ago
     Docs: 
 Main PID: 14666 (dockerd)
    Tasks: 10
   Memory: 115.8M
   CGroup: /system.slice/docker.service
           └─14666 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

7月 11 23:59:03 Rain systemd[1]: Started Docker Application Container Engine.
7月 11 23:59:03 Rain dockerd[14666]: time="2021-07-11T23:59:03.514512552+08:00" level=info msg="API listen on /var/run/docker.sock"
7月 11 23:59:16 Rain dockerd[14666]: time="2021-07-11T23:59:16.779758085+08:00" level=error msg="stream copy error: reading from a closed fifo"
7月 11 23:59:16 Rain dockerd[14666]: time="2021-07-11T23:59:16.789624452+08:00" level=error msg="stream copy error: reading from a closed fifo"
7月 11 23:59:16 Rain dockerd[14666]: time="2021-07-11T23:59:16.875893134+08:00" level=error msg="d0097eef061e8664df61d3e33356259b9d145aacea7fd6b8f5c182b4f585d12e cleanup: failed to delete container from containerd: no such container"
7月 11 23:59:16 Rain dockerd[14666]: time="2021-07-11T23:59:16.875951736+08:00" level=error msg="Handler for POST /v1.41/containers/d0097eef061e/start returned error: dial unix \x00/run/containerd/s/f93cfe2f6a6d610cbca7b7b5ef0134e8df32a4d5664f2ceb89c77325c5652ff8: connect: connection refused: unknown"
7月 12 00:08:33 Rain dockerd[14666]: time="2021-07-12T00:08:33.169566559+08:00" level=error msg="stream copy error: reading from a closed fifo"
7月 12 00:08:33 Rain dockerd[14666]: time="2021-07-12T00:08:33.169797977+08:00" level=error msg="stream copy error: reading from a closed fifo"
7月 12 00:08:33 Rain dockerd[14666]: time="2021-07-12T00:08:33.252149869+08:00" level=error msg="d0097eef061e8664df61d3e33356259b9d145aacea7fd6b8f5c182b4f585d12e cleanup: failed to delete container from containerd: no such container"
7月 12 00:08:33 Rain dockerd[14666]: time="2021-07-12T00:08:33.252202601+08:00" level=error msg="Handler for POST /v1.41/containers/d0097eef061e/start returned error: dial unix \x00/run/containerd/s/f93cfe2f6a6d610cbca7b7b5ef0134e8df32a4d5664f2ceb89c77325c5652ff8: connect: connection refused: unknown"

When I run docker start, I am prompted to report an error

[root@Rain ~]# docker start d0097eef061e
Error response from daemon: dial unix /run/containerd/s/f93cfe2f6a6d610cbca7b7b5ef0134e8df32a4d5664f2ceb89c77325c5652ff8: connect: connection refused: unknown
Error: failed to start containers: d0097eef061e