Error response from daemon, failed to create task for containers on Khadas VIM4

Hello folks!

I have an issue I cannot wrap my head around. I already tried solving it for several hours including:

  • reinstalling docker following the guide on the official docker website
  • trying to find a solution to the specific error message online
  • trying to find solutions to similar error messages online, that could help me identify the problem
  • consulting the website mentioned in the deprecation notice when running docker info

I hope someone who reads this and has a better understanding of there error messages and docker in general, can answer me the following questions:

  1. How do I solve this issue so I can run my containers again?
  2. How can I prevent it from happening again?
  3. What are the underlying issues, so I can not only solve it but also understand the problem?

Thank you!

Edit: Added the client info

How I encountered the issue:

After routine maintenance of my server, that includes updating and upgrading the installed packages, Docker behaves weirdly. I get the following message when I try to start my container, which before worked perfectly fine:

Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: failed to call BPF_PROC_ATTACH (BPF_CGROUP_DEVICE, BPF_ALLOW_MULTI): attach program: invalid argument: unknown

Results from running docker info (I changed the Name, ID, and IP address for privacy/security reasons):

Client: Docker Engine - Community
  Version: 27.4.0
  Context: default
  Debug Mode: false
  Plugins:
    buildx: Docker Buildx (Docker Inc.)
      Version: v0.19.2
      Path: /usr/libexec/docker/cli-plugins/docker-buildx
    compose: Docker Compose (Docker Inc.)
      Version: v2.31.0
      Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
  Containers: 6
    Running: 0
    Paused: 0
    Stopped: 6
  Images: 6
  Server Version: 27.4.0
  Storage Driver: overlay2
    Backing Filesystem: extfs
    Supports d_type: true
    Using metacopy: true
    Native Overlay Diff: false
    userxattr: false
  Logging Driver: json-file
  Cgroup Driver: systemd
  Cgroup Version: 2
  Plugins:
    Volume: local
    Network: bridge host ipvlan macvlan null overlay
    Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
  Swarm: inactive
  Runtimes: runc io.containerd.runc.v2
  Default Runtime: runc
  Init Binary: docker-init
  containerd version: 88bf19b2105c8b17560993bee28a01ddc2f97182
  runc version: v1.2.2-0-g7cb3632
  init version: de40ad0
  Security Options:
    seccomp
      Profile: builtin cgroupns
  Kernel Version: 5.4.180
  Operating System: Ubuntu 22.04.5 LTS
  OSType: linux Architecture: aarch64
  CPUs: 8
  Total Memory: 7.575GiB
  Name: MyMachine
  ID: <MyMachine_ID>
  Docker Root Dir: /var/lib/docker
  Debug Mode: false
  Experimental: false
  Insecure Registries:
    127.0.0.0/8
  Live Restore Enabled: false

[DEPRECATION NOTICE]: API is accessible on http://192.168.XX.XX:2375 without encryption. Access to the remote API is equivalent to root access on the host. Refer to the 'Docker daemon attack surface' section in the documentation for more information: [https://docs.docker.com/go/attack-surface/](https://docs.docker.com/go/attack-surface/)
In future versions this will be a hard failure preventing the daemon from starting! Learn more at: https://docs.docker.com/go/api-security/

Thank you!! :slightly_smiling_face:

The last lines of the docker info output show that neither your os, nor your docker installation is vanilla:

  • You must have either created a custom /etc/docker/daemon.json, or modified the systemd unit.
  • The default Ubuntu 22.04 kernel is a 5.15.x kernel by default, yours is a 5.4.x kernel.

Since you didn’t share the client details from the docker info output., we don’t know if cliet and server version are aligned.

Even though kernel 5.4. should already come with cgroup v2 support, I would recommend reverting to the default 5.15.x kernel

Continuing the discussion from Error response from daemon, failed to create task for containers:

Hello meyay, thanks for your reply!

Could you specify what you mean by “last line of docker info”? Do you mean the deprecation notice?

I’m rather new to Linux, so I may have overlooked a lot of things. My hardware is the Khadas Vim 4 and I installed the OS it came with (Ubuntu). I did not know it is apparently a modified version of Ubuntu.

As for the daemon.json, I did not create one (by myself), and there is no such file in /etc/docker. I remember systemd unit, I think I edited it in order to automatically mount a NFS shared drive, which I had to mount manually after every reboot of my Khadas Vim 4.

I did follow the Ubuntu guide on the official docker website for my initial install and the two reinstalls I performed to solve this issue.

I edited my initial post to now include the client info.

Thanks again for helping!

I guess @meyay meant the Docker daemon is listening on a unix socket by default, not on a TCP socket, so it was changed. If not by you, then by the installer, but you wrote you followed the official guide. Don’t allow the daemon to listen on a public IP without encryption unless there is no way that someone else will be able to access that IP address on a private LAN.

Edited how and what? I hope you didn’t mount an NFS drive to /var/lib/docker.

I’m not familiar with that device, but you can ask the Khadas community

The referred documentation does not exist, but you can probably find the new docs or ask them about where it is.

Hi rimelek, thanks for your reply!

I guess @meyay meant the Docker daemon is listening on a unix socket by default, not on a TCP socket, so it was changed. If not by you, then by the installer, but you wrote you followed the official guide. Don’t allow the daemon to listen on a public IP without encryption unless there is no way that someone else will be able to access that IP address on a private LAN.

I understand. I remember messing around with the docker socket in order to get some dashboard working. I ended up using a docker socket proxy in the end, but I may have forgotten to reset to the default.

Edited how and what? I hope you didn’t mount an NFS drive to /var/lib/docker .

No I did mount my NAS in /mnt/xyz. I mistakenly mixed up systemd and /etc/modules. I edited /etc/modules to load the nfsv4 module at boot. Sorry for this confusing statement.

I’m not familiar with that device, but you can ask the Khadas community

I will look into that, thanks.

I appreciate your willingness to help, but I think I will save all of my docker container data and start with a fresh Ubuntu install as I cannot get rid of this error message even after consulting a friend who works in IT.

If I somehow find a solution to this error, I will post it here.

Thanks to both of you for helping!
Srugger

Hi folks!

Just so you know: I ended up reinstalling the OS. In fact I probably chose the wrong Ubuntu image when I first installed the OS to my Khadas device. This time I made sure to select the appropriate image. After restoring all my docker relevant files from backup, creating some users, giving the right permissions to the docker directories, and some minor configuration changes my system now works again as intended.

Thanks for pointing our that I didn’t run standard Ubuntu and also the other pointers you gave me! I‘m still learning and appreciate your input!

Merry Christmas and a Happy New Year! :partying_face:

Best
Srugger

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.