Running sysbox on an initrd device does not work - jailing process inside rootfs caused: permission denied: unknown

General
1

I am trying to run sysbox inside a physical device.
It has kernel 6.1.107 which was compiled by me.
I’ve added all the necessary kernel modules in the kernel’s x86_64-all.config.

iptables/nf/netfilter
bridge
bpf
fuse/cuse/virtio
user ns
...

I’ve changed the /etc/docker/daemon.json so the data-root is on a mounted disk - not the initrd file system (docker yells about it). Afterwards, docker works fine (running docker run --rm -it hello-world works).
Also, I had to add Environment="DOCKER_RAMDISK=true" to docker’s systemd file:

# /etc/systemd/system/docker.service.d/override.conf
[Service]
Environment="DOCKER_RAMDISK=true"

Afterwards I’ve tried to install sysbox properly according to this site’s installation guide (installing the .deb).
Same as docker, I’ve changed /lib/systemd/system/sysbox-mgr.service to have a --data-root on the mounted disk.
Also /lib/systemd/system/sysbox-fs.service with --mountpoint.

Then, when trying to run: docker run --rm -it --runtime=sysbox-runc hello-world I get:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: container_linux.go:439: starting container process caused: process_linux.go:608: container init caused: rootfs_linux.go:119: jailing process inside rootfs caused: permission denied: unknown

Please let me know - is there a missing kernel module? Do I miss some configuration?
Thanks.

~# sysbox-mgr --version
sysbox-mgr
        edition:        Community Edition (CE)
        version:        0.6.7
        commit:         aaeff6c5dc70c137e62166474a309ca5fc42d044
        built at:       Fri May  9 21:15:06 UTC 2025
        built by:       Cesar Talledo

~# docker --version
Docker version 28.3.0, build 38b7060
~# cat /etc/docker/daemon.json
{
    "data-root": "/mnt/mymount/var/lib/docker",
    "runtimes": {
        "sysbox-runc": {
            "path": "/usr/bin/sysbox-runc"
        }
    },
    "bip": "172.20.0.1/16",
    "default-address-pools": [
        {
            "base": "172.25.0.0/16",
            "size": 24
        }
    ]
}

~# cat /lib/systemd/system/sysbox-mgr.service
...
ExecStart=/usr/bin/sysbox-mgr --data-root /mnt/mymount/var/lib/sysbox
...

~# cat /lib/systemd/system/sysbox-fs.service
...
ExecStart=/usr/bin/sysbox-fs --mountpoint /mnt/mymount/var/lib/sysboxfs
...

~# docker info
Client: Docker Engine - Community
 Version:    28.3.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.25.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.37.3
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 28.3.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc sysbox-runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.1.107
 Operating System: Ubuntu 22.04.1 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 13.32GiB
 Name: localhost.localdomain
 ID: f3a21194-c6a0-2224-b55d-29e9eee64f32
 Docker Root Dir: /mnt/mymount/var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.25.0.0/16, Size: 24

WARNING: No io.weight support
WARNING: No io.weight (per device) support
WARNING: No io.max (rbps) support
WARNING: No io.max (wbps) support
WARNING: No io.max (riops) support
WARNING: No io.max (wiops) support
2

I noticed I did not get an answer in the sysbox repository

I hope you will, because I never tried sysbox, but hopefully someone has a better idea about it.

Since it is a permission denied error and mentions rootfs, I would assume that it tries to create the rootfs or anything that is required for jailing the processes (creating a mount namespace) and it doesn’t have permission to do that on the mounted filesystem.

I would check system logs as well. Maybe journalctl -e can show something near the permission denied error if it is logged there too or anything related.

3

Thank you.
Unfortunately I can’t find anything interesting in the logs :confused: