OCI runtime create failed: unable to join session keyring: disk quota exceeded" on Docker inside LXC (Incus, TrueNAS Scale)

General
, , , ,
1

Hi everyone,

I’m encountering an intermittent issue with Docker when running inside an LXC container managed by Incus on TrueNAS Scale (ZFS-based storage). I’m hoping someone here might help shed light on it.

When I run the following command inside the container:

docker compose down && docker compose up -d

I occasionally get the following error during the container startup phase:

[+] Running 3/3
 ✔ Container hass               Removed
 ✔ Container mariadb            Removed
 ✔ Network docker_internal_net  Removed
[+] Running 2/3
 ✔ Network docker_internal_net  Created
 ✔ Container mariadb            Started
 ⠴ Container hass               Starting
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: unable to join session keyring: unable to create session key: disk quota exceeded: unknown

Setup:

  • Host OS: TrueNAS Scale 25.04
  • Container Manager: Incus (formerly LXD)
  • LXC Guest OS: Ubuntu (minimal)
  • Docker version: 28.1.1 (inside LXC)
  • Services involved: Home Assistant (hass) and mariadb
  • Storage: External ZFS dataset mounted inside the LXC container, used as Docker volumes for persistent storage

Behavior:
This does not happen consistently—sometimes everything starts just fine.
Restarting the LXC container or waiting a bit often “fixes” the issue temporarily.
I suspect it might be related to how the external ZFS mountpoints are handled, or possibly an issue with kernel namespaces/session keyrings and quotas inside the LXC container.

What I’ve tried:

  1. Verified permissions (chown/chmod) of the mountpoints.
  2. Ensured the container has the required capabilities and nesting enabled.
  3. Checked available disk space on both the host and inside the container (no issues).

Questions:

  • What does the error unable to join session keyring: disk quota exceeded actually mean in the context of LXC and Docker?
  • Could this be related to keyctl or kernel namespace limitations in unprivileged containers?
  • Are there best practices or additional steps for ensuring keyring/session handling is properly set up in Incus containers running Docker?
  • Would switching to a privileged container (or tweaking apparmor/seccomp settings) help?

Any insights, suggestions, or guidance on how to debug this deeper would be greatly appreciated.

Thanks in advance!

2

I have never seen that error message, but since you have special setup based on TrueNAS and Incus, an Incus or TrueNAS more likely experienced it. Have you also asked about the issue on their forums?

Probably.

I searched for the error message and found this

If I remember correctly, Incus was a fork of LXD by the original developers but LXD still exists and developed by Canonical.

3

Hello @rimelek thanks for your answer. I did not try on the TrueNAS forum yet since their Incus implementation is pretty new and people usually stick with its docker apps support (not using the lxc/incus virtualization). I will have a try anyway.

I already saw that topic and already checked that my keyring usage was low and my default limits pretty high (root_maxkeys = 1,000,000 and root_maxbytes = 25,000,000).

For now I tried to run the incus container into privileged mode (which I am not totally aware of what it means) but it seems like docker can run the container now.

I would like to deep dive into this issue to better understand how the two system coexists.