The directory “/run/usr/986” is filling up fast.
1) Stop XSOAR
systemctl stop demisto
2) Enable lingering for the service user
sudo loginctl enable-linger demisto
3) Create a per-user storage config
sudo -u demisto mkdir -p /home/demisto/.config/containers
sudo cp /etc/containers/storage.conf /home/demisto/.config/containers/storage.conf
sudo chown demisto:demisto /home/demisto/.config/containers/storage.conf
sudo chmod 700 /home/demisto/.config/containers/storage.conf
Edit /home/demisto/.config/containers/storage.conf and set:
runroot = "/run/user/$(id -u demisto)/runroot"
graphroot = "/home/demisto/.local/share/containers/storage"
4) Reset stale Podman directories
sudo rm -rf /home/demisto/.cache/containers/
sudo rm -rf /home/demisto/.local/share/containers/
sudo rm -rf /tmp/podman-run-$(id -u demisto)
sudo rm -rf /tmp/containers-user-$(id -u demisto)
sudo rm -rf /tmp/tmp/run-$(id -u demisto) 2>/dev/null || true
Note: Skip deleting .local/share/containers/ if you must preserve local images.
5) Start XSOAR again
systemctl start demisto (for server)
6) Verify new runroot
sudo -u demisto podman info | grep -E 'runRoot|graphRoot'
sudo -u demisto podman pull docker.io/library/alpine:latest
sudo -u demisto podman run --rm alpine:latest echo OK
After applying the steps, the storage usage reduced to 18% and the issue occurs again after 2 weeks.
2025-07-14 06:31:22.6039 error Returning error for docker code runner request. script:[SplunkPTOS_SplunkPy_fetch-incidents], error: [Docker code script is in inconsistent state, expected pong but didn’t got any response, exit error: Container exit with error. container name: [demistoserver_pyexec-3c4bf840-a877-4f5f-8b97-43d6ac9bea8b-demistosplunksdk-py31.0.0.2074112–213956] error: [exit status 126] stderr: [time=“2025-07-14T06:31:22+07:00” level=error msg=“Unable to write pod event: \“write /run/user/986/libpod/tmp/events/events.log: no space left on device\””
time=“2025-07-14T06:31:22+07:00” level=error msg=“Unable to write image event: \“write /run/user/986/libpod/tmp/events/events.log: no space left on device\””
time=“2025-07-14T06:31:22+07:00” level=error msg=“Unable to write pod event: \“write /run/user/986/libpod/tmp/events/events.log: no space left on device\””
Error: OCI runtime error: runc: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:496: store init state caused: