Docker 27.4.0 inducing Quota propogation Bug

gramachandra · May 7, 2025, 12:06pm

External environmental factors or scripts running on the host created the resource contention
that triggered systemd’s re‐allocation and exposed the latent Docker CPU‐quota propagation
bug (Red Hat Bugzilla #1925343 and Docker Issue #11193 - Docker 27.4.0 + systemd) leaving
the EliteAAA container hard‐capped at ~2 cores instead of 40.
● Docker Issue #11193: “CPU quota not properly handled when container is moved between
cgroups”
o Initially Reported: Docker 1.8
o Partially Fixed: Docker 20.10.6 (April 2021)
o Fully Fixed: Docker 28.0.0 (January 2025)
o The fix involved changing how Docker interacts with cgroup hierarchies managed by
systemd, adding notification hooks to detect when systemd changes cgroup
parameters.

Need to know if using Docker 28.0.0 will fix the above issue in RHEL 8.10 as it mentions solving in Ubuntu?

rimelek · May 7, 2025, 6:10pm

If a release info stated it fixed an issue, I don’t have any reason to doubt it, but I Don’t know what issue numbers you are referring to. The Docker issue number that you also shared on GitHub

github.com/moby/moby

Docker CPU‐quota propagation bug

opened 12:10PM - 07 May 25 UTC

gowtham66867

status/0-triage kind/bug

### Description External environmental factors or scripts running on the host c…reated the resource contention that triggered systemd’s re‐allocation and exposed the latent Docker CPU‐quota propagation bug (Red Hat Bugzilla #1925343 and Docker Issue #11193 - Docker 27.4.0 + systemd) leaving the EliteAAA container hard‐capped at ~2 cores instead of 40. ● Docker Issue #11193: "CPU quota not properly handled when container is moved between cgroups" o Initially Reported: Docker 1.8 o Partially Fixed: Docker 20.10.6 (April 2021) o Fully Fixed: Docker 28.0.0 (January 2025) o The fix involved changing how Docker interacts with cgroup hierarchies managed by systemd, adding notification hooks to detect when systemd changes cgroup parameters. Need to know if using Docker 28.0.0 will fix the above issue in RHEL 8.10 as it mentions solving in Ubuntu? ### Reproduce Detailed Reproduction Steps To reproduce this issue in a controlled environment: Test Environment Setup # Prerequisites: A system with Docker 27.4.0 and systemd # Set up a multi-core test environment similar to production # 1. Create a test container with access to multiple cores docker run -d --name test_container --cpus=40 ubuntu:20.04 sleep infinity # 2. Verify initial CPU quota settings CONTAINER_ID=$(docker inspect --format '{{.Id}}' test_container) echo "Initial CPU quota:" cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_quota_us cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_period_us Triggering the Bug # 1. Run a CPU-intensive process inside the container docker exec test_container bash -c 'apt-get update && apt-get install -y stress-ng' docker exec -d test_container stress-ng --cpu 4 --timeout 3600s # 2. Start a competing CPU-intensive process outside the container # This simulates the log compression script for i in {1..4}; do nice -n 0 gzip -c /dev/zero > /dev/null & done # 3. Check CPU quota during resource contention echo "CPU quota during contention:" cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_quota_us cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_period_us # 4. Verify container CPU usage is restricted docker stats test_container --no-stream Verifying and Resolving # 1. Verify the bug is occurring docker exec test_container bash -c 'cat /proc/self/cgroup | grep cpu' systemd-cgls | grep -A 5 $(docker inspect --format '{{.Id}}' test_container | cut -c1-12) # 2. Restart Docker to fix the issue systemctl restart docker # 3. Check CPU quota after restart CONTAINER_ID=$(docker inspect --format '{{.Id}}' test_container) echo "CPU quota after restart:" cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_quota_us cat /sys/fs/cgroup/cpu/docker/$CONTAINER_ID/cpu.cfs_period_us Expected Results Before triggering the bug: Container has access to all cores After running competing processes: Container restricted to ~1-2 cores After Docker restart: Container regains access to all cores ### Expected behavior Should use all the Cores and not just 2 cores ### docker version ```bash 27.4.0 ``` ### docker info ```bash 27.4.0 ``` ### Additional Info _No response_

Doesn’t seem to be related to any quota issue. It was about video recording so I don’t get that either.

Topic		Replies	Views
Linux kernel becomes non-responsive on high end (200+ hw threads) platform for dockers with low cpu (cfs) quota and large number of threads General docker	0	504	January 14, 2019
Is there a cgroup impact to docker service create --reserve-cpu General	0	1338	June 20, 2018
Swar-m CPU shares is the number of CPUs rather than shares General docker	0	614	April 3, 2017
No apparent cause for runtime/cgo: pthread_create failed: Resource temporarily unavailable General docker	1	11074	June 19, 2022
Cpu_shares and docker swarm - assigning less than 1 CPU for a service Swarm	0	1542	December 6, 2016

Docker 27.4.0 inducing Quota propogation Bug

Related topics