Cgroup V2 the saga continues

pirolla · January 15, 2024, 2:26pm

In the recent past I was able to have old systemd (because testing ansible roles using molecule+docker is nice) versions running (dreadful amazonlinux:2) fine by running the container with:
–privileged --cgroupns=host -v /sys/fs/cgroup:/sys/fs/cgroup:rw

But I also had to add cgroup_no_v1=all to linuxkit kernel parameters in:
/Applications/Docker.app/Contents/Resources/linuxkit/cmdline

Which is not anymore in the latest version of Docker desktop (24.0.7)… So could any helpful guru show me the enlightened path to send this parameter to the latest linuxkit docker image?

ps. I really wanted to create my own linuxkit image to avoid going through the whole internet every time docker devs changes it but it seems really really hard to tame the underlying linux image docker desktop uses…

Testing Dockerfile

FROM amazonlinux:2

VOLUME [ "/tmp", "/run", "/run/lock" ]

# Mostly tested with SuSe and Amazon Linux
RUN yum -y update && yum install -y rsyslog logrotate tar unzip gzip dbus systemd systemd-sysv python3 python3-setuptools python3-pip bash iproute net-tools sudo vim

WORKDIR /lib/systemd/system/sysinit.target.wants/
# hadolint ignore=SC2086

RUN cd /lib/systemd/system/sysinit.target.wants/ ; \
    for i in *; do [ $i = systemd-tmpfiles-setup.service ] || rm -f $i ; done ; \
    rm -f /lib/systemd/system/multi-user.target.wants/* ; \
    rm -f /etc/systemd/system/*.wants/* ; \
    rm -f /lib/systemd/system/local-fs.target.wants/* ; \
    rm -f /lib/systemd/system/sockets.target.wants/*udev* ; \
    rm -f /lib/systemd/system/sockets.target.wants/*initctl* ; \
    rm -f /lib/systemd/system/basic.target.wants/* ; \
    rm -f /lib/systemd/system/anaconda.target.wants/*

RUN sed -i 's/OOMScoreAdjust=-900//' /lib/systemd/system/dbus.service

WORKDIR /

ENTRYPOINT ["/lib/systemd/systemd"]

rimelek · January 15, 2024, 7:50pm

That’s the version of Docker, not the Desktop. The Destkop version can change even if the Docker version stays the same, but the Docker version can’t change without the Docker Desktop version.

I didn’t know about the cmdline file and I’m surprised that was even there. What wouldn’t surprised me if Docker Inc was trying to make any customization even harder for easier support. I would still tell you if I knew a solution, but I don’t.

What I could recomend is asking for Systemd support in Docker Desktop on GitHub: Issues · docker/roadmap · GitHub

Obviously you need a solution sooner, so the second I would do is using a custom virtual machine for the systemd tests. You could also try Podman Desktop. Podman claims to support Systemd, so I would assume they support it in Podman Desktop as well, but I don’t know for sure, because I don’t use it. I just tried once…

You can also try something like Jeff Geerling does in his images. I’m pretty sure you know his name if you work with Ansible. He doesn’t use a real systemd, but a fake one. I tried that and I think I had some problem with it, but I don’t remember what. Here is an issue related to macOS:

github.com/geerlingguy/docker-ubuntu2204-ansible

Molecule doesn't start systemd

opened 02:29AM - 13 Feb 23 UTC

closed 02:35AM - 03 Apr 23 UTC

artis3n

It has been some time, and this is close to #6 , but given I am not receiving th…at error I opted for a new issue. I cannot get Molecule working with cgroupsv2 and this container (or most others since the cgroup change, admittedly). This container works fine when executed directly, mirroring the commands in the README, but fails under Molecule during any execution of a systemd task with: ``` fatal: [instance]: FAILED! => {"changed": false, "msg": "failure 1 during daemon-reload: System has not been booted with systemd as init system (PID 1). Can't operate.\nFailed to connect to bus: Host is down\n"} ``` This occurs locally on OSX and on an ubuntu 22.04 GitHub Actions runner. My `molecule.yml` is: ```yaml --- dependency: name: galaxy driver: name: docker platforms: - name: instance image: geerlingguy/docker-ubuntu2204-ansible:latest volumes: - /sys/fs/cgroup:/sys/fs/cgroup:rw cgroupns_mode: host privileged: true pre_build_image: true provisioner: name: ansible verifier: name: ansible scenario: name: default test_sequence: # - dependency - destroy - syntax - create - prepare - converge - idempotence # - verify # - cleanup - destroy ``` A minimally reproducible `converge.yml` is: ```yaml --- - name: Converge hosts: all tasks: - name: ":(" become: true ansible.builtin.systemd: daemon_reload: true ``` Running `molecule test` will result in the above error both locally and on the Ubuntu 22.04 runner. Given the molecule PR introducing `cgroupns_mode` is merged and I don't see issues about this from others in the molecule and molecule-plugins repos, I am assuming I'm missing something dumb in the `molecule.yml`. Any thoughts as to what I should be doing to resolve this error?

Here is a repo for Amazon Linux 2023:

and one for Amazon Linux 2

pirolla · January 16, 2024, 11:39am

Thanks for the great answer, rimelek.

Yesterday evening I’ve ended up just using a local directory (e.g. /tmp/systemd) with cgroupns set to host:

docker run --rm -it --privileged --cgroupns host -v /tmp/systemd:/sys/fs/cgroup:rw al2

al2 being the container built using Dockerfile of the post. Somehow linux is able to do it’s cgroup thing:

cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)

And the old version of systemd is working.

p.s.: I’ve taken the version from this page: Docker Desktop release notes | Docker Docs
I thought the engine (24.0.7) was the Docker version

rimelek · January 17, 2024, 6:45pm

Yes, but not the Docker Desktop version which you wrote in your post. The page is Docker Desktop release notes, so the Docker Desktop release versions are the biggest numbers in size you can find there

But thank you for sharing the solution. I wasn’t sure if current desktop versions required the cmdline parameter, because it was so long when I tried systemd in a container. When I did, I mounted folders, but I never even heard about the parameter you mentioned. So I learned something too

victorsmonster · March 20, 2025, 3:45pm

Just wanted to thank you for posting your fix. I use an old version of neo4j for a project I’m working on (4.2.19) and I was having similar issues. Using your configuration got me going again as well! Thanks again!

in my case the command was docker run --rm -it --privileged --cgroupns host -v /tmp/systemd:/sys/fs/cgroup:rw neo4j:4.2.19-community

Topic		Replies	Views
I need to create systemd cgroup on Docker Beta host Docker Desktop beta	4	3081	June 11, 2016
Systemd will only start correctly after mounting cgroups rw once Docker Desktop	1	5734	June 28, 2016
Dockerd uses systemd containerd without --containerd argument General	5	2022	May 5, 2022
Docker - Desktop only launching with 'systemctl --user restart' Docker Desktop linux	4	1539	July 30, 2024
Cannot run systemd in container Docker Desktop	0	937	July 22, 2016

Cgroup V2 the saga continues

Related topics