Skip to content

feat: enable cgroup v2 by default for all installers (PE-8520)#608

Merged
santhoshdaivajna merged 3 commits into
mainfrom
pe-8520-enable-cgroupv2-default
May 22, 2026
Merged

feat: enable cgroup v2 by default for all installers (PE-8520)#608
santhoshdaivajna merged 3 commits into
mainfrom
pe-8520-enable-cgroupv2-default

Conversation

@santhoshdaivajna
Copy link
Copy Markdown
Collaborator

@santhoshdaivajna santhoshdaivajna commented May 21, 2026

Summary

Kubernetes 1.35 enforces a hard kubelet validation that refuses to start on cgroup v1 hosts. This caused edge clusters running k3s v1.35.2+k3s1 to enter a permanent crash-restart loop on upgrade

Root cause: OS images (e.g. SLE Micro 5.4 / Kairos v4.0.3) boot in cgroup v1 hybrid mode because systemd.unified_cgroup_hierarchy=1 was not set in the kernel cmdline.

Fix: Add systemd.unified_cgroup_hierarchy=1 unconditionally to all installer paths:

  • Earthfile — patches /etc/cos/bootargs.cfg during image build, so all installed OS images boot with cgroup v2. Covers all grubmenu.cfg entries (kairos.reset, stylus.registration) which inherit from ${extra_cmdline}.
  • overlay/files-iso/boot/grub2/grub.cfg — adds the parameter to all 3 ISO installer menu entries (default, manual, interactive), so the live installer environment also runs under cgroup v2.
  • Dockerfile — activates the previously-commented cgroup v2 sed line for custom image builds using the Dockerfile path directly.

Backward compatibility: cgroup v2 went GA in Kubernetes 1.25. This change is safe for all k8s versions supported in CanvOS (≥ 1.28). Kubernetes 1.31 deprecated cgroup v1; 1.35 makes it a hard failure.

Changed Files

File Change
Earthfile Unconditionally inject systemd.unified_cgroup_hierarchy=1 into bootargs.cfg post-build
overlay/files-iso/boot/grub2/grub.cfg Add kernel param to all 3 installer grub menu entries
Dockerfile Activate previously-commented cgroup v2 line (idempotent guard added)

Test plan

  • Build a provider image and verify /proc/cmdline on the installed OS contains systemd.unified_cgroup_hierarchy=1
  • Verify stat -fc %T /sys/fs/cgroup returns cgroup2fs (not tmpfs) after install
  • Boot the installer ISO and confirm the live environment uses cgroup v2
  • Deploy k3s v1.35.2 on a freshly built image — kubelet should start without cgroup v1 errors
  • Verify existing k8s 1.28/1.29/1.30 clusters still come up correctly (cgroup v2 is backward-compatible)

References

  • KEP-4569 — Kubernetes cgroup v1 deprecation
  • Existing README.md documentation on cgroup v2 (lines 733–751)

Kubernetes 1.35 (k3s v1.35.2+k3s1) enforces a hard validation that
refuses to start kubelet on hosts running cgroup v1. This caused
edge clusters to enter a permanent crash-restart loop on upgrade
(PE-8520).

Add systemd.unified_cgroup_hierarchy=1 to the kernel cmdline in
three places:

- Earthfile: unconditionally patch /etc/cos/bootargs.cfg during image
  build so all installed/running OS images boot with cgroup v2. This
  covers the grubmenu.cfg boot entries (kairos.reset, registration)
  which inherit from ${extra_cmdline}.

- overlay/files-iso/boot/grub2/grub.cfg: add the parameter to all
  three installer menu entries so the live installer environment also
  runs under cgroup v2.

- Dockerfile: activate the previously-commented cgroup v2 sed line
  for custom image builds via the Dockerfile path.

cgroup v2 has been GA since Kubernetes 1.25 and is safe for all
supported k8s versions in CanvOS (>= 1.28). Kubernetes 1.31 deprecated
cgroup v1; 1.35 makes it a hard failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR ensures CanvOS installer media and built images boot with cgroup v2 (unified hierarchy) enabled by default, addressing Kubernetes 1.35+ kubelet behavior that refuses to start on cgroup v1 hosts.

Changes:

  • Add systemd.unified_cgroup_hierarchy=1 to all ISO installer GRUB menu entries.
  • Inject systemd.unified_cgroup_hierarchy=1 into /etc/cos/bootargs.cfg during Earthly base image builds.
  • Enable the same bootargs injection in the Dockerfile path with an idempotency guard.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
overlay/files-iso/boot/grub2/grub.cfg Adds the unified cgroup hierarchy kernel parameter to all ISO installer boot entries.
Earthfile Ensures generated images have the unified cgroup hierarchy parameter added to bootargs.cfg.
Dockerfile Enables cgroup v2 injection for Dockerfile-based builds with an idempotent check.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread Earthfile Outdated
Comment thread Dockerfile Outdated
santhoshdaivajna and others added 2 commits May 22, 2026 09:02
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@santhoshdaivajna santhoshdaivajna merged commit aa021ed into main May 22, 2026
2 checks passed
@santhoshdaivajna santhoshdaivajna deleted the pe-8520-enable-cgroupv2-default branch May 22, 2026 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants