Skip to content

portal/devfs: share a dynamic char-major block instead of one major per device#87

Open
lacraig2 wants to merge 1 commit into
mainfrom
workspace/ros6devfs
Open

portal/devfs: share a dynamic char-major block instead of one major per device#87
lacraig2 wants to merge 1 commit into
mainfrom
workspace/ros6devfs

Conversation

@lacraig2

@lacraig2 lacraig2 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Problem

Devices registered with major < 0 each call alloc_chrdev_region(), which allocates one whole char-device MAJOR per device. The kernel only has ~234 free majors, so firmware modeling many /dev nodes exhausts the pool.

Concretely on RouterOS6 (armel): nova opens /dev/log%d in a busy loop, so the config enumerates /dev/log0..255 (+ other nodes, ~339 total). Registrations past the cap fail with HYPER_RESP_WRITE_FAIL → the plugin logs Failed to register devfs device 'logNNN' (kernel returned 0) ×105. The guest still boots, but those /dev/logN nodes go unmodeled → ENOENT open-spin → CPU drain, defeating the very reason they're modeled.

Fix

igloo_alloc_dynamic_devt(): reserve majors in blocks of IGLOO_DYN_MINORS (256) and hand out one MINOR per device, allocating a fresh block only when the current fills. Effective cap goes from ~234 devices to ~234×256. The cdev_add()/device_create() failure paths only unregister_chrdev_region() for static-major registrations (a shared-block minor doesn't own its region).

On the prior revert

A prior shared-major attempt (driver 0.0.75, 6fff830) was reverted (a979a09, 0.0.76) for an unrelated mipsel do_ade unaligned-access crash — never documented at the time. I reproduced that crash on a period-accurate image and traced it to a separate unaligned-access bug in the era's driver/qemu, since fixed on the current basenot to major-sharing. Evidence: 0.0.75's exact allocator ported onto the current base boots mipsel clean.

Validation

  • Real target — armel RouterOS6: 105 → 0 devfs registration failures; boots to READY, verifier all-pass.
  • MIPS matrix{mipsel, mipseb, mips64el, mips64eb} × {4.10, 6.13}: 8/8 clean boots, 0 do_ade (incl. big-endian, the worst case for unaligned access).
  • Stress — 300-device /dev flood on mipsel: all register across the 256-minor block rollover, no crash (vs 93 failures per-device).

Follow-up

A regression guard (mipsel + dynamic-devfs flood boot test) will be added in penguin's basic_target CI as a tripwire once this is released and pinned.

…er device

Devices registered with major < 0 previously each called
alloc_chrdev_region(), which burns one whole char-device MAJOR per
device. The kernel only has ~234 free majors, so firmware that models
many /dev nodes exhausted the pool: e.g. RouterOS opens /dev/log%d in a
busy loop and we enumerate /dev/log0..255, so ~105 registrations past
the limit failed with HYPER_RESP_WRITE_FAIL ("kernel returned 0"). The
guest still booted, but the /dev/logN nodes the firmware actually opens
went unmodeled -> ENOENT open spin -> CPU drain, defeating the purpose
of modeling them.

Reserve majors in blocks and hand out one MINOR per device via
igloo_alloc_dynamic_devt(): a single major covers IGLOO_DYN_MINORS (256)
devices, and a fresh block is allocated only when the current one fills.
This lifts the effective cap from ~234 devices to ~234 * 256. The two
cdev_add()/device_create() failure paths now only unregister the region
for static-major registrations, since a shared-block minor does not own
its region.

A prior shared-major attempt (0.0.75, reverted in 0.0.76) was pulled for
an unrelated mipsel do_ade unaligned-access crash. That crash was
reproduced on a period-accurate image and confirmed to originate from a
separate unaligned-access bug in the era's driver/qemu (since fixed on
the current base), not from major sharing: 0.0.75's exact allocator
ported onto the current base boots clean. This scheme was validated on
the real armel RouterOS6 target (105 -> 0 devfs registration failures,
verifier all-pass) and across the full MIPS matrix
{mipsel,mipseb,mips64el,mips64eb} x {4.10,6.13} (8/8 clean boots, no
do_ade).
@lacraig2

lacraig2 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

Regression guard added in penguin: rehosting/penguin#891 (asserts no kernel oops during basic_target boot — the tripwire that was missing when 0.0.75 was reverted).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant