perf(gpu): let macOS choose its own convert mode, and thread the conversion

[wagenet]

The bug

detect_gpu_capabilities() set the process-wide VideoConvertMode by asking whether nvh264enc exists. On a Mac it never does, so every Mac fell through to Software, a right answer reached by asking about hardware the machine does not have.

That hid the question that matters. videoconvert ships with n-threads=1, and the string appeared nowhere in the tree, so every colour conversion on an 8-core M2 ran on one core.

(a) macOS decides for itself

The mode decision is split per platform. Linux and Windows keep the existing WSL / NVENC / CUDA-GL interop path unchanged. macOS gets its own detect_convert_mode() and still picks Software, for two reasons that apply to the hardware:

• autovideoconvert offers no GPU path for the frames these call sites see. Given GL memory it picks glcolorconvert, but our blocks feed it system memory, because the GL chains in this tree download at their boundary. For system memory it selects videoconvertscale on the CPU and never creates a GL context, verified with GST_DEBUG=gl*:5. It measured 0.7% slower than single-threaded videoconvert.
• Being a bin, it exposes no n-threads, so it would forfeit (b).

(b) n-threads

configure_video_convert() sets the pool in one place, and all five call sites route through it (ndi.rs ×4, devicesrc.rs, videoenc.rs, videoformat.rs, vision_mixer/builder/pipeline_cpu.rs).

Measurements come from the authoring machine, an 8-core M2 (4 performance + 4 efficiency), whose paired-run noise floor is ±11%. 1080p RGBA→I420, paired runs alternating the within-pair order every pair:

concurrent converts	n-threads 1 → 4	pairs won	order check (Δ A-first / B-first)
1	−23.8%	20/20	+0.695 / +0.705
2	−18.4%	16/16	+0.573 / +0.569
4	−8.3%	14/16	+0.416 / +0.403

The effect is the same size whichever variant ran first, so it is not order bias. The convert stage alone drops from 1.05 s to 0.24 s per 300 frames.

Thread count

macOS reports its cores as named performance tiers, and the pool counts every tier except the efficiency one: 4 on the M2, 10 on an M4 Pro, 36 on an M5 Ultra. Reading tier 0 alone would give the Ultra 12 and discard its 24 performance cores. The tiers are distinct rather than renamed; an M6 has 2 super, 4 performance and 6 efficiency cores, and Apple describes performance cores as joining the super cores for demanding multithreaded work while efficiency cores handle background tasks.

Excluding the efficiency tier rests on mechanism, since measurement could not settle it: 4 threads against 8 landed within 2% and disagreed about the sign across contention levels. Equal-stripe splitting makes an efficiency core the straggler that the whole frame waits on.

A bound of 32 guards against a bogus syscall result and logs when it binds. It stays loose because the errors are asymmetric: 16, 32 and 64 threads cost 1–2% against a 4-thread baseline, while running 1 instead of 4 costs 24%. STROM_VIDEOCONVERT_THREADS (1–64) overrides it for anyone able to measure on a wider machine.

Scope: macOS only

configure_video_convert() is a no-op elsewhere, so Linux and Windows are unchanged. Non-NVIDIA Linux hosts are single-threaded too, and this could have gone cross-platform. It does not, because the count is an Apple-silicon heterogeneous-core heuristic that does not transfer, and because a Linux container's visible CPU count routinely overstates its cgroup quota.

Tests

• cargo test --lib, 546 passed (10 new gpu::tests); cargo test --test pipeline_lifecycle_test, 3 passed
• cargo clippy --all-targets -- -D warnings, and with --features efp,nvidia, both clean
• Not run: the full integration suite. It links dozens of ~280 MB debug binaries, the authoring machine was at ~11 GB free, and two attempts stalled. CI covers it.

configure_video_convert_sets_thread_count fails if the fix is reverted, with left: 1, right: 4. No Max or Ultra was available, so resolve_thread_count is split out from the environment and the wide-machine paths are pinned by unit tests: 10/12/24 cores, the bound, and override parsing.

autovideoconvert_still_cannot_be_threaded and no_applemedia_converter_to_reconsider guard assumptions instead of behaviour. The second fires if applemedia ever ships a Converter-klass element, which autovideoconvert would discover automatically and which would make GpuAccelerated worth re-measuring. Pointing its query at opengl confirms it trips.

All of these are #[cfg(target_os = "macos")], and the macOS CI job is workflow_dispatch-only, so they do not run on a normal PR.

Beyond the microbenchmarks, an isolated headless instance running a builtin.videoformat block showed 3 threads in gst_parallelized_task_thread_func plus the caller, giving 4 stripes. At n-threads=1 there are none.

Follow-up (not in this PR)

videoformat.rs builds videoscale → videoconvert unconditionally, walking each frame twice when both resolution and format change. A single videoconvertscale measured 25–30% faster there, and identical when only the format changes. Separate PR, different rationale and blast radius.

[wagenet]

This code all makes sense to me.

[srperens]

Verdict: Comment — correct diagnosis and thorough, well-designed test coverage, but this is ~400 lines of new macOS-only unsafe FFI and threading logic that this repo's CI has never compiled, which blocks approval per REVIEW.md.

Claims

Claim	Verdict	Evidence
The Linux/Windows CUDA-GL interop path is unchanged, only extracted into its own function	CONFIRMED	backend/src/gpu.rs:259 — `fn detect_convert_mode() -> VideoConvertMode {` under #[cfg(not(target_os = "macos"))], same WSL/NVENC/interop steps as before, same outcomes
macOS's n-threads sizing and sysctl reads are covered by unit tests	CONFIRMED	backend/src/gpu.rs:593 — `fn configure_video_convert_sets_thread_count() {` plus 8 more #[cfg(target_os = "macos")] tests exercising resolve_thread_count, fast_cores_from_tiers, and a live sysctlbyname round-trip
Every n-threads-eligible call site is covered	CONFIRMED	8 call sites across devicesrc.rs, ndi.rs (×4), videoenc.rs, videoformat.rs, pipeline_cpu.rs all call gpu::configure_video_convert() right after building the element_name()-selected converter — no other such construction site remains uncovered
The new macOS code (sysctl_u32/sysctl_string/video_convert_threads/configure_video_convert's macOS arm) has been compiled by this repo's CI	UNVERIFIED	gh pr checks 726: Build (macOS) skipping; .github/workflows/ci.yml:543 — if: github.event_name == 'workflow_dispatch' && contains(fromJSON('["both","macos"]'), inputs.platforms) — gates build-macos, which this PR run is not

Diagnosis — Root cause matches the PR's own trace: detect_gpu_capabilities() asked an NVENC question that is always "no" on a Mac, which hid the real cost (n-threads=1 serializing every frame). Splitting detect_convert_mode() per platform and threading the pool is the right layer — it's a startup-time decision plus a per-element property, not a runtime workaround. Coverage is ABSOLUTE for the stated bug (every element_name()-selected converter now gets configure_video_convert()), not BOUNDED.

Blast radius — GLOBAL: configure_video_convert() is called from 8 sites spanning device input, NDI in/out, video encode, format conversion and the CPU vision-mixer path, and detect_convert_mode() replaces the process-wide startup decision VIDEO_CONVERT_MODE is set from. The macOS branch is a no-op elsewhere (backend/src/gpu.rs:495 — `let _ = element;` under the not(macos) arm of configure_video_convert), and the non-macOS detect_convert_mode body is byte-for-byte the prior logic, so Linux/Windows behaviour is unaffected — but that equivalence, and the sysctl-based core count, rest entirely on macOS, the one platform this run cannot exercise.

Tests & CI — Check (Linux), both Build (Linux) jobs, Check & Build (WASM), API Contract Check all green at 57c1ee2. sysctl_u32/sysctl_string reading real Apple integer/string sysctls (hw.nperflevels, hw.perflevelN.name/.logicalcpu) is EXTERNAL (Darwin's own ABI), stated as an assumption, not this repo's to verify — but whether the calls compile and the property actually lands on a live videoconvert (configure_video_convert_sets_thread_count) is settleable and unsettled here. Dispatch before merge: gh workflow run ci.yml --ref wagenet/macos-convert-mode -f platforms=macos — that job does run cargo test --package strom, which would execute all 9 new macOS tests.

Design record — States the rejected alternative (measuring n-threads uniformly across platforms) and why it doesn't transfer to Linux (cgroup quota overstatement).

Confidence: HIGH