Skip to content

feat(gst): give pipeline threads a macOS QoS class so they run on P cores - #723

Draft
wagenet wants to merge 4 commits into
Eyevinn:mainfrom
wagenet:wagenet/macos-thread-qos
Draft

feat(gst): give pipeline threads a macOS QoS class so they run on P cores#723
wagenet wants to merge 4 commits into
Eyevinn:mainfrom
wagenet:wagenet/macos-thread-qos

Conversation

@wagenet

@wagenet wagenet commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Stacked on #722. The base branch for that PR does not exist on this repo, so this PR targets main and therefore shows four commits — the first three (feat(monitor), docs(monitor) and fix(monitor)) are #722's and are already under review there. Only the last commit, feat(gst): give pipeline threads a macOS QoS class, is new here. Please merge #722 first; this collapses to a single commit once it lands. Depends on #735 — without it App Nap erases the QoS class in headless mode about thirty seconds in.

What

On Apple Silicon it is the QoS class, not the pthread priority, that decides whether a thread is eligible for a performance core. Nothing in the tree called pthread_set_qos_class_self_np, so a streaming thread could be scheduled onto an efficiency core with nothing to signal it had happened. This sets a QoS class on the pipeline's streaming threads.

Why it replaces the priority path on macOS rather than adding to it

The two settings are mutually exclusive, in both orders (verified on macOS 15 / M2):

  • pthread_setschedparam first makes a later pthread_set_qos_class_self_np return EPERM.
  • Setting a class first is silently reset to UNSPECIFIED by a later pthread_setschedparam.

So macOS no longer calls the priority API at all. Linux and Windows keep their existing priority paths untouched.

This also retires a macOS priority path that never worked: the crossplatform value 80 is outside the 15..=47 range macOS accepts, so every "High" flow logged a warning and reported achieved=false on every thread.

Mapping

High maps to USER_INITIATED, not USER_INTERACTIVE. Both are performance-core classes — only BACKGROUND is confined to the efficiency cluster — but USER_INTERACTIVE is the band Apple reserves for main-thread UI work, which is where Strom's own native GUI runs. Encoder threads saturate every core they are given, so sharing that band makes the GUI compete with them at equal priority. Realtime is an explicit request for the maximum, so it does take USER_INTERACTIVE, falling back to USER_INITIATED when the task runs under a QoS clamp and the kernel refuses it with EPERM. Normal leaves the class alone.

Where the call goes

On the streaming thread itself — the bus StreamStatus::Enter handler and the session pad probe — because a thread that sets no class of its own gets QOS_CLASS_DEFAULT rather than inheriting anything meaningful. Placing it there also reaches element-internal workers that never post a StreamStatus of their own (libx264's frame threads): those are created from the streaming thread during caps negotiation, after Enter, and a thread created by a thread with an explicit class inherits it.

Setting the class is followed by a read-back, so a class that does not stick is reported as an error instead of leaving a silently unplaced thread.

Measured effect: 4.3% overall, ~7% under CPU contention

Supersedes the earlier no-effect result, which was taken before #735 existed and could not observe the setting: App Nap had clamped both arms to the efficiency cores, so every thread sat at scheduling priority 4 regardless of its class.

Re-measured on an M2 (4P+4E), every arm carrying #735, one binary with a runtime switch so the arms differ only in whether the class is set. Paired runs, fresh backend each, within-pair order counterbalanced:

experiment n pairs effect 95% CI
A/A control, both arms QoS on 40 +0.54% [-1.25, +2.33] null, as it must be (Wilcoxon p = 0.97)
A/B, host load median 5.7 40 -4.32% [-7.03, -1.61] significant, Wilcoxon p = 0.0011, 29/40 pairs
A/B, idle host 13 +0.75% [-1.04, +2.55] ns

Negative means QoS-on is faster. The A/A control is what licenses reading the -4.32% as real rather than as harness bias.

The benefit scales with contention. Splitting the 40-pair run by host load: -1.55% in the low-load half (mean load 5.0), -7.09% in the high-load half (mean load 9.1), corr(load, effect) = -0.425, t = -2.89. That is the expected shape — with P cores idle the scheduler lands streaming threads there anyway, so the class changes nothing; it earns its keep when it has to win a P core against competition. The idle-host row is consistent with the low-load half rather than contradicting it.

Mechanism: the QoS arm logs thread QoS class set to UserInitiated per streaming thread with readback confirmed, and ps -M shows those threads at priority 37; the other arm leaves them Default at 31.

Caveats: the idle-host arm is 13 pairs rather than 40, the load split is post-hoc, and the benchmark sets x264 threads=1, so it does not exercise the frame-thread inheritance path described above.

Tests

7 unit tests in thread_priority.rs, including read-back assertions that High lands in USER_INITIATED, that Realtime lands in one of the two performance-core classes, and that Normal leaves the class untouched.

Rebased onto #722's fix(monitor) commit with no conflict: this commit changes what happens inside the two StreamStatus arms, that one changes how the thread id is captured around them. Re-verified after the rebase on macOS (Apple Silicon, macOS 26.5): cargo test --features efp — 558 lib tests and every integration test pass, cargo clippy --all-targets --features efp -- -D warnings and cargo fmt --check clean.

One unrelated flake, once in eight full runs: config::tests::test_from_figment_cli_args_override panicked on Invalid argument (os error 22). It is the only config test without #[serial], and its serialized neighbours set_current_dir() into a TempDir and drop it while this one reads the process cwd. Pre-existing and untouched here — the added tests only shift the scheduling that exposes it.

wagenet and others added 2 commits August 28, 2026 17:54
The thread and CPU views were empty on macOS: per-thread sampling read
/proc/{pid}/task/{tid}/stat and /proc/stat behind cfg(target_os = "linux"),
and the other arm returned nothing.

Add a macOS backend using thread_info(THREAD_BASIC_INFO), which reports
cumulative user+system time per thread. The registry now stores the mach
thread port rather than the pthread_t, captured on the streaming thread
itself in get_current_thread_native_id(), so the sampler never dereferences
a pthread_t whose thread may already have exited.

mach has no /proc/stat equivalent, but the Linux denominator is by
construction elapsed wall time times the core count, so computing it
directly yields the same percentage with the same meaning: 100.0 is one
saturated core, N * 100.0 the maximum on N cores. Both platforms now share
cpu_usage_percent() for that conversion, which leaves the Linux result
unchanged.

A thread that has exited returns MACH_SEND_INVALID_DEST; it yields no
sample, drops its stored baseline so a recycled port name cannot produce a
bogus spike, and is not logged, since the race recurs on every tick until
the registry entry is cleaned up.

Uses libc, already a dependency; no new crate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyhYeoitqyNn272UkivKMj
On heterogeneous CPUs the percentage alone is misleading: the clocks on both
platforms charge a thread for time spent on a core without adjusting for how
fast that core is. Two threads both reading 100.0 measured 4.5x apart in
throughput on an M2, and a thread moved onto a faster core reports a *lower*
percentage for the same work.

This matters for the upcoming thread QoS work, which will be evaluated with
exactly this number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyhYeoitqyNn272UkivKMj
@wagenet
wagenet marked this pull request as draft August 30, 2026 00:21
@srperens srperens mentioned this pull request Aug 31, 2026
wagenet and others added 2 commits September 3, 2026 14:57
…gistered

pthread_mach_thread_np() returns the calling thread's port name without taking a
user reference on it. When the thread exits the name is freed and the kernel may
hand it to any other port, so a name cached in the thread registry can end up
naming something else entirely. thread_info() on a name that has been recycled
to a guarded port -- libdispatch and XPC guard theirs -- raises EXC_GUARD and the
kernel kills the process. There is no error return to check:

  EXC_GUARD / GUARD_TYPE_MACH_PORT / ILLEGAL_MOVE on mach port 161251
  ThreadCpuSampler::sample <- AppState::get_thread_stats <- handle_socket_inner

Registration now goes through ThreadHandle, which captures the port with
mach_thread_self() -- the same name, with a send right -- and releases it in
Drop. Every way a registry entry goes away releases the reference exactly once,
because they are all HashMap drops: unregister, unregister_flow, dropping the
registry, and one register overwriting another key. The reference lives in an
Arc, so a snapshot from get_all() keeps the port alive across the sampler's mach
calls even if the entry is unregistered concurrently.

read_thread_cpu_time() takes a &ThreadHandle rather than a u64, so the invariant
holds by construction: a port name reaches thread_info() only while something
guarantees it still names the same thread. A thread that has exited keeps its
name and mach declines to answer for it, which the sampler already treats as no
reading this tick.

Threads registered from the session pad probe never post a Leave message, so
their handles are held until the flow's entries are dropped at stop: a bounded
number of names per running flow.

Tests cover retention, clone lifetime and each removal path, plus
thread_port_lifetime_test, which drives real GStreamer streaming threads through
the bus sync handler and the sampler. GStreamer returns streaming threads to a
glib pool that keeps idle ones for 15 seconds, so that test shortens the
retention to make teardown actually end threads; at the default no port is ever
released and it guards nothing. All five fail if the fix is reverted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ores

On Apple Silicon it is the QoS class, not the pthread priority, that decides
whether a thread is eligible for a performance core. Nothing in the tree called
pthread_set_qos_class_self_np, so a streaming thread could be scheduled onto an
efficiency core with nothing to signal it had happened.

The two settings turn out to be mutually exclusive on macOS, in both orders --
pthread_setschedparam makes a later pthread_set_qos_class_self_np return EPERM,
and setting a class first is silently reset to UNSPECIFIED by a later
pthread_setschedparam (both verified on macOS 15 / M2). So this is a
replacement on macOS rather than an addition: the QoS class is the setting that
governs core placement, and macOS no longer calls the priority API at all.
Linux and Windows keep their existing priority paths untouched.

High maps to USER_INITIATED rather than USER_INTERACTIVE. Both are
performance-core classes -- only BACKGROUND is confined to the efficiency
cluster -- but USER_INTERACTIVE is the band Apple reserves for main-thread UI
work, which is where Strom's own native GUI runs. Encoder threads saturate
every core they are given, so sharing that band with the GUI makes the GUI
compete with them at equal priority. Realtime, an explicit request for the
maximum, does take USER_INTERACTIVE, falling back to USER_INITIATED when the
task runs under a QoS clamp and the kernel refuses it with EPERM.

The call goes on the streaming thread itself, in the bus StreamStatus::Enter
handler and the session pad probe, because a thread that sets no class of its
own gets QOS_CLASS_DEFAULT rather than inheriting anything meaningful. Placing
it there also reaches the element-internal workers that never post a
StreamStatus of their own (libx264's frame threads): those are created from the
streaming thread during caps negotiation, after Enter, and a thread created by
a thread that has an explicit class inherits it.

Setting the class is followed by a read-back, so a class that does not stick is
reported as an error instead of leaving a silently unplaced thread.

Incidentally this also retires a macOS priority path that never worked: the
crossplatform value 80 is outside the 15..=47 range macOS accepts, so every
"High" flow logged a warning and reported achieved=false on every thread.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P9VpaRyFRD76GpdtLdAczi
@wagenet
wagenet force-pushed the wagenet/macos-thread-qos branch from 8a7e1a6 to 23b6435 Compare September 3, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant