Skip to content

feat(liveaudiorouter): add a live matrix audio router with capability parity - #737

Draft
srperens wants to merge 2 commits into
mainfrom
agent/feat-661-live-audio-router
Draft

feat(liveaudiorouter): add a live matrix audio router with capability parity#737
srperens wants to merge 2 commits into
mainfrom
agent/feat-661-live-audio-router

Conversation

@srperens

@srperens srperens commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

BLOCKED — the baseline failure is unresolved, and #740 supersedes this design. Refs #661. Nothing was compiled on the agent runner (no cargo, no GStreamer); every claim below comes from CI.

Status 2026-09-02

Run 33498186703 on e3e57c2: fmt, clippy, WASM, API contract and both Linux builds green, so it compiles. Check (Linux) fails at Run tests on backendliveaudiorouter_test: 7 passed, 1 failed. The 5 parity tests pass; the 2 matrix tests are pure matrix_from_properties units with no audio.

The end-to-end test fails at its baseline, before any matrix change: expected audio on output channel 0 before the change, got peaks [-inf, -inf]. So no audio has been shown to pass through this block at all, and the live-change claim is untested.

I could not resolve it: the block's warn!/error! emit nothing under cargo test (no logger), so CI gives no diagnostic, and there is no GStreamer here to bisect with.

#740 supersedes this by design, not by fixing this failure. It measures three properties of the interleave + audiomixmatrix foundation and replaces it with tee → volume → audiomixer. One correction: its interleave-stall measurement concerns an input pad requested but left unconnected, which does not explain this failure — the failing test configures a single connected input (backend/tests/liveaudiorouter_test.rs:378 ("num_inputs", PropertyValue::UInt(1)),). The baseline cause is still unknown.

Fixing this would mean adopting #740's design and duplicating that PR, so I stopped. Keeping this open as the comparison point or closing it is the maintainer's call; I have not touched #740.

Reading of the decision

The reply of 2026-08-31 11:06Z carries /agent-fix C --accept-radius SHARED --accept-excluded none, superseding the retraction three minutes earlier; I read it as a decision. The marker records no radius and work=unset; I assessed radius SHARED, excluded=none.

Problem

builtin.audiorouter encodes routing as topology — an audiomixer per summed output channel (backend/src/blocks/builtin/audiorouter.rs:181 let mixer = gst::ElementFactory::make("audiomixer")) — so changing a crosspoint rebuilds the graph, hence live: false on that block.

Change

New builtin.liveaudiorouter: same property names, types, defaults and pad shape, routing_matrix live: true; builtin.audiorouter untouched.

The reply asked for a translate_property handler like audiogain.rs. Not usable: matrix is a double array-of-arrays and PropertyValue has no array variant (types/src/element.rs:138pub enum PropertyValue {). I used the interceptor the repo already has — backend/src/blocks/builtin/time_offset.rs:72pub fn try_apply_live_offset(. The only deviation from the decision's letter.

Not verified

The routing path, end to end; more than 2 channels; rates other than 48 kHz; macOS and Windows; openapi.json. The frontend change has no test and was not run in a browser. The layout registry (iXcY keys) is keyed by element id and never pruned.

Blast radius

get_builder and get_all_builtin_blocks gain one arm each; no existing arm changes. update_element_property gains one interceptor, returning early only for routing_matrix on an id ending :matrix.

This was referenced Sep 1, 2026
@srperens
srperens force-pushed the agent/feat-661-live-audio-router branch from 4eafedf to ff29aa8 Compare September 1, 2026 10:27

@srperens srperens left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: Request changes — mechanical, per REVIEW.md: Check (Linux) is red on a test this diff itself adds, and the crosspoint-fade claim underlying the design is contradicted by the code. Also: a stronger, tested alternative for the same issue (#740) now exists and should factor into the maintainer's decision here.

Claims

Claim Verdict Evidence
Live routing_matrix writes route through a dedicated intercept, not the generic translate path CONFIRMED backend/src/gst/pipeline/properties.rs:245 calls liveaudiorouter::try_apply_live_matrix ahead of the generic translation block
The deinterleave_out pad-added closure captures no strong pipeline/element ref CONFIRMED backend/src/blocks/builtin/liveaudiorouter.rs:404-432 — only instance_id.to_string() and WeakRefs move in; the bin comes from element.parent()
Check (Linux) is green on liveaudiorouter_test at head e3e57c2 CONTRADICTED CI run 33498186703/job 99825164281, re-pulled directly (not just the PR body's own quote): test result: FAILED. 7 passed; 1 failed, panic at liveaudiorouter_test.rs:437:5: `expected audio on output channel 0 before the change, got peaks [-inf, -inf]` — this is the pre-change baseline assertion, not the live-move assertion
A routing change fades smoothly rather than stepping CONTRADICTED liveaudiorouter.rs:182 `element.set_property("matrix", matrix_to_value(&matrix));` — a hard 0.0/1.0 coefficient replace on audiomixmatrix; no crosspoint_fade_ms property exists anywhere in this block's definition (zero matches in the file)
The per-instance channel-layout registry is reclaimed when a flow tears down CONTRADICTED record_layout (liveaudiorouter.rs:212-213) inserts into the process-global LAYOUTS static; the file has no matching .remove(...) call. Self-flagged in the PR body as "worth a look" — confirmed still open

Diagnosis — The module doc (liveaudiorouter.rs:1-14) collects every input channel through a single interleave ahead of the audiomixmatrix. The failing test configures one input stream and fails before any routing_matrix write — baseline audio flow is unreliable even in the simplest case, worse than "a live move stalls." This matches the interleave-stall failure mode #740's design record names as its reason to reject this approach; #740 ships a green equivalent test against a non-interleave crossbar.

Blast radiusSHARED: the new intercept in properties.rs sits ahead of the generic per-property path, but is guarded by element_id.ends_with(":matrix") and prop_name == "routing_matrix", so it cannot affect builtin.audiorouter or any other block. New file liveaudiorouter.rs only; mod.rs registration is additive.

Tests & CIAPI Contract Check, Check & Build (WASM), both Build (Linux) jobs green; macOS/Windows skipping as expected. Check (Linux) red as above — the covering test for this change, so a blocking finding per REVIEW.md, not a caveat. Frontend changes (audiorouter.rs, graph/rendering.rs, properties.rs) have no CI coverage — strom-frontend never appears in a cargo test --package line in .github/workflows/ci.yml (same gap noted against #740 this run).

Design record — First review of this PR; no prior marker to supersede. #661 now has two live candidates: this PR's single-audiomixmatrix design (fewer elements, but the collect-pads stall shown above and no fade support) versus #740's tee/volume/audiomixer crossbar (a queue per output bus, per-crosspoint volume fades, its own equivalent test green). The maintainer's choice should weigh #740's passing baseline against this PR's failing one.

Confidence: HIGH

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant