fix(recorder): retry an EOS the stalled track refused, and end it sooner

[wagenet]

Stacks on #757. GitHub cannot base a PR on a branch in the fork, so the diff below includes
757's commit; the change to review is the tip commit.

In the field the #757 watchdog fired correctly, its EOS was refused, and the recording stayed
frozen — rec_p3:video_input_0 refused the EOS that would end its track.

Why push_event returned false

Traced through gst_pad_push_event in GStreamer 1.28.6. A sticky, serialized event on a src pad
is refused in exactly five states: the pad is flushing, the pad is no longer activated
(GST_PAD_MODE_NONE), the pad already carries a sticky EOS, the pad is unlinked (push_sticky
forgives NOT_LINKED for every event except EOS), or the peer is flushing, already at EOS, or
out of its parent. A busy downstream is not among them — gst_pad_send_event_unchecked takes
the peer's stream lock, so a blocked branch blocks the caller. A refusal is therefore positive
evidence that the branch was coming apart, but says nothing about which of the five it was.

I could not reproduce a refusal from the WHIP teardown. On a headless rig I got the field's
ordering — session cleanup 5s before the watchdog fired, against 4s in the field — and the EOS
was accepted both times (sent=true, from a temporary probe). That matches the code: a WHIP
session lives in its own gst::Pipeline, and tearing it down starves the slot's appsrc in the
main pipeline without flushing anything below it. So the flushing-pad mechanism named in the
report is plausible but unproven, and this fix does not depend on which state it was.

The change

A refused EOS no longer retires the track. retired is set inside end_stalled_track, and
only after delivery; the watchdog retries on later polls.

A second route out of the wait. A failed push still leaves the sticky EOS on the pad, so that
pad refuses every retry — retrying alone would not have helped. The EOS now falls back to the
splitmuxsink sink pad: the pad the muxer is actually waiting on, which no upstream teardown can
invalidate, and whose stream lock is free because the stalled track is by definition the one
nothing is pushing into. The block's input stays first choice, since it lets the parser drain its
last frame on the way past.

The stall is detected on the spread between tracks. The 5s timeout was never the reason it
took 32s: frozen required all live tracks to be quiet, and splitmuxsink kept taking the
surviving track into its internal queues for another 27s. Ending on the lag instead — the
furthest-behind track is quiet and the recording has moved at least the timeout past it —
fires while the others are still delivering. An overloaded box stalls every track at the same
running time, so the spread stays flat and this does not fire; the all-quiet rule stays for
tracks that stop together, where no spread ever opens.

I did not wire the recorder to WHIP session teardown. An output block cannot know which of its
tracks a given session feeds, and the spread is a direct measurement of the thing that matters —
the muxer waiting — rather than a proxy for one cause of it.

Tests

Full suite on macOS with STROM_REQUIRE_GST_PLUGINS=1: green, nothing skipped. No new elements,
so no CI package change. Each new test checked against the reverted code:

Test	Reverted
a_track_that_refused_the_eos_is_not_retired	fails — retired on an undelivered EOS
an_unlinked_input_falls_back_to_the_muxer_pad	fails — no second route
a_track_the_recording_has_moved_on_without_is_the_one_ended	fails — waits for every input
a_stalled_track_whose_branch_is_coming_apart_still_ends	fails: 0 video buffers reached the muxer in 5 s

The integration test unlinks the stopped track's branch to make the input refuse — the one of
the five states a test can arrange without racing a real teardown — and asserts the refusal
before relying on it, so it cannot silently stop testing anything.

Two caveats. I could not build a rig that reproduces the 32s delay: in mine even an AAC track
backs up at the muxer within 5s, so old and new rules both recovered in 7s. The 27s is from the
field log's own numbers and the rule change is guarded by unit tests, not by a timing test.
End-to-end on the rig (WHIP seat SIGKILLed, so cleanup runs with no DELETE) the dead track is
ended 7s after the kill, the live track is left alone, and the file keeps growing through the
teardown that follows.

Out of scope

GStreamer-CRITICAL: Trying to dispose element bin31, but it is in PAUSED instead of the NULL state comes from the WHIP session cleanup path. Per CLAUDE.md that is a P0 and the usual cause
is a strong gst::Element/gst::Bin reference captured in a signal-handler closure. I did not
look for it and have no view on where it is — worth its own issue.

🤖 Generated with Claude Code