Skip to content

perf(shred): reuse recovered shred metadata — trim recovery-path parse and copy work#92

Merged
Lythaeon merged 3 commits into
mainfrom
perf/fec-recovered-parse-skip
Apr 9, 2026
Merged

perf(shred): reuse recovered shred metadata — trim recovery-path parse and copy work#92
Lythaeon merged 3 commits into
mainfrom
perf/fec-recovered-parse-skip

Conversation

@Lythaeon

@Lythaeon Lythaeon commented Apr 9, 2026

Copy link
Copy Markdown
Owner

Description

Reduce recovered-shred packet-worker overhead without changing SOF API contracts or feature scope.
This PR stops reparsing recovered data shreds in packet workers and reuses recovered header metadata to build payload fragments without cloning the recovered payload into a second buffer.

Changes

Detailed list of what changed:

  • crates/sof-observer/src/shred/fec/recover.rs: return recovered data packets together with parsed data-header metadata instead of only raw bytes.
  • crates/sof-observer/src/shred/fec/core.rs: thread the recovered metadata through FecRecoverer::ingest_packet.
  • crates/sof-observer/src/app/runtime/runloop/packet_workers.rs: verify recovered bytes in place, then build SharedPayloadFragment from the recovered packet bytes and parsed offsets instead of reparsing and cloning payload data.
  • crates/sof-observer/src/app/runtime/prelude.rs: remove now-unused shred parse imports exposed by the old path.
  • Any architecture/runtime/infra implications: none; this is an internal packet-worker/FEC recovery-path optimization.

For slice-related changes, include:

  • Affected slices: shred/fec, app/runtime/runloop.
  • Cross-slice communication changes (if any) and why: the FEC layer now returns recovered packet bytes plus parsed header metadata so the runloop can reuse work already performed during recovery.
  • Migration requirements (if any): none.

Motivation

Business motivation:

  • Keep improving throughput on the existing runtime before expanding feature scope.

Technical motivation:

  • The previous path parsed recovered data shreds twice: once in FEC recovery for validation and again in packet workers before dispatch.
  • It also cloned recovered payload bytes into a second owned buffer for SharedPayloadFragment::owned.
  • Reusing recovered metadata and borrowing the payload slice from the recovered packet bytes removes both costs.

Alternative approaches considered:

  • Leaving the recovered path unchanged after the primary-path reparse fix.
  • Folding this into a larger FEC storage refactor instead of landing the smaller measured win first.

Scope and impact

  • Affected slices: packet-worker/FEC recovery hot path.
  • Data/API changes: none.
  • Backward compatibility: fully backward compatible.
  • Performance impact: positive in the recovery FEC fixture; local A/B dropped from 328 ms to 316 ms over 20,000 recovery iterations.
  • Security impact: none.

Testing

  • Unit tests
  • Integration tests
  • Manual verification
  • Performance checks (if applicable)
  • Security checks (if applicable)

Commands/results:

cargo fmt --all
cargo test -p sof --lib --no-run
cargo test -p sof --lib packet_worker_ -- --nocapture
cargo test -p sof --lib purge_older_than_removes_old_sets -- --nocapture
SOF_PACKET_WORKER_FEC_RECOVERY_PROFILE_ITERS=20000 CARGO_TARGET_DIR=/home/ac/.cache/sof-fec-recovered-current/target cargo test -p sof --lib packet_worker_recovery_fec_profile_fixture -- --ignored --nocapture
cd /tmp/sof-fec-primary-baseline && SOF_PACKET_WORKER_FEC_RECOVERY_PROFILE_ITERS=20000 CARGO_TARGET_DIR=/home/ac/.cache/sof-fec-recovered-baseline/target cargo test -p sof --lib packet_worker_recovery_fec_profile_fixture -- --ignored --nocapture

Results: passed.

Related issues and documentation

  • Fixes:
  • Related: packet/FEC perf roadmap follow-up in docs/operations/tx-reconstruction-and-packet-processing-todo-2026-03-24.md
  • Architecture docs: docs/architecture/README.md
  • Relevant ARD/ADR:
  • Operations/runbook updates: none.

Reviewer checklist

  • Code follows project standards and architecture constraints
  • Slice boundaries are respected (docs/architecture/ard/0003-slice-dependency-contracts.md)
  • Tests added/updated and passing
  • Documentation updated (README/docs/operations as needed)
  • No undocumented breaking change
  • Performance trade-offs documented where relevant
  • Security considerations addressed where relevant

Additional notes

This PR is intentionally stacked on perf/fec-primary-reparse-skip because it reuses the new parsed-header handoff added there.

@Lythaeon Lythaeon force-pushed the perf/fec-primary-reparse-skip branch from 87842d6 to aaca6c5 Compare April 9, 2026 13:21
@Lythaeon Lythaeon force-pushed the perf/fec-recovered-parse-skip branch from 38f9df5 to a747226 Compare April 9, 2026 13:24
Base automatically changed from perf/fec-primary-reparse-skip to main April 9, 2026 13:56
@Lythaeon Lythaeon force-pushed the perf/fec-recovered-parse-skip branch from ec1cc83 to 6e3d80a Compare April 9, 2026 14:00
@Lythaeon Lythaeon merged commit 5c38809 into main Apr 9, 2026
2 checks passed
@Lythaeon Lythaeon deleted the perf/fec-recovered-parse-skip branch April 9, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant