Skip to content

spike(rust/fixtures): extract audio fixtures and compute JS baseline offset #94

Description

@natashaannn

User story

As a coding agent implementing the Rust FFT spike (tracked in #93), I want committed WAV fixture files and a committed JS-computed baseline offset, so that downstream issues have deterministic inputs and a ground-truth value to assert against.

Background

RFC 0001 (#90) proposes rewriting the editing pipeline in Rust. To validate the agent-driven Rust development loop before committing to that rewrite, the team is running a spike: port the FFT cross-correlation algorithm from scripts/sync/AudioSyncer.js to a standalone Rust binary and verify output matches on Mac M2, Mac M3, and Windows/NVIDIA.

This issue is the prerequisite for all other spike sub-issues. It produces two artefacts:

  1. Two WAV fixture files (reference audio and target audio, extracted from a real episode) committed to spike/audio-sync/fixtures/
  2. spike/audio-sync/fixtures/baseline.json — the offset in seconds that AudioSyncer.js computes for those same fixtures, which the Rust tests will assert against

The fixture WAVs must be extracted using the exact same FFmpeg parameters that AudioSyncer.js uses internally (-ac 1 -ar 8000 -acodec pcm_s16le -t 300) so that the JS baseline and the Rust binary are operating on identical sample data.

Acceptance criteria

Happy path

Given any two matching audio tracks from a shipped episode exist in public/sync/output/ or public/transcribe/output/
When the FFmpeg extraction commands are run and AudioSyncer.computeLag() is called against the resulting WAVs
Then spike/audio-sync/fixtures/ contains video-audio.wav, audio-track.wav, and baseline.json with non-null lagSeconds and snr fields — all three committed to the branch

Error path / edge case

Given the chosen episode's audio tracks have very low SNR (SNR < 3.0)
When AudioSyncer.computeLag() runs
Then baseline.json still records the computed lagSeconds but also sets isReliable: false — this is a valid fixture for testing the low-confidence path

Out of scope

  • Any Rust code — this issue only produces fixture files and a JS baseline
  • Modifying AudioSyncer.js or any existing pipeline file
  • Choosing which episode to use — any shipped episode with two distinct audio tracks is acceptable

Technical context

FFmpeg extraction command (matches AudioSyncer.extractVideoAudio / convertAudioToWav at scripts/sync/AudioSyncer.js L115–135):

# Reference track (video's embedded audio):
ffmpeg -i <video.mp4> -t 300 -vn -ac 1 -ar 8000 -acodec pcm_s16le -f wav spike/audio-sync/fixtures/video-audio.wav -y

# Target track (external audio):
ffmpeg -i <audio.mp3|wav> -t 300 -ac 1 -ar 8000 -acodec pcm_s16le -f wav spike/audio-sync/fixtures/audio-track.wav -y

JS baseline computation. The simplest approach is a one-off Node script that imports AudioSyncer and calls computeLag() against the two WAV files already in fixtures/. The WAV files must be the 8 kHz mono versions produced above (not the original high-res files) so that the sample count matches exactly what the Rust binary will receive.

Baseline JSON schema:

{
  "lagSeconds": -1.234,
  "snr": 12.5,
  "isReliable": true,
  "sampleRate": 8000,
  "syncFrameRate": 30
}

sampleRate and syncFrameRate are recorded as documentation for the Rust implementors — they must use these same constants.

Fixture size: at 8 kHz mono s16le for 300 s, each WAV is ≈ 4.8 MB. Both files together are under 10 MB — acceptable to commit directly.

Output directory: spike/audio-sync/fixtures/ — create it if it does not exist. No other directories or files outside spike/ should be touched.

Implementation details

  1. Identify two matching audio tracks from any shipped episode (video file + its corresponding external audio file). Synced outputs are in public/sync/output/; raw episode audio may be in public/transcribe/output/ or public/audio/.
  2. Run the two FFmpeg commands above to produce spike/audio-sync/fixtures/video-audio.wav and spike/audio-sync/fixtures/audio-track.wav.
  3. Write and run a one-off Node script (does not need to be committed) that calls AudioSyncer.computeLag() with videoPath and audioPath set to the two fixture WAVs. Capture lagSeconds, snr, isReliable.
  4. Write spike/audio-sync/fixtures/baseline.json with those values plus sampleRate: 8000 and syncFrameRate: 30.
  5. Commit spike/audio-sync/fixtures/video-audio.wav, spike/audio-sync/fixtures/audio-track.wav, and spike/audio-sync/fixtures/baseline.json.

Additional test scenarios

  • N/A — this issue produces test fixtures, it does not write tests

Hard constraints

  • The WAV files must be extracted at 8 kHz mono s16le with -t 300 — no other sample rate or bit depth; the Rust binary will use these exact files
  • baseline.json must be produced by actually running AudioSyncer.computeLag(), not estimated manually
  • No files outside spike/ may be created or modified

Dependency issues

Independent — this is the first issue in the spike and has no predecessors.
All other spike sub-issues depend on this one.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions