feat(transcribe): add multilingual transcription & alignment for Mandarin, Hindi, and Bahasa Indonesia

[natashaannn]

User story

As a video editor producing line-caption shorts from non-English source video, I want captions:create to transcribe and force-align Mandarin Chinese, Hindi, or Bahasa Indonesia audio correctly, so that lines.json contains accurate text and per-line timestamps in the source language instead of garbled English-model output.

Background

scripts/line-captions/create-line-captions.js drives transcription and alignment for the line-caption shorts pipeline (see docs/implementation-guides/LINE_CAPTION_SHORTS.md), but every stage is hardcoded to English:

• Transcriber (scripts/transcribe/Transcriber.js:10) defaults this.model to medium.en — an English-only whisper.cpp model. create-line-captions.js never overrides options.model, so any audio is transcribed through an English-only model regardless of its actual language.
• Transcriber.runWhisper() (scripts/transcribe/Transcriber.js:110-126) never passes a --language arg to whisper.cpp's transcribe() call at all.
• align-transcript.js defaults language: cli.language || 'en' (scripts/align/align-transcript.js:248) for the WhisperX forced-alignment step, and accepts --language on its CLI (align-transcript.js:18) — but create-line-captions.js spawns it with only --audio/--raw, never forwarding a language (create-line-captions.js:124-126).
• run_whisperx_align.py calls whisperx.load_align_model(language_code=args.language, ...) (scripts/align/run_whisperx_align.py:187-188). WhisperX's built-in alignment-model registry (whisperx/alignment.py:32-76, installed version 3.8.5) has entries for zh and hi, but no entry for id (Bahasa Indonesia). Aligning Indonesian audio will fail model resolution unless an explicit override model is supplied.
• scripts/line-captions/chunkLines.js:16-30 (buildWordGroups) reconstructs whisper's BPE sub-word tokens into whole "words" using a leading-space heuristic (!t.text.startsWith(' ')) that assumes whitespace-delimited words. Mandarin Chinese text has no spaces between words at all, so this heuristic is meaningless for zh — depending on whisper's tokenizer behavior it will either merge whole phrases into one "word" or split every character into its own "word," making wordsPerLine = 3 chunking incoherent. Hindi and Bahasa Indonesia are space-delimited, so this heuristic likely works unmodified for them, but needs verification against real transcripts.

None of this fails loudly — a non-English video run through the current pipeline today silently produces English (mis)transcriptions with no error.

Acceptance criteria

Happy path

Given a Mandarin Chinese source video
When I run npm run captions:create -- --video <path> --language zh
Then Transcriber uses a multilingual whisper.cpp model (not a .en variant), whisper.cpp is invoked with --language zh, WhisperX aligns using language_code="zh", and the resulting lines.json contains correctly segmented Chinese text with sane per-line timestamps (verified by manual read-through of lines.doc.txt).

Given a Hindi source video
When I run npm run captions:create -- --video <path> --language hi
Then transcription, alignment, and chunking all complete successfully and lines.doc.txt reads as coherent Hindi (Devanagari) text grouped by speaker.

Given a Bahasa Indonesia source video
When I run npm run captions:create -- --video <path> --language id
Then the pipeline either successfully aligns using an explicit override alignment model, or — if no adequate model exists — the alignment step degrades gracefully to the existing CUT_START_BIAS timestamp heuristic (per token.t_end being unpopulated, as already documented in CLAUDE.md's Data Schemas section) rather than crashing.

Error path / edge case

Given an unsupported/unrecognized --language code
When I run captions:create with it
Then the script fails fast with a clear error before spending time on transcription, rather than passing an invalid code through to whisper.cpp/WhisperX and failing deep in the pipeline.

Given --language is omitted
When I run captions:create
Then behavior is unchanged from today (defaults to English, medium.en model) — this issue must not change default behavior for existing English workflows.

Out of scope

• Burned-in caption rendering / font glyph support for non-Latin scripts (tracked separately — see dependency below).
• Bilingual (dual-language) subtitle rendering (tracked separately — see dependency below).
• Automatic language detection — this issue is explicit-language-flag only.
• Any change to scripts/diarize/* — diarization is acoustic/language-agnostic and needs no changes.

Technical context

• scripts/transcribe/Transcriber.js — DEFAULT_MODEL = 'medium.en' (line 10), constructor accepts options.model (line 38) but not options.language; runWhisper() (lines 110-126) builds the additionalArgs array passed to @remotion/install-whisper-cpp's transcribe().
• scripts/align/align-transcript.js — resolveArgs() (~line 240-250) resolves --language; main() spawns run_whisperx_align.py with '--language', language (line 274).
• scripts/align/run_whisperx_align.py:110,187-188 — argparse default 'en'; whisperx.load_align_model(language_code=..., model_name=...) accepts an explicit model_name override for languages missing from the default registry.
• scripts/line-captions/create-line-captions.js:20-29 (parseArgs) — needs a new --language flag; :124-126 needs to forward it to align-transcript.js; the new Transcriber({...}) call at :115 needs a model/language derived from it.
• scripts/line-captions/chunkLines.js — buildWordGroups() word-boundary heuristic; needs a per-script chunking strategy (see acceptance criteria).
• whisper.cpp multilingual models (no .en suffix, e.g. medium) support a --language arg; confirm via @remotion/install-whisper-cpp's downloadWhisperModel/transcribe API which language codes it accepts.

Implementation details

1. Add a --language <code> CLI flag to create-line-captions.js, defaulting to en (preserves current behavior when omitted).
2. Map the language code to a whisper.cpp model name (e.g. en → medium.en, anything else → medium) and pass it as Transcriber's options.model.
3. Add a language option to Transcriber and thread it through runWhisper() as a whisper.cpp --language arg.
4. Forward --language from create-line-captions.js into the align-transcript.js spawn call.
5. For id specifically: research whether a usable community wav2vec2 alignment model exists to pass as run_whisperx_align.py's model_name override; if not, make the WhisperX alignment step optional/skippable per-language so the pipeline falls back to whisper's own token timestamps instead of hard-failing.
6. Add a language-aware branch to chunkLines.js's word-boundary detection: keep the existing space-prefix heuristic for space-delimited scripts, add a character-count-based chunking mode for zh (since "words" don't exist as a concept), gated on the language passed through from create-line-captions.js.
7. Validate the --language value against a fixed allow-list (en, zh, hi, id) and fail fast with a clear message otherwise.

Additional test scenarios

• Unit test: chunkLines.js's new character-count chunking path for a synthetic Chinese-token transcript, verifying lines never exceed the character cap and never span a speaker boundary.
• Unit test: language-code validation rejects an unsupported code before any subprocess is spawned.
• Integration test (tests/integration/): create-line-captions.js --language zh against a short fixture audio clip produces a lines.json with non-empty, non-Latin text fields.
• Manual test: run all three target languages end-to-end and read lines.doc.txt for coherence (no automated correctness check for transcription quality itself).

Hard constraints

• Must not change default (en) behavior when --language is omitted.
• Must not silently mistranscribe — an unsupported language code must fail before transcription starts, not after.
• The Bahasa Indonesia alignment gap must be handled explicitly (documented fallback or explicit override model), not left to crash mid-pipeline.
• Any new field added to lines.json/transcript.json (e.g. a language meta field) must be declared in remotion/types/lineCaptions.ts and documented in CLAUDE.md's Data Schemas section, per the repo's Agent Implementation Convention.

Dependency issues

• Independent of the bilingual-subtitles issue, but that issue depends on this one (needs a working non-English transcript to translate/render against).
• Independent of the CJK/Devanagari font-rendering issue — this issue only produces correct lines.json data; it does not render captions.