Skip to content

Add Mandarin and Spanish lip-sync profiles - #1

Draft
Muurrphy wants to merge 1 commit into
mainfrom
codex/multilingual-lipsync
Draft

Add Mandarin and Spanish lip-sync profiles#1
Muurrphy wants to merge 1 commit into
mainfrom
codex/multilingual-lipsync

Conversation

@Muurrphy

Copy link
Copy Markdown
Owner

What changed

  • add language-tagged Spanish G2P with dedicated five-vowel articulation targets
  • support generic/Latin-American seseo and the es-ES dental distinction
  • add optional phrase-aware Mandarin pinyin with initial/final decomposition, u/ü, retroflex/palatal/dental categories, compound-final paths, and tone metadata
  • preserve append-only streaming with a two-Han-character planning horizon and emitted-prefix validation
  • route ElevenLabs alignment through the selected language profile
  • update the English and Chinese READMEs, architecture, roadmap, research boundaries, and changelog
  • add Mandarin/Spanish fixtures, examples, and research notes

Why

The previous implementation treated Spanish Latin text as English and reduced every Han character to one neutral SYLLABLE target. That produced incorrect visible articulation for both languages.

Impact

Users can generate distinct English, Mandarin, and Spanish timelines through the CLI and provider APIs. Spanish requires no new runtime dependency. Phrase-aware Mandarin is available through the optional mandarin extra.

Validation

  • pytest: 35 passed
  • ruff check src tests examples benchmarks
  • ruff format --check src tests examples benchmarks
  • generated and inspected Spanish and Mandarin HTML/timeline demos
  • verified dependency-free Mandarin fallback emits low-confidence neutral events instead of failing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant