Skip to content

[codex] Add Loom3 lip-sync sequence compiler - #143

Closed
meekmachine wants to merge 1 commit into
mainfrom
codex/lipsync-sequence
Closed

[codex] Add Loom3 lip-sync sequence compiler#143
meekmachine wants to merge 1 commit into
mainfrom
codex/lipsync-sequence

Conversation

@meekmachine

Copy link
Copy Markdown
Owner

Summary

  • Add a Loom3-owned lipsyncSequence(input, options) API that compiles provider viseme timing or existing viseme snippet JSON into one managed mixer clip.
  • Add compileLipsyncSequence(...) plus public types for Microsoft/Azure event shapes, sequence options, normalized events, and compiled sequence data.
  • Resolve provider source IDs through visemeSlots[*].providerIds, optional source mappings, profile slot ids/labels, morph-target overrides, and per-sequence jaw activation.
  • Let clip playback use per-sequence visemeJawAmounts so jaw rotation stays inside Loom3's existing auto-viseme-jaw mixer path.
  • Update README docs to separate provider source IDs from profile/rig viseme slots and document the new API.

Research notes

Microsoft's current Azure Speech viseme docs describe 22 viseme IDs and state that Audio offset is emitted in 100 ns ticks. The same docs expose an Animation payload for SVG/blend-shape output. This PR accepts those source IDs/ticks directly and preserves raw animation payloads on normalized events for downstream inspection/future blend-shape support.

Source: https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-speech-synthesis-viseme

Validation

  • npm run typecheck
  • npm test
  • npm run build

Fixes #141

@meekmachine

Copy link
Copy Markdown
Owner Author

Full triage — CLOSE_PR

lipsyncSequence / provider event compiler is a planner/adapter, not Embody apply. Move to Polymer LipSync (JALI #9–14).

Plan: https://github.com/meekmachine/Polymer/discussions/59

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move lip-sync sequence compilation into Loom3

1 participant