Meter TTS usage on characters sent to the provider - #10
Merged
Conversation
TTS providers bill on characters sent over the wire, not characters the user hears: Pipecat pushes each sentence to the provider as the LLM streams it, so on interruption Deepgram has already received (and bills for) text whose audio is discarded by Clear. Pipecat's built-in TTSUsageMetricsData metric cannot measure this — the Deepgram websocket service never emits it, and the base-class fallback drops accumulated text on interruption, which is exactly where billing diverges from the transcript. Add TtsUsageMeterMixin counting characters at the run_tts seam — the exact prepared text sent to the provider — on both instrumented TTS services, and emit a usage.tts trace event (provider, model, voice, sent_characters) at session teardown next to usage.stt. Also rewrite docs/cost-metering-design.md §6 with the verified billing semantics: sent = billed for Deepgram; ElevenLabs deducts credits on successful generation so sent_characters is a tight upper bound there; per-provider differences belong in the pricing catalog, not the meter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TTS providers charge for the characters you send them, not the characters the user ends up hearing. Pipecat forwards each sentence to the TTS service as soon as the LLM produces it, which is usually a few sentences ahead of what's actually playing. So when a user interrupts, the provider has already received text it will never speak, and we still pay for it. Counting characters from the transcript will always come in under the real bill.
We also can't lean on Pipecat's built-in usage metric. The websocket Deepgram service never fires it, and the base class throws away its accumulated text on interruption, which is the one case we care about most.
The fix is to count it ourselves inside
run_tts, since that's where text actually goes out to the provider. It's the same trick the STT byte meter already uses for audio.Changes
TtsUsageMeterMixincountslen(text)inrun_ttson bothAdkDeepgramTTSServiceandAdkElevenLabsTTSService— the exact prepared text sent to the provider.usage.ttstrace event per session at teardown, next tousage.stt:{provider, model, voice, sent_characters}. Usage facts only; cost is computed later from the pricing catalog.docs/cost-metering-design.md§6: the earlier plan (flipenable_usage_metrics=True, readTTSUsageMetricsDatain MetricsSink) doesn't work — the Deepgram websocket service never emits that metric, and interrupted turns are dropped by the base class. Also documents verified billing semantics: sent = billed for Deepgram; for ElevenLabs (credits deducted on successful generation, context closed on interruption)sent_charactersis a tight upper bound.server/tests/test_tts_usage_meter.py, mirroring the STT meter tests.Verification
uv run pytest: 4 new tests pass; the 2test_agent_configfailures also fail on the base branch (pre-existing).uv run ruff check .: the 4 reported errors pre-exist on the base branch; this change adds none.sent_charactersagainst the Deepgram console per-request character counts and the ElevenLabs subscription character counter (design doc §6.4).Stacked on #9 (
feat/stt-usage-metering) since it extends the same runtime file and teardown block.