Skip to content

fix(budget): debit from metered usage instead of input-only estimates - #592

Open
yzxcj797 wants to merge 1 commit into
yc-software:mainfrom
yzxcj797:fix/budget-metered-usage-586
Open

fix(budget): debit from metered usage instead of input-only estimates#592
yzxcj797 wants to merge 1 commit into
yc-software:mainfrom
yzxcj797:fix/budget-metered-usage-586

Conversation

@yzxcj797

@yzxcj797 yzxcj797 commented Aug 18, 2026

Copy link
Copy Markdown

Summary

Implements the "smallest useful change" from #586.

Root cause (as diagnosed in the issue, verified)

The budget debited estimateCostUsd(rec.inputTokens) — a fixed $5/MTok over an input-only estimate — at every recordModelCall site, while the provider-metered numbers (input, output, cacheRead/Write, costUsd) were already arriving one callback over, on recordLlmRequest, and being persisted to session_llm_requests with full usage. Consequences per the issue's table: cheap-input models overcharged up to 5×, and output tokens — the larger share of a real bill — completely invisible.

Fix

  1. modelCallCostUsd(rec, usage) (in ratelimit/budget.ts) resolves one call's cost in priority order:
    • provider-computed usage.costUsd when positive → used verbatim;
    • metered input + output at the existing fixed rate when tokens exist but the provider didn't price them (output no longer invisible);
    • the legacy input-only estimate when no metered usage exists.
  2. The turn path's recordLlmRequest callback now also debits the budget with the metered figure — charging only the positive difference over what the upfront input-only estimate already booked for that step's input. No double-counting (the upfront charge isn't refunded when metered cost is lower), no refunds.
  3. The upfront recordModelCall estimate is untouched — budget ceilings still trip promptly for turns that never report usage (detection, compaction, security-screen paths).

Scope note

The turn_metrics observability gap (no cost/output columns at the metrics sink) is the other half of the issue and is deliberately left for its own change.

Tests

modelCallCostUsd resolution tests in test/budget.test.ts (provider cost verbatim; metered-tokens fallback prices input+output; null/undefined usage falls back to the estimate). Budget suite: 5/5 green.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

The budget priced every model at a fixed $5/MTok over an input-token
estimate: output tokens (the larger share of a real bill) were
invisible, cheap-input models were overcharged up to 5x, and the
provider-metered usage the harnesses already report through
recordLlmRequest was never consulted (yc-software#586).

Add modelCallCostUsd(rec, usage): provider-computed costUsd when
positive; metered input+output at the fixed rate when tokens exist
without a provider price; the legacy input-only estimate otherwise.
The turn path's recordLlmRequest now debits the budget with the
metered figure, charging only the positive difference over the
upfront estimate already booked for that step's input (no double
count, no refunds). The upfront estimate is untouched so ceilings
still trip promptly for turns that never report usage.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant