fix(server): read the AI meter alone for a cost quota's pricing blackout - #1201
Conversation
The blackout verdict counted every usage event for the project, so a window holding only unpriced platform events read as a pricing blackout and an enforcing cost cap refused every generation. That could not recover: the refusal blocked the very generation that would have landed the first priced AI event, so the window stayed all-unpriced. A deployment that prices no compute hit it on its first cost cap, because no prices ship by default and every orchestration node meters compute_execution. The verdict now reads llm_tokens events alone. A platform meter is priced by the operator from a soat SKU rather than by a tenant's provider, so leaving it unpriced says nothing about whether AI spend is measurable. Reading the AI meter alone also stops a priced platform event from masking a genuine AI blackout, which the whole-window verdict allowed. The aggregate the cap compares against is unchanged: every priced meter is real spend and still counts, platform included. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DRwsGecUbNX8PcAzZ5L2ov
Deploy Outputs
|
…ed window The added step asserted a generation would succeed under a live cost cap. Every price this stack writes is dated 2099, so no llm_tokens event here is ever priced and the window is a genuine AI blackout that block must refuse — the premise was wrong, not the refusal. Asserting the 409 covers the regression the meter scoping could actually introduce: a verdict narrowed far enough to never fire. The platform-only window and the priced-platform-masking cases are pinned by the unit tests, which can control the meter mix that this stack cannot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DRwsGecUbNX8PcAzZ5L2ov
|
The failure is Why it is not this change:
The flake has a real mechanism, not just bad luck. The test waits on the wrong condition: await waitFor(() => {
return callsToUrl(url).length > 0; // the outbound call has been observed
});
await stranded.reload();
expect(stranded.status).toBe('success'); // ...but this is written after it resolves
I am not fixing that here: it is an unrelated module and would widen a quota fix into a webhook one. It is worth its own small change, and I would rather it be visible than silently re-run forever. Generated by Claude Code |
Summary
A
cost_usdquota's pricing-blackout verdict counted every usage event for the project, whatever its meter type. A window holding only unpricedcompute_executionevents therefore read as a blackout, and an enforcing cap refused every generation with409 QUOTA_UNENFORCEABLE.That state could not recover. The refusal blocked the very generation that would have landed the first priced
llm_tokensevent, so the window stayed all-unpriced and the cap stayed dead. It is not an edge case: no prices ship by default, and every orchestration node execution meterscompute_execution, so any deployment that has run an orchestration and then sets its first cost cap hits it.The verdict now reads
llm_tokensevents alone.Why the AI meter alone is the right line
A platform meter is priced by the operator from a
soatSKU, not by a tenant's AI provider. A deployment that leaves compute unpriced has not lost the ability to measure AI spend, which is what the cap is about, so an unpriced platform meter is not evidence that the cap protects nothing.Reading the AI meter alone also fixes the opposite error the whole-window verdict allowed: a single priced platform event masked a genuine AI blackout, because the old check only needed one priced event anywhere in the window to conclude all was well. Both directions are covered by unit tests.
The aggregate the cap compares against is unchanged. Every priced meter is real spend and still counts toward the limit, platform included. Only the blackout verdict is meter-specific.
Relationship to #1177
Same function, same underlying weakness, different bug — this does not close #1177.
That issue quotes this exact expression and objects that it is blind to a partially priced window: with model A priced and model B not,
priced.length > 0, so the verdict reads0and neither the exception nor the refusal fires while B's spend goes uncounted. After this change that case is unchanged — both models arellm_tokens, sopricedAiEventsis non-empty and the verdict is still0. The component-level sub-case in that issue is untouched too.Two ways this bears on it, worth knowing before picking it up:
soat/compute-secondas just another unpriced model on every deployment that does not price compute, which is precisely the noise this PR removes. The two changes compose: this one addsmeterTypeto the aggregation query, that one addsmodeland probablyaiProviderId.bd853d8aand will not match after this merges. The expression moves and gains the meter filter.A fuller recommendation for that issue is posted on it directly.
Changes
lib/quotaEnforcement.tscost_usdaggregate readsmeterTypeand scopesunpricedEventCounttollm_tokenstests/.../quotaGenerationEnforcement.test.tsmeterTypeoption on the seeding helperopenapi/v1/quotas.yamlon_unpriceddescribes which meter the verdict readsdocs/modules/quotas.mdtests/smoke-tests.shWhere the coverage sits, and why
The two cases this PR changes are pinned by unit tests, because they need a controlled mix of meter types in one window. A platform-only window must not refuse; a priced platform event must not mask an unpriced AI window.
The smoke assertion covers the opposite regression — a verdict narrowed so far it never fires at all. Every price the smoke stack writes is dated 2099, so no
llm_tokensevent there is ever priced and the project's window is a genuine AI blackout thatblockmust still refuse. That is the one thing this environment can prove about the change, and it now asserts the409rather than a success.Worth recording, since it is the kind of thing that reads as a fix regression later: my first version of that smoke step asserted the generation would succeed, and CI rejected it. The premise was wrong, not the refusal — I had assumed AI usage in the smoke stack was priced. The failure output confirms the project accumulates well past the three-event blackout threshold, which is also what makes the corrected assertion stable.
Verification
Red/green, in that order.
reason: "unpriced_usage"breach wherenullis expected. Then they pass.pnpm testinpackages/server: 239 suites, 6709 tests, all passing against a local Postgres 16 with pgvector.pnpm typecheck: passes.eslinton the changed files: clean.pnpm docs-lint: OK. SDK and CLI regenerated: no output change.bash -non the smoke script: clean. Smoke and tutorials need a live stack, so CI is the real run — the first one caught the bad assertion above.Context
This surfaced as five failures in naturali's live e2e on its first run against
v0.37.0, all of them this one cause. Worth noting for anyone bisecting:v0.37.0did not introduce the bug. The quota enforcement file is byte-identical betweenv0.36.0andv0.37.0, and compute metering is older still. The release changed how many node executions get recorded, which is what pushed an existing latent condition past the three-event blackout threshold.🤖 Generated with Claude Code
https://claude.ai/code/session_01DRwsGecUbNX8PcAzZ5L2ov