fix(cost): write ledger records with json.dumps so quoted model ids aren't dropped#166
Conversation
|
Note on CI: the |
21e5dd4 to
d5e3373
Compare
_ledger_append hand-built each ledger line with an f-string, so a model id containing a double quote, backslash, or control character produced an invalid JSON line. ledger_cost_report catches json.JSONDecodeError and skips such lines, so those calls were silently dropped from the cost total (under-reported API spend) and the JSONL file was corrupted for any other reader. Serialize the record with json.dumps so the writer and reader agree on escaping. Added an offline round-trip test covering a model id with a quote and a backslash. Closes mini-router#165 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Updated: rebased onto current The earlier red here was not from the cost-ledger change — it was the pre-existing |
After the submission size check became warning-only, a wrong theta length no longer produces a hard error, so test_wrong_theta_length_fails could never pass: it asserted result.ok is False and relied on make_spec().n_total differing from a hardcoded 128 (that n_total is 128 in the CI environment). main's CI is currently red on this test for every PR branched off it (mini-router#160). Rename to test_wrong_theta_length_warns and assert the current behavior: a theta length that disagrees with the summary's declared n_total is surfaced as a non-fatal warning (ok stays True), independent of the canonical n_total.
Problem
_ledger_append(src/trinity/llm/openai_compatible_pool.py) builds each cost-ledger line by hand:modelisroute.model_id, read verbatim fromconfigs/models*.yaml. A model id containing a double quote ("), backslash (\), or control character makes the line invalid JSON. The reader,ledger_cost_report(src/trinity/costing.py), wrapsjson.loadsintry/except json.JSONDecodeError: continue, so those lines are silently skipped.Impact
Silent under-reporting of API spend: every call for such a model disappears from
cost_usd,cost_calls, and the per-model breakdown, with no error — and the JSONL file is corrupted for any other consumer. The cost ledger exists to prevent exactly this kind of drift.Fix
Serialize the record with
json.dumpsso the writer and reader agree on escaping:The line format (keys
provider/m/p/c, one JSON object per line) is unchanged for well-behaved ids, so existing ledgers keep parsing.Tests
New offline
tests/test_cost_ledger.pyround-trips_ledger_append→ledger_cost_report, including a model id containing a quote and a backslash. Verified the metacharacter test fails on the old f-string (cost_calls == 0, call dropped) and passes with the fix. Full router suite green locally (only the pre-existingtest_run_remote/test_setup_remotepython3-on-PATH failures remain, unrelated).Risk / tradeoffs
Minimal. Same on-disk format for normal ids; only adds correct escaping. No API or config changes.
Closes #165