spike: capture the wire to the model API and diff it against the JSONL - #32
Open
Mark-Life wants to merge 1 commit into
Open
spike: capture the wire to the model API and diff it against the JSONL#32Mark-Life wants to merge 1 commit into
Mark-Life wants to merge 1 commit into
Conversation
Timeboxed spike on whether a local intercepting proxy shows enough beyond the JSONL transcript to be worth building. Adds scripts/wire-capture.ts (loopback capture proxy + decomposition summarizer) and .docs/spike/wire-capture.md with the measured findings. Two Claude Code sessions and one Codex session were captured against the real binaries. The headline: 77-94% of a real context window is tool-definition JSON that appears nowhere on disk, and the base-URL override makes capture possible without a local CA. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem:
Peektrace tells a user that 95% of their context window is "System + tool definitions — inferred floor, not in transcript" and stops there. The JSONL the client writes never contains tool schemas or the system prompt, so
computeSystemOverheadinpackages/core/src/services/sessions/analyze.tscan only subtract visible events from the first turn'susageand render the remainder as one opaque bar. The user learns the number and nothing they can act on.This spike asked whether an intercepting proxy would recover the missing detail, and whether the TLS cost of getting it is acceptable.
Solution:
Adds
scripts/wire-capture.ts— a loopback-bound capture proxy plus a decomposition summarizer — and.docs/spike/wire-capture.mdwith the measured result.Two findings decide the idea. First,
ANTHROPIC_BASE_URLand Codex'smodel_providers.<id>.base_urlboth accept a plain-HTTP127.0.0.1target, so capture needs no local CA and touches no trust store; the TLS problem that made this look expensive does not exist on that route. Second, on a real first turn with six claude.ai MCP connectors loaded, 142,546 of 152,017 context tokens were tool-definition JSON — 93.8%, none of it on disk. The wire decomposition reproduces the analyzer's opaque floor to within 1 token on both captured sessions, so the floor is exact but undivided, and the wire supplies the division.Nothing ships. The write-up recommends a narrow build (derive per-tool costs, discard the body, join by session id) and names the privacy gap found on the way:
redactTexthandles wire bodies including the OAuth bearer, but missesmetadata.user_id, which carries a device and account identifier that is not secret-shaped.Security Impact:
The harness terminates TLS for the agent, so it necessarily sees the bearer token. It binds
127.0.0.1only, dropsauthorization/cookie/x-api-key/proxy-authorizationbefore serialization, and passes every body through core's existingredactText. Verified against a real capture: nosk-ant-string survives in the output.--no-redactexists for debugging the redactor..wire-capture/is gitignored so raw bodies cannot be committed by accident.Testing:
Ran the committed harness against a real
claude -psession, then summarized the capture:Exact token counts in the write-up come from
/v1/messages/count_tokensdifferencing, not the chars/4 estimate shown above.bun run typecheckpasses (9/9) andultracite checkis clean. No product code is touched, so no existing behaviour changes.