Skip to content

spike: capture the wire to the model API and diff it against the JSONL - #32

Open
Mark-Life wants to merge 1 commit into
mainfrom
atm/task-019fe01b-8a25-79b4-91aa-5d024df48d01
Open

spike: capture the wire to the model API and diff it against the JSONL#32
Mark-Life wants to merge 1 commit into
mainfrom
atm/task-019fe01b-8a25-79b4-91aa-5d024df48d01

Conversation

@Mark-Life

Copy link
Copy Markdown
Owner

Problem:

Peektrace tells a user that 95% of their context window is "System + tool definitions — inferred floor, not in transcript" and stops there. The JSONL the client writes never contains tool schemas or the system prompt, so computeSystemOverhead in packages/core/src/services/sessions/analyze.ts can only subtract visible events from the first turn's usage and render the remainder as one opaque bar. The user learns the number and nothing they can act on.

This spike asked whether an intercepting proxy would recover the missing detail, and whether the TLS cost of getting it is acceptable.

Solution:

Adds scripts/wire-capture.ts — a loopback-bound capture proxy plus a decomposition summarizer — and .docs/spike/wire-capture.md with the measured result.

Two findings decide the idea. First, ANTHROPIC_BASE_URL and Codex's model_providers.<id>.base_url both accept a plain-HTTP 127.0.0.1 target, so capture needs no local CA and touches no trust store; the TLS problem that made this look expensive does not exist on that route. Second, on a real first turn with six claude.ai MCP connectors loaded, 142,546 of 152,017 context tokens were tool-definition JSON — 93.8%, none of it on disk. The wire decomposition reproduces the analyzer's opaque floor to within 1 token on both captured sessions, so the floor is exact but undivided, and the wire supplies the division.

Nothing ships. The write-up recommends a narrow build (derive per-tool costs, discard the body, join by session id) and names the privacy gap found on the way: redactText handles wire bodies including the OAuth bearer, but misses metadata.user_id, which carries a device and account identifier that is not secret-shaped.


Security Impact:

The harness terminates TLS for the agent, so it necessarily sees the bearer token. It binds 127.0.0.1 only, drops authorization / cookie / x-api-key / proxy-authorization before serialization, and passes every body through core's existing redactText. Verified against a real capture: no sk-ant- string survives in the output. --no-redact exists for debugging the redactor. .wire-capture/ is gitignored so raw bodies cannot be committed by accident.


Testing:

Ran the committed harness against a real claude -p session, then summarized the capture:

wire-capture on http://127.0.0.1:8891 -> https://api.anthropic.com
captures: /home/agent/spike/wc/  redaction: on
[003] POST /v1/messages 385258B

=== 003_v1_messages.json  model=claude-sonnet-4-5-20250929  96314 est tok total
  system[2]      6969 est tok [cache 1h]
  144 tool definitions   86589 est tok
    mcp__claude_ai_Canva__edit-design    5534 est tok
    Workflow                             5333 est tok

Exact token counts in the write-up come from /v1/messages/count_tokens differencing, not the chars/4 estimate shown above. bun run typecheck passes (9/9) and ultracite check is clean. No product code is touched, so no existing behaviour changes.

Timeboxed spike on whether a local intercepting proxy shows enough beyond the
JSONL transcript to be worth building.

Adds scripts/wire-capture.ts (loopback capture proxy + decomposition
summarizer) and .docs/spike/wire-capture.md with the measured findings.

Two Claude Code sessions and one Codex session were captured against the real
binaries. The headline: 77-94% of a real context window is tool-definition JSON
that appears nowhere on disk, and the base-URL override makes capture possible
without a local CA.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant