Shape tool output at the boundary, the way pi does, instead of only after the fact

[mattzcarey]

Summary

Everything we do about context happens after a tool has already returned. A tool reads a 3.5 MB file, the whole thing lands in the transcript, and later passes truncate it, evict it, or summarize it away. Pi does the opposite: it caps at the tool boundary, before anything is stored, and hands the model a way to ask for the rest.

We should do the same in agents/context, because the tools that produce oversized output are mostly written by users of this SDK, and a guardrail every tool author has to implement themselves is one almost nobody has.

Where we are against pi

	pi	Think today
text read cap	2,000 lines or 50 KB, whichever hits first	2,000 lines, 3.5 MB
over-cap behavior	truncates and tells the model offset=N to continue	truncates the line
images	optional imageProcessor, autoResizeImages on by default	nothing

Two differences matter more than the numbers.

Pi's cap is about seventy times smaller. DEFAULT_MAX_BYTES is 50 KB (pi-agent-core/harness/utils/truncate.ts); Think's MAX_MODEL_FILE_BYTES is 3.5 MB (packages/think/src/tools/workspace.ts).

And pi's truncation carries a continuation affordance: [Showing lines 1-40 of 900 (50KB limit). Use offset=41 to continue.]. That is what makes an aggressive cap safe — the model is never stuck, it just fetches the next window if it actually needs it. Our truncation is a dead end, which is why our cap has to be generous, which is why large files reach the model.

Think also has no image path. Pi resizes before the bytes enter the request; a full-resolution screenshot from one of our tools enters context at whatever size the tool produced.

Proposal

Put intake shaping in agents/context as a wrapper applied to tool results on the way in:

1. A byte budget for any tool result, defaulting to pi's numbers.
2. Truncation that always emits a continuation hint naming the exact way to get the next window.
3. An image step that downscales before the bytes become part of the request, with a hook so a host can supply its own encoder.

Hosts get it by installing the module rather than by each tool author remembering. A tool that genuinely needs to return something huge opts out explicitly.

Open questions

• Does shaping run before or after the result is persisted? Before means the transcript never holds what the model never saw, which is smaller and simpler. After means a later policy change can still recover the full output. Storage is lossless either way, so this is a context decision, not a storage one.
• How does this interact with prompt caching? A cap that trims at a sliding boundary rewrites the prompt prefix every turn. See Investigate which context-reduction mechanisms break prompt caching #2200 before choosing where the boundary anchors.
• Do we cap by bytes, by tokens, or both? Bytes are cheap and deterministic; tokens are what actually costs. Pi uses bytes.

Context

design/context.md records the module's boundaries: prompt assembly and history shaping belong in agents/context, retention (Think's media eviction to its Workspace) belongs with the host that owns the file store. Intake shaping is the third piece and the only one that does not exist yet.

Why this also gates content-addressed attachment storage

A separate attachment table with pointers would give deduplication: identical bytes stored once, referenced from many messages. We removed exactly that from agents/sessions, and it is worth being precise about why, because the version that would be worth building depends on this issue.

The store we removed extracted payloads above a size threshold. That is neither uniform nor principled: it split messages by content type and size rather than by what they are, so most attachments were never in the store at all, which is what made dedup opportunistic rather than structural.

A version worth having would extract every attachment, uniformly, with no threshold. Then pointers are the normal representation, dedup is real, and lifetime is one rule.

That requires being able to parse attachments out of a message reliably, and today we cannot. packages/agents/src/tests/harness/pdf-in-session.test.ts proves a PDF read through pi's tools reaches the transcript as plain tool-output text with no media type, while a GIF becomes a typed file part, purely because pi sniffs five image signatures and decodes everything else as text. So "the set of attachments in this message" is not currently computable. Any store built on it would silently cover images and miss documents.

Intake shaping is the prerequisite. If tool results carry typed content on the way in — the media type known and recorded at the boundary, the way pi does for images — then attachments become an identifiable category, and a uniform content-addressed store becomes coherent to build on top.

Order: shape intake first, and only then revisit attachment storage, with the threshold removed and the extraction applied to everything.