[Feature] Separate the user's question from the context Claude Code injects around it

[xinyu-dev]

Problem

When Switchyard runs behind Claude Code, the router often can't tell which
question it is supposed to route.

Claude Code adds extra text to every message a user sends: its own reminders,
and — if a memory plugin such as Hindsight is installed — recalled notes plus
snippets of earlier conversations. Switchyard merges all of that with the real
question and hands the classifier one undivided block. The recalled snippets
quote past questions word for word, so they look exactly like the current one.

Why it matters

In one routing session the classifier averaged 41,225 tokens per call, peaking
at 51,636. Most of that was recalled memory rather than conversation. The real
question is a line or two buried in the middle, and nothing marks it as the one
that matters.

Proposed behavior

Give the injected text its own section, and leave the user's question by itself
at the end:

Route the request in <query>, the last section below. <memory> and <history>
are background: never route on them alone.

<memory>…recalled notes and reminders…</memory>
<history>…recent turns…</history>
<query>…what the user actually typed…</query>

If it all has to be trimmed to fit, cut the memory first and never the question.

Notes

Working locally by splitting on the tags the client already emits
(<system-reminder>, <hindsight_*>) inside
switchyard/lib/processors/llm_classifier/request_processor.py. Happy to open a
PR. Related: #246.

[xinyu-dev]

The same root cause shows up a second time, in the agent path.

When a request routes to an agent target instead of a model, the demo adapter
(examples/gym_biology_agent/agent_adapter.py) forwards the client's whole
message list. The agent then receives Claude Code's entire system prompt —
CLAUDE.md, AGENTS.md, the MCP server instructions, the skills list, git status —
plus the hook-injected blocks as extra [USER] turns. Trimmed, this is what the
BioMni agent was actually asked:

[SYSTEM]     You are Claude Code... # MCP Server Instructions ... (thousands of tokens)
[USER]       <system-reminder>…</system-reminder> Write a Python merge-sort function.
[USER]       Available agent types for the Agent tool: - claude: ... - Explore: ...
[ASSISTANT]  def merge_sort(items): ...
[USER]       Explain the mechanism of mRNA splicing
[ASSISTANT]  **mRNA splicing** removes introns...
[USER]       Note: MEMORY.md was modified by a linter.
[USER]       <hindsight_recall_query_rewrite>…</hindsight_recall_query_rewrite>
             Compute BSA protein's protparam parameters: MW, pI, extinction coefficient.

The biology question is the last line of all that.

One detail for whoever implements this: the hook emits three tags, not one.
<hindsight_memories> holds recalled facts, <hindsight_recall_query_rewrite>
is a diagnostic, and <hindsight_chunks> carries raw transcript excerpts that
quote earlier questions word for word. The chunks are the most misleading of the
three, so any filter needs to cover all of them.

What works locally: drop the client's system prompt (the adapter already sends
its own instructions), drop messages that are purely harness scaffolding, and
gather the recalled blocks into a single trailing turn on the developer role
so they read as background rather than as another question.

[xinyu-dev]

Refined the shape after running it against a live session. Two behaviours worth
knowing before anyone implements this.

Recall piles up across turns. Each turn carries the recall from the moment it
was sent, so forwarding all of them hands the agent one stale rewrite per past
question. In a three-question session the agent received rewrites for "write a
merge sort" and "explain mRNA splicing" sitting directly in front of the protein
question it was actually meant to answer. Only the recall attached to the newest
user turn is relevant.

Reminders repeat verbatim. Claude Code re-sends the same <system-reminder>
(the CLAUDE.md and AGENTS.md contents) on every turn, so keeping them all pastes
the project guidelines once per turn.

The shape that reads cleanly leaves the conversation untouched and puts one
labelled block at the end:

[USER]       Write a Python merge-sort function
[ASSISTANT]  def merge_sort(items): ...
[USER]       Explain the mechanism of mRNA splicing
[ASSISTANT]  **mRNA splicing** removes introns...
[USER]       Compute BSA protein's protparam parameters

[DEVELOPER]
CLIENT CONTEXT (from the coding agent, not part of the request):
<system-reminder>…</system-reminder>              <- deduped across turns

MEMORY RECALLED FROM THE LAST USER QUERY:
<hindsight_recall_query_rewrite>…</…>             <- newest turn only
<hindsight_memories>…</…>

Worth keeping this context rather than discarding it: the reminders carry the
user's own project instructions, and the recalled memory is often genuinely
useful. It only needs a label so it cannot be read as the question.