Cherry-pick and validate upstream Neroued#10 (recover drifted Qwen tool calls) and reconcile our tool-message content-parts change with upstream Neroued#57.
Why this is a true bottleneck
Agentic clients are the primary consumers of this deployment, and tool-calling is where small-model output drifts under load (malformed <tool_call> payloads, near-miss function tags). Today a drifted call surfaces as a plain-text turn and the agent loop stalls — the worst failure mode for an autonomous client. #10 recovers those.
Reconciliation note (Neroued#57 vs ours)
Our branch already ships an equivalent of Neroued#57: serve/openai_schema.cpp accepts structured content parts in tool role messages — but ours also accepts image/video parts (agentic clients return screenshots inside tool results; the chat template already renders tool turns through the ordinary media-placeholder path). Upstream Neroued#57 is text-only. Ours rides feat/prefix-seed-store and was submitted upstream as part of Neroued#90. Task: keep one implementation, prefer the broader one, and follow whichever upstream merges first to avoid a divergent schema.
Validation plan
- Fixture set of drifted Qwen tool calls (truncated closing tags, stray text before
<tool_call>, duplicated parameter blocks) -> recovered tool_calls with correct finish_reason.
- Round-trip: tool result containing a screenshot image part -> 200, media bound, model answers about the image content.
- Multi-tool discrimination and parallel-call loops remain untested engine-wide — extend coverage here if time allows (this is the largest known untested surface for agentic serving).
Cherry-pick and validate upstream Neroued#10 (
recover drifted Qwen tool calls) and reconcile our tool-message content-parts change with upstream Neroued#57.Why this is a true bottleneck
Agentic clients are the primary consumers of this deployment, and tool-calling is where small-model output drifts under load (malformed
<tool_call>payloads, near-miss function tags). Today a drifted call surfaces as a plain-text turn and the agent loop stalls — the worst failure mode for an autonomous client. #10 recovers those.Reconciliation note (Neroued#57 vs ours)
Our branch already ships an equivalent of Neroued#57:
serve/openai_schema.cppaccepts structured content parts intoolrole messages — but ours also accepts image/video parts (agentic clients return screenshots inside tool results; the chat template already renders tool turns through the ordinary media-placeholder path). Upstream Neroued#57 is text-only. Ours ridesfeat/prefix-seed-storeand was submitted upstream as part of Neroued#90. Task: keep one implementation, prefer the broader one, and follow whichever upstream merges first to avoid a divergent schema.Validation plan
<tool_call>, duplicated parameter blocks) -> recoveredtool_callswith correctfinish_reason.