fix(frontend): keep literal vision tokens out of media binding - #88
Open
geoffwatts wants to merge 1 commit into
Open
fix(frontend): keep literal vision tokens out of media binding#88geoffwatts wants to merge 1 commit into
geoffwatts wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5eb3176a23
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Neutralize exact Qwen Vision control-token spellings in textual chat inputs so only structured image and video parts create media placeholders. Preserve Anthropic tool_result and nested content order, and cover reversed parallel results with mixed and repeated media.
geoffwatts
force-pushed
the
fix/anthropic-tool-result-media
branch
from
August 24, 2026 13:38
5eb3176 to
1c4bfa5
Compare
Author
|
The renderer regression now splits all four Vision control tokens across text-part boundaries while structured media is present. Verification:
Thanks for catching this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A valid Anthropic Messages request containing parallel
tool_usehistory andimage-bearing
tool_resultblocks returned:The captured request returned tool results in completion order rather than the
preceding invocation order. It also contained literal Qwen Vision control-token
spellings inside system text that quoted a chat template.
Root cause
The Anthropic schema path already preserved both
tool_resultorder and nestedcontent order. Media collection followed that same order.
The mismatch occurred later in the Qwen frontend. Media binding searched the
complete rendered prompt for raw
<|image_pad|>and<|video_pad|>strings.Literal occurrences in system text, reasoning, tool definitions, or tool
arguments were therefore mistaken for structured media placeholders. This
shifted binding away from the actual image blocks and produced the ordering
error.
Design
The compiled chat renderer now neutralizes exact Qwen Vision control-token
spellings in textual input by inserting U+2060 WORD JOINER after
<|.This keeps the quoted text readable while preventing it from becoming an exact
added token or media-binding marker. The rule is applied to:
Structured image and video parts continue to emit the original raw control
tokens and remain the sole source of media placeholders.
Tool results are not sorted or remapped to the original
tool_useorder.Message, block, and nested-content traversal remain in client-supplied order.
Affected behavior
tool_resultblocks may arrive in a different order from thepreceding parallel
tool_useblocks.nested order.
The serving documentation now states these ordering and literal-token
semantics.
Verification
Clean V100-port checkout, Tesla V100-SXM2-32GB, GCC 13.3, CUDA 12.8.61: