Skip to content

fix(translation): preserve final answers in buffered Responses - #284

Merged
ayushag-nv merged 1 commit into
mainfrom
fix/responses-buffered-output
Aug 4, 2026
Merged

fix(translation): preserve final answers in buffered Responses#284
ayushag-nv merged 1 commit into
mainfrom
fix/responses-buffered-output

Conversation

@nachiketb-nvidia

@nachiketb-nvidia nachiketb-nvidia commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

What

Normalize buffered OpenAI Responses output items into one semantic assistant output.

Why

first_output() means the first semantic ResponseOutput, not the first content block or modality. One ResponseOutput can contain ordered reasoning, text, tool calls, images, and other normalized content blocks. Chat, Anthropic, and accumulated streams already represent one completed assistant response this way.

The Responses wire format can instead split one logical completion across multiple top-level output[] items, such as a reasoning item followed by a message containing the visible final answer. The buffered decoder incorrectly mapped each wire item to a separate ResponseOutput. Anthropic and Chat then encoded the first semantic output, preserving its reasoning but never reaching the separately stored final message. Requests therefore returned HTTP 200 while silently dropping the visible answer.

This change normalizes the supported Responses output item types: reasoning, message, and function call. It does not broaden multimodal output support; audio, video, and unknown top-level Responses output items remain separate translation coverage gaps.

This fixes SWITCH-1187.

How

  • Fold decoded Responses reasoning, message, and function-call items into ordered content blocks in one ResponseOutput.
  • Retain the last meaningful role and stop reason.
  • Preserve the existing empty-response fallback.
  • Keep the Anthropic, Chat, and streaming production codecs unchanged.
  • Verify buffered normalization matches the equivalent accumulated stream.

What to review

  • Responses output-item ordering during normalization.
  • Stop-reason selection across reasoning, message, and tool-call items.
  • Buffered/stream semantic parity.

Live validation

Built the release server from this branch and ran the issue reproducer with live credentials:

  • Direct Responses control: HTTP 200, reasoning,message, exact final-answer marker present.
  • Chat through Switchyard: HTTP 200, reasoning non-empty, exact final-answer marker present.
  • Anthropic through Switchyard: three consecutive HTTP 200 responses containing thinking,text, exact final-answer marker present.

One earlier Anthropic-format generation returned reasoning only. Since each call is a separate stochastic generation, it did not establish translation loss; every retry that produced final text preserved it.

Validation

  • cargo fmt --all --check
  • cargo test -p switchyard-translation
  • cargo clippy -p switchyard-translation --all-targets -- -D warnings

Signed-off-by: nachiketb <nachiketb@nvidia.com>
@nachiketb-nvidia
nachiketb-nvidia requested a review from a team as a code owner August 4, 2026 18:04
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

decode_response now combines multiple OpenAI Responses output items into one ResponseOutput. Tests verify reasoning and assistant content across Anthropic Messages, OpenAI Chat, buffered decoding, and streamed events.

Changes

Responses output handling

Layer / File(s) Summary
Buffered output aggregation and translation coverage
crates/switchyard-translation/src/codecs/responses/buffered.rs, crates/switchyard-translation/tests/response_translation.rs
decode_response aggregates reasoning and assistant output. Translation tests verify separate Anthropic thinking and text blocks, plus OpenAI Chat reasoning and visible answer content.
Buffered and streamed output parity
crates/switchyard-translation/tests/stream_translation.rs
A regression test verifies equivalent normalized output from buffered Responses decoding and streamed event accumulation.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

A rabbit watched the responses flow,
With thoughts kept private, answers shown.
One output gathered, neat and bright,
Stream and buffer now match just right.
“Hop!” said the rabbit, “tests take flight.”

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the main change: preserving final answers in buffered OpenAI Responses translation.

Comment @coderabbitai help to get the list of available commands.

@ayushag-nv
ayushag-nv merged commit 0e99eb4 into main Aug 4, 2026
19 checks passed
@ayushag-nv
ayushag-nv deleted the fix/responses-buffered-output branch August 4, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants