Skip to content

feat(proxy): clamp max_tokens, fix Responses content shape, surface reasoning#38

Open
GoDiao wants to merge 1 commit into
clean-masterfrom
feat/proxy-max-tokens-reasoning
Open

feat(proxy): clamp max_tokens, fix Responses content shape, surface reasoning#38
GoDiao wants to merge 1 commit into
clean-masterfrom
feat/proxy-max-tokens-reasoning

Conversation

@GoDiao

@GoDiao GoDiao commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Summary

OpenAI-compatible proxy quality fixes for the Anthropic↔OpenAI translation layer.

Changes

  • maxOutputTokens provider config + clamp: Add optional maxOutputTokens to SavedProviderSchema/CreateProviderSchema/UpdateProviderSchema (int, positive, ≤10,000,000). When configured, Anthropic max_tokens is clamped via Math.min before forwarding to OpenAI endpoints. Without a cap, the field is still omitted. Fixes Claude Code sending 128K max_tokens that exceeds provider limits (DeepSeek: 8192, etc.).
  • OpenAI Responses content shape: Emit input_text/input_image content parts (the correct Responses shape) instead of reusing Chat-shaped image_url objects. Corrects the flush logic so accumulated content is not dropped when a tool block follows text/image.
  • Reasoning streaming (Responses): Handle response.reasoning_summary_text.delta / response.reasoning_text.delta / *.done / reasoning_summary_part.done and the reasoning output item, mapping them to Anthropic thinking content blocks. Includes lazy open for providers that omit the output_item.added event. All reasoning blocks are closed before message_delta per Anthropic spec.
  • Cache token mapping: Forward prompt_tokens_details.cached_tokenscache_read_input_tokens on both Chat and Responses streaming paths (previously only output_tokens was reported).
  • Shared extractToolResultContent: Dedupe the tool_result content extraction in both transform paths; non-text blocks (images, nested tool_use, thinking) now produce placeholders like [image omitted: …] instead of being silently dropped.

Files

  • src/server/types/provider.ts — add maxOutputTokens to schemas
  • src/server/services/providerService.ts — thread maxOutputTokens through getProviderForProxy
  • src/server/proxy/handler.ts — pass maxOutputTokens to both transforms
  • src/server/proxy/transform/anthropicToOpenaiChat.ts — clamp max_tokens, use shared helper
  • src/server/proxy/transform/anthropicToOpenaiResponses.ts — clamp max_output_tokens, correct content part shape, use shared helper
  • src/server/proxy/transform/toolArguments.ts — new extractToolResultContent
  • src/server/proxy/transform/types.ts — add OpenAIResponsesContentPart
  • src/server/proxy/streaming/openaiChatStreamToAnthropic.tsmapStreamUsage with cached tokens
  • src/server/proxy/streaming/openaiResponsesStreamToAnthropic.ts — reasoning block handling + cached tokens
  • desktop/src-tauri/Cargo.lock — follow dreamcoder-desktop 0.4.0→0.4.5 version bump
  • src/server/__tests__/proxy-streaming.test.ts / proxy-transform.test.ts — coverage for the above

Validation

bun test src/server/__tests__/proxy-streaming.test.ts src/server/__tests__/proxy-transform.test.ts
# 98 pass, 0 fail, 300 expect() calls
git diff --check

Note: providers.test.ts / provider-presets.test.ts have 4 pre-existing failures on clean-master (verified by running tests against the unmodified branch); this PR does not touch the preset data or activation env logic those tests cover.

…easoning

- Add provider maxOutputTokens config (Saved/Create/Update schemas);
  clamp Anthropic max_tokens to the provider's known limit instead of
  always omitting it. Claude Code sends 128K which exceeds many providers
  (DeepSeek: 8192, etc.); without a configured cap we still omit the field.
- Fix OpenAI Responses API content parts: emit input_text/input_image
  instead of reusing Chat-shaped image_url objects, and correct the
  flush logic so accumulated content is not dropped.
- Stream reasoning items from OpenAI Responses (response.reasoning_*.
  delta/done and reasoning_summary_part.done) into Anthropic thinking
  content blocks, with lazy open for providers that omit output_item.added.
- Map prompt_tokens_details.cached_tokens -> cache_read_input_tokens
  on both Chat and Responses streaming paths (previously only
  output_tokens were forwarded).
- Extract shared extractToolResultContent helper; non-text tool_result
  blocks (images, nested tool_use, thinking) now produce placeholders
  instead of being silently dropped.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant