feat(amazon-bedrock): forward Anthropic beta flags via additionalMode…#4
Open
adhajar-amzn wants to merge 1 commit into
Open
feat(amazon-bedrock): forward Anthropic beta flags via additionalMode…#4adhajar-amzn wants to merge 1 commit into
adhajar-amzn wants to merge 1 commit into
Conversation
fafce13 to
1c14748
Compare
…odelRequestFields Bedrock Converse ignores the `anthropic-beta` HTTP header that the direct Anthropic API honors. The Bedrock-correct channel is the top-level `additionalModelRequestFields.anthropic_beta` array on the ConverseStream request, which Bedrock forwards to the underlying Anthropic model. Mirror `PI_AI_DEFAULT_ANTHROPIC_BETAS` from extensions/anthropic/stream-wrappers.ts so Bedrock-hosted Claude behaves the same as the direct Anthropic API: - fine-grained-tool-streaming-2025-05-14 - interleaved-thinking-2025-05-14 Motivation: `fine-grained-tool-streaming` makes large tool_use input JSON (e.g. multi-KB heredoc scripts) stream incrementally via input_json_delta events instead of being buffered until the tool_use block is complete. Users observed 60-80s of silence on turns where the model generates large tool_use arguments on Bedrock — a gap that does not exist on the direct Anthropic path because the anthropic extension already sets these betas. Surgical change: swap the Anthropic branch of `baseWrapStreamFn` from a no-op passthrough to a wrapper that injects the betas. Non-Anthropic families are unchanged.
1c14748 to
65f2542
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…lRequestFields
Bedrock Converse ignores the
anthropic-betaHTTP header that the direct Anthropic API honors. The Bedrock-correct channel for opting into Anthropic experimental features is the top-leveladditionalModelRequestFields.anthropic_betaarray on the ConverseStream request, which Bedrock forwards to the underlying Anthropic model.This exposes a new
anthropicBetas.betasplugin config that, when set, wraps the stream function for Anthropic-family Bedrock models and merges the configured beta flags into the outbound request body. The wrapper is a no-op for non-Anthropic families (Nova, Titan, Llama, etc.), which would reject the unknown field.Motivation: enabling
fine-grained-tool-streaming-2025-05-14so that large tool_use input JSON (e.g. multi-KB heredoc scripts) streams incrementally viainput_json_deltaevents instead of being buffered until the tool_use block is complete. Users observed 60-80s of silence on turns where the model generates large tool_use arguments.Implementation mirrors the existing
createGuardrailWrapStreamFnpattern and composes cleanly with guardrail injection.Example config:
{
"plugins": {
"entries": {
"amazon-bedrock": {
"config": {
"anthropicBetas": {
"betas": ["fine-grained-tool-streaming-2025-05-14"]
}
}
}
}
}
}
Summary
Describe the problem and fix in 2–5 bullets:
If this PR fixes a plugin beta-release blocker, title it
fix(<plugin-id>): beta blocker - <summary>and link the matchingBeta blocker: <plugin-name> - <summary>issue labeledbeta-blocker. Contributors cannot label PRs, so the title is the PR-side signal for maintainers and automation.Change Type (select all)
Scope (select all touched areas)
Linked Issue/PR
Root Cause (if applicable)
For bug fixes or regressions, explain why this happened, not just what changed. Otherwise write
N/A. If the cause is unclear, writeUnknown.Regression Test Plan (if applicable)
For bug fixes or regressions, name the smallest reliable test coverage that should catch this. Otherwise write
N/A.User-visible / Behavior Changes
List user-visible changes (including defaults/config).
If none, write
None.Diagram (if applicable)
For UI changes or non-trivial logic flows, include a small ASCII diagram reviewers can scan quickly. Otherwise write
N/A.Security Impact (required)
Yes/No)Yes/No)Yes/No)Yes/No)Yes/No)Yes, explain risk + mitigation:Repro + Verification
Environment
Steps
Expected
Actual
Evidence
Attach at least one:
Human Verification (required)
What you personally verified (not just CI), and how:
Review Conversations
If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.
Compatibility / Migration
Yes/No)Yes/No)Yes/No)Risks and Mitigations
List only real risks for this PR. Add/remove entries as needed. If none, write
None.