Skip to content

feat(amazon-bedrock): forward Anthropic beta flags via additionalMode…#4

Open
adhajar-amzn wants to merge 1 commit into
mainfrom
feat/bedrock-anthropic-betas
Open

feat(amazon-bedrock): forward Anthropic beta flags via additionalMode…#4
adhajar-amzn wants to merge 1 commit into
mainfrom
feat/bedrock-anthropic-betas

Conversation

@adhajar-amzn
Copy link
Copy Markdown

…lRequestFields

Bedrock Converse ignores the anthropic-beta HTTP header that the direct Anthropic API honors. The Bedrock-correct channel for opting into Anthropic experimental features is the top-level
additionalModelRequestFields.anthropic_beta array on the ConverseStream request, which Bedrock forwards to the underlying Anthropic model.

This exposes a new anthropicBetas.betas plugin config that, when set, wraps the stream function for Anthropic-family Bedrock models and merges the configured beta flags into the outbound request body. The wrapper is a no-op for non-Anthropic families (Nova, Titan, Llama, etc.), which would reject the unknown field.

Motivation: enabling fine-grained-tool-streaming-2025-05-14 so that large tool_use input JSON (e.g. multi-KB heredoc scripts) streams incrementally via input_json_delta events instead of being buffered until the tool_use block is complete. Users observed 60-80s of silence on turns where the model generates large tool_use arguments.

Implementation mirrors the existing createGuardrailWrapStreamFn pattern and composes cleanly with guardrail injection.

Example config:

{
"plugins": {
"entries": {
"amazon-bedrock": {
"config": {
"anthropicBetas": {
"betas": ["fine-grained-tool-streaming-2025-05-14"]
}
}
}
}
}
}

Summary

Describe the problem and fix in 2–5 bullets:

If this PR fixes a plugin beta-release blocker, title it fix(<plugin-id>): beta blocker - <summary> and link the matching Beta blocker: <plugin-name> - <summary> issue labeled beta-blocker. Contributors cannot label PRs, so the title is the PR-side signal for maintainers and automation.

  • Problem:
  • Why it matters:
  • What changed:
  • What did NOT change (scope boundary):

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

  • Closes #
  • Related #
  • This PR fixes a bug or regression

Root Cause (if applicable)

For bug fixes or regressions, explain why this happened, not just what changed. Otherwise write N/A. If the cause is unclear, write Unknown.

  • Root cause:
  • Missing detection / guardrail:
  • Contributing context (if known):

Regression Test Plan (if applicable)

For bug fixes or regressions, name the smallest reliable test coverage that should catch this. Otherwise write N/A.

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file:
  • Scenario the test should lock in:
  • Why this is the smallest reliable guardrail:
  • Existing test that already covers this (if any):
  • If no new test is added, why not:

User-visible / Behavior Changes

List user-visible changes (including defaults/config).
If none, write None.

Diagram (if applicable)

For UI changes or non-trivial logic flows, include a small ASCII diagram reviewers can scan quickly. Otherwise write N/A.

Before:
[user action] -> [old state]

After:
[user action] -> [new state] -> [result]

Security Impact (required)

  • New permissions/capabilities? (Yes/No)
  • Secrets/tokens handling changed? (Yes/No)
  • New/changed network calls? (Yes/No)
  • Command/tool execution surface changed? (Yes/No)
  • Data access scope changed? (Yes/No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS:
  • Runtime/container:
  • Model/provider:
  • Integration/channel (if any):
  • Relevant config (redacted):

Steps

Expected

Actual

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios:
  • Edge cases checked:
  • What you did not verify:

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes/No)
  • Config/env changes? (Yes/No)
  • Migration needed? (Yes/No)
  • If yes, exact upgrade steps:

Risks and Mitigations

List only real risks for this PR. Add/remove entries as needed. If none, write None.

  • Risk:
    • Mitigation:

@adhajar-amzn adhajar-amzn force-pushed the feat/bedrock-anthropic-betas branch 2 times, most recently from fafce13 to 1c14748 Compare April 20, 2026 23:04
…odelRequestFields

Bedrock Converse ignores the `anthropic-beta` HTTP header that the direct
Anthropic API honors. The Bedrock-correct channel is the top-level
`additionalModelRequestFields.anthropic_beta` array on the ConverseStream
request, which Bedrock forwards to the underlying Anthropic model.

Mirror `PI_AI_DEFAULT_ANTHROPIC_BETAS` from extensions/anthropic/stream-wrappers.ts
so Bedrock-hosted Claude behaves the same as the direct Anthropic API:

  - fine-grained-tool-streaming-2025-05-14
  - interleaved-thinking-2025-05-14

Motivation: `fine-grained-tool-streaming` makes large tool_use input JSON
(e.g. multi-KB heredoc scripts) stream incrementally via input_json_delta
events instead of being buffered until the tool_use block is complete.
Users observed 60-80s of silence on turns where the model generates large
tool_use arguments on Bedrock — a gap that does not exist on the direct
Anthropic path because the anthropic extension already sets these betas.

Surgical change: swap the Anthropic branch of `baseWrapStreamFn` from a
no-op passthrough to a wrapper that injects the betas. Non-Anthropic
families are unchanged.
@adhajar-amzn adhajar-amzn force-pushed the feat/bedrock-anthropic-betas branch from 1c14748 to 65f2542 Compare April 20, 2026 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant