fix(bedrock): serve mantle default model and strip region prefix - #419
Open
cpsievert wants to merge 2 commits into
Open
fix(bedrock): serve mantle default model and strip region prefix#419cpsievert wants to merge 2 commits into
cpsievert wants to merge 2 commits into
Conversation
- Bump ChatBedrock()'s default model from us.anthropic.claude-sonnet-4-6 to us.anthropic.claude-sonnet-5, which mantle's api="messages" endpoint actually serves (sonnet-4-6 404s there). - Strip the cross-region inference prefix (us., eu., etc.) from the model id used in requests to api="messages" and api="responses". Converse needs the prefix to pick an inference profile, but mantle rejects it, so a prefixed model id that routes to mantle (e.g. model="us.openai.gpt-5.4") previously 404d. Fixes #411
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #411.
Two related bugs in
ChatBedrock()'s mantle APIs (api="messages"/api="responses"):Default model 404s on mantle.
DEFAULT_MODELwas"us.anthropic.claude-sonnet-4-6", which mantle's Anthropic endpoint does not serve. Bumped it to"us.anthropic.claude-sonnet-5", matching ellmer's default, which mantle does serve (and which Converse continues to serve too, so the default routing to"converse"is unaffected).Cross-region prefix leaks into mantle requests.
CROSS_REGION_PREFIXwas only stripped when looking up a model in theMODEL_APISrouting table, not from the model id actually sent in the request. Converse needs the prefix (it picks the inference profile), but mantle rejects it, so a prefixed model id that routes to mantle (e.g.model="us.openai.gpt-5.4") 404'd. Addedbedrock_strip_region_prefix()and applied it to the model passed intoBedrockResponsesProviderandBedrockMessagesProvider(the Converse path keeps the prefix, since it needs it).Testing
tests/test_provider_bedrock_mantle.pycovering: the default model works whenapi="messages"is requested explicitly, and the cross-region prefix is stripped for bothapi="messages"andapi="responses".tests/test_provider_bedrock_converse.py's default-model assertion to match the new default.uv run pytest,uv run ruff check chatlas,uv run ruff format --check,uv run pyrighton the touched files).Note: my local sandbox had stale/expired ambient
AWS_*env vars left over from a prior session, which makes AWS-credential-dependent tests fail regardless of this change (confirmed same failures onmain). Unsetting them locally, the whole suite (1256 tests) passes.