Skip to content

ChatBedrock() mantle APIs: default model isn't served, and cross-region prefix isn't stripped from requests#411

Description

@thisisnic

When porting #345 to ellmer in tidyverse/ellmer#1101, I found an edge case; 馃 summary below:

Two related problems with model ids on the messages and responses APIs.

ChatBedrock(api="messages") with no model uses DEFAULT_MODEL (us.anthropic.claude-sonnet-4-6), but mantle's Anthropic endpoint doesn't serve Sonnet 4.6, with or without the prefix. list_models() on that endpoint currently returns only claude-fable-5, claude-haiku-4-5, claude-opus-4-7, claude-opus-4-8, claude-opus-5, and claude-sonnet-5.

import chatlas as ctl

ctl.ChatBedrock(api="messages").chat("Say hi")
#> 404: The model 'us.anthropic.claude-sonnet-4-6' does not exist

Separately, CROSS_REGION_PREFIX is only stripped for the MODEL_APIS lookup, not from the model id in the request. Converse needs the prefix but mantle rejects it, so a prefixed id that routes to mantle 404s:

ctl.ChatBedrock(model="us.openai.gpt-5.4").chat("Say hi")
#> 404: The model 'us.openai.gpt-5.4' does not exist

ellmer handles both by defaulting per API and stripping the prefix from the request when the API is messages or responses (aws_strip_region_prefix() in R/provider-aws.R).

Tested against chatlas main (0.22.1.dev5).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Priority: MediumValid bug or well-defined request with moderate impact or a workaround.ai-triage:doneMarks an issue whose AI triage workflow is complete.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions