Anthropic assertions 404 when ANTHROPIC_BASE_URL is set as a bare host (Claude Code / Cursor environments)

[slusset]

Incidental finding from integrating passmark as a journey-test lane — filing in case it saves others the diagnosis. Not a passmark bug per se (root cause is in @ai-sdk/anthropic), but passmark inherits it in a way that's especially confusing under multi-model consensus.

Symptom

In environments that export ANTHROPIC_BASE_URL as a bare host — AI coding harnesses like Claude Code and Cursor set ANTHROPIC_BASE_URL=https://api.anthropic.com (no /v1) for their own runtime — every Anthropic-driven step/assertion 404s at https://api.anthropic.com/messages, while Gemini-driven calls work fine. With consensus assertions the result is a puzzling half-working run rather than a clean failure, and nothing points at the env var.

Root cause

@ai-sdk/anthropic resolves options.baseURL → ANTHROPIC_BASE_URL env var → default https://api.anthropic.com/v1. A bare-host env value replaces the default including the /v1 path. passmark's getAnthropicProvider() calls createAnthropic({ apiKey }) without a baseURL, so the ambient env var silently wins.

This is known upstream — vercel/ai#15542 and vercel/ai#15580 (open), with normalization PRs vercel/ai#15545 / vercel/ai#15581 still open and vercel/ai#15586 closed unmerged — so it may be a while before the SDK handles it.

Suggested mitigation in passmark

Until the SDK normalizes, either:

• normalize in getAnthropicProvider() — append /v1 when the env value lacks a version path before constructing the provider, or
• a note in the troubleshooting guide ("Anthropic 404s but Gemini works → check ANTHROPIC_BASE_URL").

Workaround we use

Normalize before configure():

const base = process.env.ANTHROPIC_BASE_URL;
if (base && !/\/v1\/?$/.test(base)) {
  process.env.ANTHROPIC_BASE_URL = `${base.replace(/\/$/, '')}/v1`;
}

In-context: https://github.com/slusset/edyoucate-ai/blob/8c16384d9022c024e3fdf816f2b40b22fa8a6c76/sveltekit/e2e/passmark/passmark-shared-config.ts

Happy to PR the getAnthropicProvider() normalization if you'd take it.