feat(providers): endpoint failover and provider path close-out

CONTEXT.md

@@ -207,6 +207,97 @@ An LLM vendor adapter (`providers/`: Anthropic, OpenAI, Gemini, …), shared by
 agent loop and the Curator.
 _Avoid_: conflating provider (vendor) with model (a model name a provider serves)
 
+A Provider is described along four independent axes -- identity, connection, routing,
+and what its models can do -- each with its own home. Mixing them in one record is what
+left per-model facts nowhere to live and per-provider facts stated in several places at
+once. The terms below name the pieces those axes are built from; they are properties of
+a model or of a connection, not four synonyms for Provider.
+
+**Model Ref**:
+The canonical way a model is written down: `provider/model`, naming whoever serves it.
+Usually that is the section it was configured under; where a Provider declares
+`skip_prefixes` it may instead be the gateway already named in the id
+(`openrouter/z-ai/glm-4.6` stored under `zai` keeps OpenRouter's name, because
+OpenRouter is what serves it). Produced by `providers/wire.py::stored_model_id`, which
+every surface that persists a choice goes through.
+_Avoid_: "model id" for the stored form when the sent form is also in play — say Model
+Ref or Wire Model.
+
+**Merge Key**:
+The identity of a Model Ref for comparison and de-duplication — the provider and the
+vendor's own id, spelling-folded. Two refs naming one model share a Merge Key whatever
+spelling either was written in.
+
+**Wire Model**:
+The form a Model Ref takes on the request: a LiteLLM route string, an Azure deployment
+name, or a Codex slug. Derived, never stored, and derived in one place
+(`providers/wire.py::wire_model`).
+_Avoid_: treating the stored and sent forms as one string — they differ per provider.
+
+**Auth Method**:
+One way of connecting to a Provider: what credential material it needs (as an AND of
+OR-groups), how that material is obtained, where it is kept, and how it is verified.
+A Provider may declare several and is usable when any one is satisfied.
+`providers/auth.py::credential_status` answers "is this Provider usable", and is the
+only place that may: seven surfaces once decided it independently and disagreed with
+each other on the two configurations that made the rewrite necessary.
+_Avoid_: "credential kind" for the whole shape — that names only the material.
+
+**Model Row**:
+One model as a person reads it: a Model Ref plus a label and a description, tagged with
+the source that supplied them. Display only — nothing shaping a request reads a Model
+Row (`providers/catalog.py`).
+_Avoid_: confusing it with what a model can *do*. Whether a request may carry
+`cache_control` blocks is a Prompt Cache Breakpoint question, not a Model Row one.
+
+**Model Overlay**:
+What a user states about a model no catalogue carries — a label and a description for a
+self-hosted deployment. Beats the catalogue for the fields it sets.
+
+**Prompt Cache Breakpoint**:
+An Anthropic-shaped `cache_control` marker placed on a request so the prefix before it is
+cached. Whether one may be placed is **(wire x model family)**: the wire has to have
+somewhere to carry the field (`ProviderSpec.supports_prompt_caching`, a property of the
+API being spoken) *and* the model's vendor has to be the one that reads it. A gateway
+accepting the field is not the same as its upstream honouring it -- OpenRouter carries it
+for every model it fronts and forwards it to vendors that bill the prompt twice.
+Decided once, in `providers/prompt_cache.py`, which every marker asks.
+_Avoid_: reading LiteLLM's per-model `supports_prompt_caching`, which answers "does this
+model cache at all" -- a different question, and the one that produced the doubled bill.
+
+**Token Rates**:
+What a model costs per token, and separately how much context it holds. Both are facts
+about a Provider's catalogue, so both are resolved in `providers/rates.py` rather than by
+whoever is about to report a number. The two are deliberately sourced differently: rates
+price a call after it happened, so the ladder may reach a community-maintained catalogue;
+a context window sizes trimming and therefore shapes the *next* request, so only the
+tables that also route may answer it. The window walks its own ladder
+(`effective_context_window`): an explicitly configured value wins outright, then the
+model's real window, then the module's documented fallback -- and a gauge that cannot
+resolve the real window reports 0 so the UI shows its empty state rather than a number
+that is nobody's.
+_Avoid_: "pricing" for the resolution -- that names the arithmetic on top
+(`token_wise/pricing.py`), which is a different module for a reason.
+
+**Provider Pin**:
+`agents.defaults.provider`: an explicit override of the Provider a Model Ref names.
+Every surface that changes the model rewrites it by one rule
+(`providers/pin.py::resolve`), because a pin left behind routes the new model to the old
+vendor with the old vendor's key.
+_Avoid_: reading it as a provider *signal* -- a pinned name says which section to ask
+about, never that the section holds credentials.
+
+**Provider Endpoint**:
+One url/key/headers group a provider section offers, of possibly several
+(`ProviderConfig.endpoints`, resolved through `providers/endpoints.py::provider_endpoints`
+whichever spelling the section used -- explicit list, Gemini's `api_key_list`, or the
+flat fields). Several endpoints on one section mean several accounts on the same vendor;
+`EndpointRotorProvider` spreads and fails over across them.
+_Avoid_: two same-sounding neighbors. Routing's `ModelEndpoint` (`RoutingConfig.models`)
+keys by *model* and picks a backend per request; a Provider Endpoint keys by *account*
+under one provider. And a bare `api_base` is one endpoint's address, not the endpoint --
+an endpoint is the whole credential group under a label.
+
 ### TUI-RPC
 
 **TUI-RPC**:

README.md

@@ -503,6 +503,9 @@ Raven is early, and useful contributions are welcome across runtime
 architecture, TUI polish, provider support, memory workflows, proactivity,
 benchmarks, documentation, and issue reports.
 
+Model vendors interested in a first-party integration or an open-source
+partnership (as MiniMax already has) are welcome to open an issue and say so.
+
 Before opening a PR:
 
 1. Read [AGENTS.md](AGENTS.md).

benchmarks/clawbench/stream.py

@@ -98,9 +98,10 @@ def __init__(
         restrict_to_workspace: bool,
     ) -> None:
         from raven.agent.loop import AgentLoop
-        from raven.cli.commands import _make_provider
+        from raven.cli._helpers import make_provider as _make_provider
         from raven.config.loader import load_config, set_config_path
         from raven.config.raven import ContextConfig
+        from raven.providers.rates import effective_context_window
         from raven.session.manager import SessionManager
 
         workspace.mkdir(parents=True, exist_ok=True)
@@ -126,7 +127,9 @@ def __init__(
         self.config.agents.defaults.workspace = str(workspace.resolve())
         self.provider = UsageTrackingProvider(_make_provider(self.config))
         self.model = self.config.agents.defaults.model
-        self.context_window = int(context_window or self.config.agents.defaults.context_window_tokens)
+        self.context_window = effective_context_window(
+            self.model, context_window or self.config.agents.defaults.context_window_tokens
+        )
         self.curator_model = curator_model or self.model
         self.session_id = session_id
         self.previous_totals = dict(self.provider.accumulated)

benchmarks/pinchbench/direct/raven_executor.py

@@ -330,38 +330,20 @@ def _make_benchmark_provider(model: str, api_key: str, api_base: str, provider_n
 
 
 def _estimate_cost_usd(model: str, prompt_tokens: int, completion_tokens: int) -> float | None:
-    """Estimate USD cost using LiteLLM's pricing database with manual fallbacks.
+    """Estimate USD cost for one call, or None when no source prices the model.
 
-    Falls back to _fallback_pricing for models not yet in LiteLLM's DB.
-    Returns None if the model is unknown to both.
+    A thin wrapper on purpose. This carried its own two-tier copy of the
+    resolution -- LiteLLM, then a hand-written table -- while the shipped ladder
+    grew a live gateway table and a per-vendor catalogue between them, so a
+    benchmark could report a figure the product would not. There is one answer to
+    "what does this cost" and it lives with the provider.
     """
-    # Manual fallback pricing ($/token) for models absent from LiteLLM's DB.
-    # Source: OpenRouter model pages (as of 2026-03).
-    _fallback_pricing: Dict[str, tuple[float, float]] = {
-        "z-ai/glm-4.5-air": (0.13e-6, 0.85e-6),  # $0.13/$0.85 per 1M tokens
-    }
-
-    try:
-        import litellm
-
-        # LiteLLM expects "openrouter/<provider>/<model>" format for OpenRouter models.
-        or_model = f"openrouter/{model}" if not model.startswith("openrouter/") else model
-        prompt_cost, completion_cost = litellm.cost_per_token(
-            model=or_model,
-            prompt_tokens=prompt_tokens,
-            completion_tokens=completion_tokens,
-        )
-        return prompt_cost + completion_cost
-    except Exception:
-        pass
-
-    # Fallback for models not in LiteLLM DB
-    base_model = model.removeprefix("openrouter/")
-    if base_model in _fallback_pricing:
-        p_per_tok, c_per_tok = _fallback_pricing[base_model]
-        return p_per_tok * prompt_tokens + c_per_tok * completion_tokens
+    from raven.token_wise.pricing import estimate_cost_usd
 
-    return None
+    # Benchmarks name OpenRouter models bare; the ladder is keyed by stored ids,
+    # which name their provider.
+    stored = model if model.startswith("openrouter/") else f"openrouter/{model}"
+    return estimate_cost_usd(stored, prompt_tokens, completion_tokens)
 
 
 async def _run_turn_text(agent, message: str, *, session_key: str, chat_id: str) -> str:

pyproject.toml

@@ -147,6 +147,8 @@ include = [
     "raven/templates/**/*.md",
     "raven/skills/**/*.md",
     "raven/skills/**/*.sh",
+    # Bundled model-label snapshot, so a fresh install labels models offline.
+    "raven/providers/data/*.json",
     # Tracing dashboard viewer (dependency-free Node server + static client).
     "raven/tracing/viewer/**/*.js",
     "raven/tracing/viewer/**/*.css",