feat(server): report declared model capabilities on /v1/models

[elyasmnvidian]

schema_version = 1

[llm_clients.primary]
format = "openai_chat"
base_url = "http://localhost:8000/v1"

[targets.deep]
id = "nvidia/deepseek-ai/deepseek-v4-pro"
llm_client = "primary"

[routes.smart]
id = "smart"
type = "passthrough"
target = "deep"

With this config, the route has no way to declare its context window or tool support. GET /v1/models returns this entry:

{
  "id": "smart",
  "object": "model",
  "type": "model",
  "created": 0,
  "owned_by": "switchyard",
  "display_name": "smart",
  "capabilities": {
    "streaming": true,
    "tool_calling": null,
    "context_window": null,
    "supported_inbound_formats": [
      "openai-chat-completions",
      "openai-responses",
      "anthropic-messages"
    ]
  }
}

Fix

context_window and tool_calling belong on the route because the route ID is the public model returned by /v1/models. Two routes can share a target and still advertise different settings.

[routes.smart]
id = "smart"
type = "passthrough"
target = "deep"
context_window = 1000000
tool_calling = true

The same endpoint now reports:

{
  "streaming": true,
  "tool_calling": true,
  "context_window": 1000000,
  "supported_inbound_formats": [
    "openai-chat-completions",
    "openai-responses",
    "anthropic-messages"
  ]
}

These fields only describe the route; they do not enforce a request limit. Both are optional, so existing configs still return null.

The server rejects target-level declarations and a zero context window during config loading:

unknown field `context_window`
route smart context_window must be greater than zero

Routing and request handling are unchanged.

Test

cargo test -p switchyard-server

The endpoint test gives three routes the same target. One route reports true, one reports false, and one leaves both fields null.

This is the operator-declared follow-up from #105.

[coderabbitai]

Walkthrough

The change adds declared model capabilities for context-window size and tool calling. Route construction aggregates serving-target declarations, excludes classifier and judge targets, stores capabilities with routes, and exposes them through /v1/models.

Changes

Model Capability Advertisement

Layer / File(s)	Summary
Capability contract and aggregation crates/switchyard-server/src/capabilities.rs	Adds optional context_window and tool_calling fields. Aggregation selects the smallest declared context window and requires complete tool-calling declarations. Missing declarations and empty routes produce None.
Route capability derivation crates/switchyard-server/src/config.rs	Adds target capability hints, rejects zero context windows, aggregates serving-target capabilities, and excludes classifier and stage-router judge targets.
State and models endpoint integration crates/switchyard-server/src/lib.rs, crates/switchyard-server/tests/server.rs	Stores capabilities with route algorithms, serializes them in /v1/models, updates test state builders, and adds endpoint coverage for declared, missing, and excluded target capabilities.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

A rabbit checks each declared sign,
Smallest windows align.
Judges wait outside the route,
Models carry fields throughout.
Tests hop brightly: all is fine.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR preserves null capabilities without declarations and adds explicit operator declarations as the linked issue permits [#105].
Out of Scope Changes check	✅ Passed	The configuration, aggregation, endpoint, API, and tests directly support declared capability reporting and compatibility requirements.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes reporting declared model capabilities through /v1/models.

✨ Finishing Touches 💡 1

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch emehtabuddin/models-capabilities

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/switchyard-server/src/capabilities.rs`:
- Around line 23-40: Extend ModelCapabilities and ModelCapabilities::for_models
to infer and store a tool_calling capability alongside context_window, using the
established default for empty routes and the least-capable result across
model_ids. Update downstream GET /v1/models serialization to use this field
instead of hard-coding tool support, and add coverage for a model that does not
support tool calling.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d1483d77-6999-4092-b018-2090c7a0ded0

📥 Commits

Reviewing files that changed from the base of the PR and between 6b9a168 and 538fb3d.

📒 Files selected for processing (4)

• crates/switchyard-server/src/capabilities.rs
• crates/switchyard-server/src/config.rs
• crates/switchyard-server/src/lib.rs
• crates/switchyard-server/tests/server.rs

[elyasmnvidian]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[grahamking]

@elyasmnvidian Should we declare this at the route level (synthetic model) rather than the target? That would make it obvious what happens in case of conflict.

[route.random
id = "nvidia/random"
type = "random"
targets = ["model_a", "model_b"]
context_window = 1000000
tool_calling = true

Maybe Slack thread?

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-200/
Built to branch gh-pages at 2026-08-04 16:48 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[elyasmnvidian]

@grahamking Done. I moved context_window and tool_calling to the route config and removed the target aggregation. /v1/models now reads these values from the route, so two routes can share a target and advertise different settings. I added a test for that case.