[feature] Add a decision-only API for routing integrations

[dnandakumar-nv]

Problem

Some integrations need Switchyard to choose a model without also executing the
selected model request.

For example, a router plugin embedded in an existing LLM gateway may already
own provider credentials, retries, fallbacks, request execution, and response
handling. It only needs a routing decision from Switchyard so it can narrow a
candidate set to one model and continue through the gateway's normal execution
path.

Duplicating the routing logic is especially problematic because behavior and
metadata can drift as Switchyard's algorithms evolve.

Proposed solution

Add an asynchronous decision-only method to the Python libsy algorithm API:

from switchyard.libsy import LlmTarget, algorithms


request = {
    "model": "auto",
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Diagnose this failing build"},
            ],
        }
    ],
}

router = algorithms.random(
    [
        LlmTarget("fast"),
        LlmTarget("capable"),
    ],
    weights=[3, 1],
)

decision = await router.decide(request)

The result would contain the selected model and any routing metadata produced
by the algorithm:

{
    "selected_model": "fast",
    "reasoning": "random routing selected target 'fast'",
    "routing_tier": None,
}

This surface would be useful across the built-in routing algorithms that can
resolve to one target, including passthrough, random, LLM-classifier, and
stage-router configurations.

Alternatives considered

Run the selected target and discard its response

This performs an unnecessary and potentially expensive inference call. It also
prevents the embedding gateway from applying its normal execution, retry,
fallback, accounting, and response-handling behavior.

Reimplement routing in each integration

This avoids the final inference call but duplicates Switchyard policy in every
host. Algorithm behavior, validation, seeded selection, and decision metadata
can then diverge from Switchyard over time.

Call a Switchyard HTTP proxy for routing

An HTTP decision service could support remote integrations, but it adds another
deployment and network hop for Python applications that already embed libsy.
An in-process API is a smaller user-facing surface for these integrations and
does not preclude a separate HTTP API in the future.

Scope notes

• This is an extension of the existing libsy algorithm API and its Python
  bindings, not a new request/response component or LLMBackend in the
  Switchyard chain.
• Decision-only routing returns the initial selected route. Runtime fallback
  caused by executing a target, such as context-window overflow fallback, is
  outside the decision-only contract because no final target is executed.
• Decision streaming and per-request replacement of the configured candidate
  set are outside the initial API proposal.

Additional context

• LiteLLM router plugins on the proxy

[messiaen]

On the rust side, what do we think of this?
#207
@grahamking @nachiketb-nvidia @dnandakumar-nv @bbednarski9

[bbednarski9]

I'm starting to have doubts about the Decision-only API. Because with run_stream we can get an async iterator from SY and consume it. Plus it's much easier to report feedback to the model via callbacks rather than an additional API. Is the decision only router only relevant to Random and LLM Classifier?

Wondering if we have an immediate UC beyond Relay integration for this. Because relay integration is underway using run_stream

[bbednarski9]

Just read Greg's PR and I love the reuse. I'm starting to be a believer again. @messiaen