feat(libsy): decide api

crates/libsy/README.md

@@ -114,13 +114,23 @@ the id it calls — they can differ (`"strong"` → `"openai/gpt-4o"`) or coinci
 
 ## Running a request
 
-Hold the algorithm as `Arc<dyn Algorithm>` and choose one of two entry points:
+Hold the algorithm as `Arc<dyn Algorithm>` and pick an entry point. They vary on two
+independent axes: **who serves the model calls**, and **whether the final call is made at
+all**.
+
+|  | libsy serves the calls | you serve the calls |
+|---|---|---|
+| **serve the final call** | `run` | `run_stream` |
+| **hand the final call back** | `decide` | `run_decision_only_stream` |
 
 ```rust
 // run: libsy drives the request to completion, serving each call with the target's
 // client, and returns (trace, response). Errors if a routed target has no client.
 let (trace, response) = algo.clone().run(Context::default(), req).await?;
 
+// decide: same, but stops one step short — see "Deciding without calling" below.
+let (trace, decided) = algo.clone().decide(Context::default(), req).await?;
+
 // run_stream: "ask, don't call" — you drive the stream and make the calls.
 let stream = algo.clone().run_stream(Context::default(), req);
 ```
@@ -192,6 +202,67 @@ while let Some(step) = stream.next().await {
 }
 ```
 
+## Deciding without calling (`decide`)
+
+Sometimes you want the routing answer, not the completion — you have your own transport,
+a cache to check first, or a proxy that will forward the request itself. `decide` runs the
+algorithm normally and stops one step short: instead of a `Response` you get the
+**decision, the request to serve it with, and any response already obtained**.
+
+"One step short" means libsy stops *committing* to the call, not that no model was called.
+Deciding routinely costs model calls of its own and those still happen — so whether the
+selected model has already been called depends on how the algorithm decides:
+
+- **Deciding from the request** — a judge scores the prompt and picks a tier. The judge is
+  called; the selected model is not. `response` is `None`.
+- **Deciding from a response** — the algorithm needs the model's *output* to decide, so it
+  calls one and analyzes the answer (escalate if it looks weak, keep it if it doesn't).
+  That call already happened, and `response` is `Some` — the selected model's answer.
+
+The routed call is the only thing left unmade. The decision still binds whatever state the
+algorithm retains — session affinity latches, and later turns follow that assignment
+whether or not you served this one — exactly as under `run`. Read `decide` as "route this
+turn, I will serve it myself", not "what would you do if I asked".
+
+```rust
+let (trace, (decision, request, response)) = algo.clone().decide(Context::default(), req).await?;
+println!("route to {}", decision.selected_model());
+
+match response {
+    // The selected model has already answered — deciding needed its output. Use this
+    // response as-is, or drop it and call `decision.selected_model()` again; both are
+    // valid, it is your cost/latency tradeoff.
+    Some(response) => { /* use it, or re-call */ }
+    // Not called yet: serve `request` against `decision.selected_model()` yourself.
+    None => { /* your call */ }
+}
+```
+
+Either way `response`, when present, corresponds to `decision` — it is that target's
+answer to `request`, not some intermediate the algorithm discarded.
+
+`run_decision_only_stream` is the "you serve the calls" form: the same `CallLlm` /
+`Decision` steps as `run_stream`, ending in `DecisionOnlyStep::ReturnToAgent` carrying that
+same triple.
+
+An algorithm does not branch on any of this. The mode is fixed by the entry point, recorded
+on the `Driver`, and applied by `Driver::final_decision` — so an algorithm that ends on
+`final_decision` supports all four entry points without mentioning them:
+
+```rust
+// the last thing create_run_task does: conclude on the winning target
+driver.final_decision(ctx, &target, request, decision, &mut already_served).await
+```
+
+`already_served` is an `Option<Response>` the algorithm may have picked up on the way (a
+classifier whose deciding call also answered the turn hands it back through
+`Classifier::score`). It is borrowed, not moved, because `Response` is not `Clone` —
+`final_decision` takes it only on the branch that consumes it.
+
+An algorithm that answers without routing — `Noop`, or one that builds a `Response`
+directly rather than concluding through `final_decision` — has no route to hand back, so
+`decide` on it fails with `LibsyError::AlgorithmError`.
+
 ## Building an algorithm (`Algorithm`)
 
 Implement `Algorithm` to add a strategy. You write `create_run_task` — one call per
@@ -206,11 +277,13 @@ pub trait Algorithm: Send + Sync + 'static {
     fn name(&self) -> &str;
     // `self: Arc<Self>` (not `&mut`): one algorithm serves requests concurrently — use
     // interior mutability for state. Offload calls/decisions on `driver`.
+    // Ends on `driver.final_decision(..)`, which yields `Response` or `Decision` according
+    // to the run's mode — so one implementation serves every entry point.
     async fn create_run_task(self: Arc<Self>, ctx: Context, driver: Driver, request: Request)
-        -> switchyard_libsy::Result<Response>;
+        -> switchyard_libsy::Result<ResponseOrDecision>;
     async fn process_signals(self: Arc<Self>, signals: Signals)
         -> switchyard_libsy::Result<()>;
-    // provided: run(ctx, request) -> (trace, response), run_stream(ctx, request) -> Stream<Step>
+    // provided: run / decide -> (trace, ..), run_stream / run_decision_only_stream -> Stream<..>
 }
 
 pub trait Decision: Send + Sync {

crates/libsy/examples/ensemble.rs

@@ -24,7 +24,9 @@ use std::sync::Arc;
 use async_trait::async_trait;
 use parking_lot::Mutex;
 
-use switchyard_libsy::{Algorithm, Driver, LibsyError, LlmTarget, LlmTargetSet, Result};
+use switchyard_libsy::{
+    Algorithm, Driver, LibsyError, LlmTarget, LlmTargetSet, ResponseOrDecision, Result,
+};
 use switchyard_llm_client::{Backend, HttpBackendConfig, ModelConfig, TranslatingLlmClient};
 use switchyard_protocol::{
     Context, Decision, Request, Response, RoutedLlmClient, completion_text, prompt_text,
@@ -181,7 +183,7 @@ impl EnsembleOrchAlgo {
         ctx: Context,
         request: Request,
         model: String,
-    ) -> Result<(Vec<Arc<dyn Decision>>, Response)> {
+    ) -> Result<(Vec<Arc<dyn Decision>>, ResponseOrDecision)> {
         let target = self.target_set.get_target(&model)?;
         let decision: Arc<dyn Decision> = Arc::new(EnsembleDecision {
             reasoning: format!(
@@ -201,20 +203,21 @@ impl EnsembleOrchAlgo {
             raw_request: request.raw_request,
             metadata: request.metadata,
         };
+        // The committed model is this run's final call, so it concludes the run.
         let response = driver
-            .call_llm_target(ctx, &target, routed, decision.clone())
+            .final_decision(ctx, &target, routed, decision.clone(), &mut None)
             .await?;
         Ok((vec![decision], response))
     }
 
     /// One exploration turn: fan out to every candidate, judge the survivors,
-    /// tally the winner, and return its response.
+    /// tally the winner, and conclude on its response.
     async fn ensemble_turn(
         &self,
         driver: &Driver,
         ctx: Context,
         request: Request,
-    ) -> Result<(Vec<Arc<dyn Decision>>, Response)> {
+    ) -> Result<(Vec<Arc<dyn Decision>>, ResponseOrDecision)> {
         let user_prompt = prompt_text(&request.llm_request);
         // The agent's inbound name rides through every sub-call unchanged; the model
         // each call hits is carried by its decision, not stamped onto the request.
@@ -284,7 +287,12 @@ impl EnsembleOrchAlgo {
                 metadata: request.metadata.clone(),
             };
             let judge_response = driver
-                .call_llm_target(ctx, &judge_target, judge_request, judge_decision.clone())
+                .call_llm_target(
+                    ctx.clone(),
+                    &judge_target,
+                    judge_request,
+                    judge_decision.clone(),
+                )
                 .await?;
             // Fail open: an unparseable pick falls back to the first response.
             let choice = parse_choice(
@@ -310,19 +318,38 @@ impl EnsembleOrchAlgo {
             state.turns += 1;
         }
 
+        let winner_target = self.target_set.get_target(&winner_model)?;
         let winner_decision: Arc<dyn Decision> = Arc::new(EnsembleDecision {
             reasoning: format!("judge selected '{winner_model}' as best response"),
             selected_model: winner_model,
             phase: EnsemblePhase::Winner,
         });
 
+        // The winner was already served as one of the candidate calls, so concluding
+        // here hands that response back rather than paying for the turn twice. The
+        // request is the one the winner actually answered, not the raw inbound.
+        let winner_request = Request {
+            llm_request: text_request(inbound, user_prompt),
+            raw_request: request.raw_request,
+            metadata: request.metadata,
+        };
+        let terminal = driver
+            .final_decision(
+                ctx,
+                &winner_target,
+                winner_request,
+                winner_decision.clone(),
+                &mut Some(winner_response),
+            )
+            .await?;
+
         // Trace order: [candidate calls..., judge?, winner].
         let mut trace = candidate_decisions;
         if let Some(judge_decision) = judge_decision {
             trace.push(judge_decision);
         }
         trace.push(winner_decision);
-        Ok((trace, winner_response))
+        Ok((trace, terminal))
     }
 }
 
@@ -382,11 +409,11 @@ impl Algorithm for EnsembleOrchAlgo {
         ctx: Context,
         driver: Driver,
         request: Request,
-    ) -> Result<Response> {
+    ) -> Result<ResponseOrDecision> {
         // Fast path: exploration is over — route straight to the committed model;
         // otherwise run a full ensemble turn. Both return a decision trace plus the
         // final response.
-        let (trace, response) = if let Some(model) = self.resolve_committed()? {
+        let (trace, terminal) = if let Some(model) = self.resolve_committed()? {
             self.route_committed(&driver, ctx.clone(), request, model)
                 .await?
         } else {
@@ -397,7 +424,7 @@ impl Algorithm for EnsembleOrchAlgo {
         for decision in trace {
             driver.info(ctx.clone(), decision).await?;
         }
-        Ok(response)
+        Ok(terminal)
     }
 }
 
@@ -450,7 +477,7 @@ async fn main() -> Result<()> {
         metadata: None,
     };
 
-    let (_, response) = algorithm.run(Context::default(), request).await?;
+    let (_, response) = algorithm.run(Context::default(), request, None).await?;
     println!(
         "{}",
         completion_text(
@@ -604,7 +631,7 @@ mod tests {
         // Judge prefers b/model; it should win and be returned.
         let (algo, calls) = algo(&["a/model", "b/model"], "judge/haiku", "b/model", 100);
         let (trace, response) = orch(algo)
-            .run(Context::default(), request("solve it"))
+            .run(Context::default(), request("solve it"), None)
             .await?;
         assert_eq!(
             response
@@ -639,15 +666,22 @@ mod tests {
         let orch = orch(algo);
 
         // Two exploration turns.
-        orch.clone().run(Context::default(), request("t1")).await?;
-        orch.clone().run(Context::default(), request("t2")).await?;
+        orch.clone()
+            .run(Context::default(), request("t1"), None)
+            .await?;
+        orch.clone()
+            .run(Context::default(), request("t2"), None)
+            .await?;
         let judge_calls_after_exploration =
             calls.lock().iter().filter(|c| *c == "judge/haiku").count();
         assert_eq!(judge_calls_after_exploration, 2);
 
         // Third request: committed fast path — routes straight to b/model with no
         // fan-out to a/model and no judge call.
-        let (trace, response) = orch.clone().run(Context::default(), request("t3")).await?;
+        let (trace, response) = orch
+            .clone()
+            .run(Context::default(), request("t3"), None)
+            .await?;
         assert_eq!(
             response
                 .llm_response
@@ -672,7 +706,9 @@ mod tests {
     #[tokio::test]
     async fn single_candidate_skips_the_judge() -> Result<()> {
         let (algo, calls) = algo(&["only/model"], "judge/haiku", "only/model", 100);
-        let (trace, response) = orch(algo).run(Context::default(), request("hi")).await?;
+        let (trace, response) = orch(algo)
+            .run(Context::default(), request("hi"), None)
+            .await?;
         assert_eq!(
             response
                 .llm_response
@@ -695,7 +731,10 @@ mod tests {
         let (algo, calls) = algo(&["a/model", "b/model"], "judge/haiku", "b/model", 0);
         let orch = orch(algo);
         for _ in 0..3 {
-            let (trace, _) = orch.clone().run(Context::default(), request("x")).await?;
+            let (trace, _) = orch
+                .clone()
+                .run(Context::default(), request("x"), None)
+                .await?;
             // Always a full ensemble turn (never a lone Committed decision).
             assert_eq!(
                 as_ensemble(&trace[trace.len() - 1])?.phase,
@@ -744,7 +783,7 @@ mod tests {
         );
         assert!(
             orch(algo)
-                .run(Context::default(), request("x"))
+                .run(Context::default(), request("x"), None)
                 .await
                 .is_err()
         );
@@ -836,7 +875,7 @@ mod tests {
         let run = |session: Arc<dyn Algorithm>, prompt: &'static str| {
             tokio::spawn(async move {
                 session
-                    .run(Context::default(), request(prompt))
+                    .run(Context::default(), request(prompt), None)
                     .await
                     .map(|(_, response)| {
                         response
@@ -878,15 +917,15 @@ mod tests {
             tokio::spawn(async move {
                 session
                     .clone()
-                    .run(Context::default(), request("t1"))
+                    .run(Context::default(), request("t1"), None)
                     .await?;
                 session
                     .clone()
-                    .run(Context::default(), request("t2"))
+                    .run(Context::default(), request("t2"), None)
                     .await?;
                 let (trace, response) = session
                     .clone()
-                    .run(Context::default(), request("t3"))
+                    .run(Context::default(), request("t3"), None)
                     .await?;
                 let phase = trace
                     .last()

crates/libsy/examples/research_agent.rs

@@ -83,7 +83,11 @@ impl ResearchAgent {
                 metadata: None,
             };
 
-            let (_trace, response) = self.algo.clone().run(Context::default(), request).await?;
+            let (_trace, response) = self
+                .algo
+                .clone()
+                .run(Context::default(), request, None)
+                .await?;
             let aggregate = response
                 .llm_response
                 .into_agg()