Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -127,9 +127,37 @@ LOG_LEVEL=INFO
# COMPUTE_PERMITTED_EXECUTABLES='["/opt/conda/envs/tomopy/bin/tomopy"]'
# CORA_ALLOW_RAW_CONDUCT=false

# LLM provider — Agent BC (Phase 8f-b)
# Required if the deployment ships the RunDebrief or CautionDrafter subscribers.
# Without it, the subscribers register but log-and-skip on every Run-terminal event.
# LLM provider: the switch and the credential
# Two settings, and BOTH are required before CORA calls an external model.
# LLM_ENABLED is the SWITCH and defaults to false; ANTHROPIC_API_KEY is the
# CREDENTIAL. Set only the key and nothing calls out: the switch decides.
#
# That split exists because a key is often present in an environment for an
# unrelated reason, and its mere presence used to be enough to register the
# RunDebriefer and CautionDrafter subscribers, which call an external API on
# EVERY terminal Run. That is experiment metadata leaving the facility and
# money being spent, switched on by a side effect. Every sibling subscriber
# already had its own default-off flag; this seam was the outlier.
#
# Leaving it off is graceful, never a crash: the two LLM-backed subscribers
# are skipped with a warning naming which of the two settings is missing,
# `regenerate_run_debrief` answers unavailable, and a conduct command that
# explicitly asks for the `llm` decide substrate gets a 422.
#
# UPGRADING: a deployment that ran the LLM subscribers on ANTHROPIC_API_KEY
# alone must now also set LLM_ENABLED=true, or those two stop registering.
# Nothing crashes; a boot warning names the switch and /readyz reports "off".
#
# NOTE this is a DIFFERENT axis from actuation (CONTROL_WRITES_ENABLED and
# COMPUTE_SUBSTRATE above). Those bound what CORA can MOVE; these bound the
# LLM path. A facility told "read-only" hears the first and would still
# object to the second, so neither promise covers the other.
#
# Read the scope narrowly: this bounds the LLM path ONLY, not egress in
# general. The HTTP checksum adapter is wired unconditionally for http/https
# Distributions, so CORA can still make outbound requests with the LLM off.
# Verify the effective state at `GET /readyz` -> `"llm": "off" | "live"`.
# LLM_ENABLED=true
# ANTHROPIC_API_KEY=sk-ant-...

# Observability — OpenTelemetry
Expand Down
2 changes: 1 addition & 1 deletion apps/api/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -18383,7 +18383,7 @@
}
}
},
"description": "RunDebriefer subscriber is not wired (ANTHROPIC_API_KEY unset)."
"description": "RunDebriefer is not wired (LLM_ENABLED off, the default, or no ANTHROPIC_API_KEY). The body names which."
}
},
"summary": "Re-invoke RunDebriefer on demand against a specific Run.",
Expand Down
12 changes: 8 additions & 4 deletions apps/api/src/cora/agent/_subscribers.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,12 @@
The deterministic subscribers (AuthorityRevocationHolder,
CautionPromoter) register independently of `kernel.llm`. The
LLM-backed subscribers (RunDebriefer, CautionDrafter) register only
when `kernel.llm` is wired (`ANTHROPIC_API_KEY` set); if it is None
they are skipped with a warning rather than refusing to boot a
deployment that wants to defer Agent rollout.
when `kernel.llm` is wired, which needs BOTH `llm_enabled` (the
switch, default off) and `ANTHROPIC_API_KEY` (the credential); if it
is None they are skipped with a warning rather than refusing to boot a
deployment that wants to defer Agent rollout. These two are the seam
through which experiment metadata would leave the facility and tokens
would be spent, on every terminal Run, so the switch defaults off.

## Registered subscribers

Expand Down Expand Up @@ -65,6 +68,7 @@

from typing import TYPE_CHECKING

from cora.agent.build_llm import llm_unwired_reason
from cora.agent.subscribers.authority_revocation_holder import (
make_authority_revocation_holder_subscriber,
)
Expand Down Expand Up @@ -136,7 +140,7 @@ def register_agent_subscribers(registry: ProjectionRegistry, deps: Kernel) -> No
_log.warning(
"agent_subscriber.skipped",
subscribers=["run_debriefer", "caution_drafter"],
reason="kernel.llm is None (ANTHROPIC_API_KEY not configured)",
reason=llm_unwired_reason(deps.settings),
)
return

Expand Down
68 changes: 57 additions & 11 deletions apps/api/src/cora/agent/build_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,23 @@
owns `PostgresClearanceLookup`, Caution BC owns
`PostgresCautionLookup`).

When `Settings.anthropic_api_key` is unset, returns `None` so the
Kernel ends up with `llm=None` and Agent subscribers fail-fast at
registration. This is intentional: a misconfigured prod deployment
should not silently downgrade to a no-LLM mode where RunDebriefer
goes silent. The subscriber-registration step fail-fasts on
`kernel.llm is None`.
Returns `None` unless BOTH `Settings.llm_enabled` (the switch) and
`Settings.anthropic_api_key` (the credential) are set, so the Kernel
carries `llm=None` and no external model is ever called.

`llm=None` is a fully supported state, not a degraded one: every
consumer already handles it because the key-absent case always
produced it. The LLM-backed subscribers log-and-skip
(`cora.agent._subscribers`) and `regenerate_run_debrief` answers
unavailable. The `llm` decide substrate cannot be selected by a remote
caller at all (`WireDecideSubstrate` admits only `in_memory` and
`grid_walk`), so `build_decide_port`'s `llm is None` guard is an
internal-caller guard, not a request-reachable path.

(An earlier version of this docstring claimed subscriber registration
"fail-fasts" on `kernel.llm is None`. It does not, and never did:
`register_agent_subscribers` logs a warning and skips, deliberately,
so a deployment may defer Agent rollout without refusing to boot.)
"""

from cora.agent.adapters.anthropic_llm import AnthropicLLM
Expand All @@ -21,21 +32,56 @@


def build_llm(settings: Settings) -> LLM | None:
"""Construct the production LLM or return `None` when unconfigured.
"""Construct the production LLM, or `None` when off or unconfigured.

Two independent reasons to return `None`, and the switch is checked
first so an operator who turns the LLM off gets that answer even
with a credential present in the environment:

Today this branches on `settings.anthropic_api_key`; a future
multi-provider deployment would branch on a `settings.llm_provider`
field with `anthropic` as one variant.
- `llm_enabled` is False (the default): this deployment does not
call an external model, so no experiment metadata leaves the
facility through this seam and no tokens are spent.
- no `anthropic_api_key`: nothing to call it with.

Today the credential branch reads `settings.anthropic_api_key`; a
future multi-provider deployment would branch on a
`settings.llm_provider` field with `anthropic` as one variant.

`SecretStr.get_secret_value()` is the ONLY place in the codebase
that unwraps the API key; passing the raw string to the adapter
constructor is the boundary at which "secret material" becomes
"live credential". Adapter scope is responsible for not re-
exposing it (eg. via `repr(adapter)` or `str(adapter._client)`).
"""
if not settings.llm_enabled:
return None
if settings.anthropic_api_key is None:
return None
return AnthropicLLM(api_key=settings.anthropic_api_key.get_secret_value())


__all__ = ["build_llm"]
def llm_unwired_reason(settings: Settings) -> str:
"""Say WHICH of the two settings left `kernel.llm` unwired.

One source of truth for every surface that has to explain the
unwired state: the subscriber-registration warning, the REST 503
body, and the MCP tool's error. They must agree, and each must send
the operator to the remedy that actually applies. Naming the
credential when the switch is simply off is the more likely mistake
now, because off is the default.

Callers are expected to have already established that the LLM is
unwired; this only explains why.
"""
if not settings.llm_enabled:
return (
"LLM_ENABLED is false, so this deployment calls no external model. "
"Set LLM_ENABLED=true (and ANTHROPIC_API_KEY) to turn the LLM features on."
)
return (
"LLM_ENABLED is true but ANTHROPIC_API_KEY is not configured. "
"Supply the credential and restart."
)


__all__ = ["build_llm", "llm_unwired_reason"]
11 changes: 7 additions & 4 deletions apps/api/src/cora/agent/features/regenerate_run_debrief/route.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from fastapi import APIRouter, Depends, Header, HTTPException, Path, Request, status
from pydantic import BaseModel, Field

from cora.agent.build_llm import llm_unwired_reason
from cora.agent.features.regenerate_run_debrief.command import RegenerateRunDebrief
from cora.agent.features.regenerate_run_debrief.handler import IdempotentHandler
from cora.infrastructure.routing import (
Expand Down Expand Up @@ -53,9 +54,8 @@ def _get_handler(request: Request) -> IdempotentHandler:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail=(
"regenerate_run_debrief is unavailable: kernel.llm is not wired "
"(ANTHROPIC_API_KEY not configured). Configure the API key "
"and restart, or use the test FakeLLM for non-prod."
f"regenerate_run_debrief is unavailable: kernel.llm is not wired. "
f"{llm_unwired_reason(request.app.state.deps.settings)}"
),
)
return handler
Expand Down Expand Up @@ -89,7 +89,10 @@ def _get_handler(request: Request) -> IdempotentHandler:
},
status.HTTP_503_SERVICE_UNAVAILABLE: {
"model": ErrorResponse,
"description": ("RunDebriefer subscriber is not wired (ANTHROPIC_API_KEY unset)."),
"description": (
"RunDebriefer is not wired (LLM_ENABLED off, the default, or no "
"ANTHROPIC_API_KEY). The body names which."
),
},
},
summary="Re-invoke RunDebriefer on demand against a specific Run.",
Expand Down
14 changes: 10 additions & 4 deletions apps/api/src/cora/agent/tools.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,8 @@ def register_agent_tools(
# regenerate_run_debrief handler is Optional in the bundle
# (None when kernel.llm is unwired). The tool's lambda dereferences
# at call time and raises a RuntimeError to surface as MCP isError;
# this matches the REST 503 semantics. Production deploys with
# ANTHROPIC_API_KEY set never hit the None branch.
# this matches the REST 503 semantics. Deploys reach the None branch
# whenever LLM_ENABLED is off (the default) or the key is unset.
regenerate_run_debrief_tool.register(
mcp,
get_handler=lambda: _resolve_regenerate_run_debrief(get_handlers()),
Expand Down Expand Up @@ -134,9 +134,15 @@ def _resolve_regenerate_run_debrief(handlers: AgentHandlers) -> RegenerateRunDeb
"""Dereference the Optional regenerate_run_debrief handler with a loud
failure when unwired. Mirrors the REST route's 503 path."""
if handlers.regenerate_run_debrief is None:
# Names BOTH settings rather than deriving which one applies:
# this resolver has no Settings in scope (the tool registry is
# wired from handlers alone), and threading one through to word a
# message would widen the registration signature. The REST 503,
# which does have Settings, names the specific one.
msg = (
"regenerate_run_debrief is unavailable: kernel.llm is not wired "
"(ANTHROPIC_API_KEY not configured)."
"regenerate_run_debrief is unavailable: kernel.llm is not wired. "
"It requires BOTH LLM_ENABLED=true (default false) and "
"ANTHROPIC_API_KEY."
)
raise RuntimeError(msg)
return handlers.regenerate_run_debrief
36 changes: 36 additions & 0 deletions apps/api/src/cora/api/_readiness.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,23 @@
deployment has no pool at all (the in-memory kernel), so there is
nothing to report rather than nothing wrong."""

LlmReach = Literal["off", "live"]
"""Whether this deployment calls an external language model.

`live` means both the switch (`llm_enabled`) and the credential
(`anthropic_api_key`) are present, so the LLM-backed subscribers are
registered and will call out on every terminal Run. `off` means no
external model is called through this seam.

Named `llm`, NOT `egress`, and the distinction is the honest part. This
reports ONE outbound path. It is not a claim that nothing leaves the
deployment: `HttpRangeChecksumAdapter` is wired unconditionally for
http/https Distributions (`cora.data.wire`), so CORA can make outbound
requests with the LLM entirely off. Calling this field `egress` would
promise a perimeter this code does not enforce, which is exactly the
overclaim `derive_actuation` is careful to avoid on its own axis.
"""

ActuationReach = Literal["inert", "reachable"]
"""Whether this deployment can drive a physical effect on the beamline.

Expand Down Expand Up @@ -194,6 +211,22 @@ def derive_actuation(settings: Settings) -> ActuationReach:
return "inert"


def derive_llm(settings: Settings) -> LlmReach:
"""Report whether an external language model gets called.

`live` requires BOTH the switch and the credential, mirroring
`build_llm`'s two guards, so this answers the question an operator
actually has ("is CORA phoning out and spending?") rather than
restating one flag. A deployment that sets `llm_enabled` and forgets
the key reads `off`, which is the truth: nothing is called.

Like `derive_actuation` this is a REPORT, not a gate. It decides
nothing; `build_llm` is the thing that refuses. And it is scoped to
the LLM seam alone, not to egress in general (see `LlmReach`).
"""
return "live" if settings.llm_enabled and settings.anthropic_api_key is not None else "off"


def readiness_body(database: DatabaseStatus, settings: Settings) -> dict[str, str]:
"""Render the probe result. Fixed vocabulary, no free text.

Expand All @@ -217,13 +250,16 @@ def readiness_body(database: DatabaseStatus, settings: Settings) -> dict[str, st
"database": database,
"app_env": settings.app_env,
"actuation": derive_actuation(settings),
"llm": derive_llm(settings),
}


__all__ = [
"ActuationReach",
"DatabaseStatus",
"LlmReach",
"derive_actuation",
"derive_llm",
"probe_database",
"readiness_body",
]
21 changes: 19 additions & 2 deletions apps/api/src/cora/api/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,12 @@
from cora.api._enclosure_permit_observer import ControlPortEnclosureObserver
from cora.api._inference_recorder import DelegatingInferenceRecorder
from cora.api._procedure_watcher import procedure_watcher_lifespan
from cora.api._readiness import derive_actuation, probe_database, readiness_body
from cora.api._readiness import (
derive_actuation,
derive_llm,
probe_database,
readiness_body,
)
from cora.api._run_initiator import run_initiator_lifespan
from cora.api._run_supervisor import run_supervisor_lifespan
from cora.api.middleware import BodySizeLimitMiddleware
Expand Down Expand Up @@ -776,6 +781,17 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
cora_allow_raw_conduct=settings.cora_allow_raw_conduct,
)

# The egress + spend axis, logged beside actuation because a
# facility hears both as one promise. `anthropic_api_key_present`
# is a BOOLEAN, never the key: SecretStr keeps the value out of
# every serialisation path and this line must not undo that.
_log.info(
"boot.llm_posture",
llm=derive_llm(settings),
llm_enabled=settings.llm_enabled,
anthropic_api_key_present=settings.anthropic_api_key is not None,
)

app.state.safety = wire_safety(deps)
app.state.caution = wire_caution(deps)
app.state.calibration = wire_calibration(deps)
Expand Down Expand Up @@ -882,7 +898,8 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
register_agent_projections(registry, deps)
# side-effecting Agent BC subscribers
# (RunDebriefer). Conditional: only registered when
# `kernel.llm` is wired (ANTHROPIC_API_KEY configured).
# `kernel.llm` is wired, which needs LLM_ENABLED (default
# off) AND ANTHROPIC_API_KEY.
register_agent_subscribers(registry, deps)
# budget BC's deterministic CampaignClosed -> seal reaction;
# unconditional (bookkeeping must not depend on an LLM key).
Expand Down
47 changes: 37 additions & 10 deletions apps/api/src/cora/infrastructure/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -165,17 +165,44 @@ class Settings(BaseSettings):
# 0.1s prevents accidental tight-loop misconfiguration.
projection_poll_interval_seconds: float = 5.0

# `llm_enabled` is the SWITCH; `anthropic_api_key` below is the
# CREDENTIAL. Both are required before CORA calls an external model,
# and this one defaults OFF.
#
# Why a switch at all, when a key is already needed: the key is often
# present in an environment for an unrelated reason, and its mere
# presence used to be enough to auto-register RunDebriefer +
# CautionDrafter, which call an external API on EVERY terminal Run.
# That is experiment metadata leaving the facility (EGRESS) and money
# being spent (SPEND), switched on by a side effect. Every one of the
# ~13 sibling subscribers carries its own `*_enabled` default-off
# flag; this seam was the outlier.
#
# Egress and spend are a DIFFERENT axis from actuation (the
# ControlPort write gate and the ComputePort exec allowlist, see
# `control_writes_enabled`). A facility told "read-only, CORA cannot
# touch my hardware" hears a claim about actuation, and would still
# be unhappy to discover CORA phoned its data out. Both promises
# belong to the observe-only posture; neither subsumes the other.
#
# Turning this off is graceful, never a crash: `build_llm` returns
# None, and every consumer already treats `llm=None` as a supported
# state because the key-absent case always produced it. The
# LLM-backed subscribers log-and-skip and `regenerate_run_debrief`
# answers unavailable. The `llm` decide substrate is not selectable
# over the wire at all, so its factory guard is an internal-caller
# guard rather than a request-reachable path.
llm_enabled: bool = False

# LLM provider — Agent BC wiring
# When None, `build_kernel` wires no LLM and the Kernel
# carries `llm=None`; subscribers that depend on the LLM (the
# RunDebriefer subscriber) raise / log-and-skip when they
# try to use it. The dev / test default of None matches the
# `AllowAllAuthorize` / `AlwaysCoveredClearanceLookup` test-
# bypass convention: tests don't need real API credentials.
# Production deployments that ship RunDebriefer MUST set this;
# the wire-up adds a startup gate that refuses
# to register `run_debriefer_subscriber` when this is unset (so
# the agent never runs blind).
# When None (or when `llm_enabled` is False above), `build_kernel`
# wires no LLM and the Kernel carries `llm=None`; the LLM-backed
# subscribers log-and-skip rather than refusing to boot, so a
# deployment may defer Agent rollout. The dev / test default of None
# matches the `AllowAllAuthorize` / `AlwaysCoveredClearanceLookup`
# test-bypass convention: tests don't need real API credentials.
# Production deployments that ship RunDebriefer MUST set BOTH this
# and `llm_enabled`.
#
# Read from `ANTHROPIC_API_KEY` env var (case-insensitive per
# pydantic-settings; matches the bare-field-name convention
Expand Down
Loading
Loading