diff --git a/.agents/skills/company-context/SKILL.md b/.agents/skills/company-context/SKILL.md index de445c546..7a2100266 100644 --- a/.agents/skills/company-context/SKILL.md +++ b/.agents/skills/company-context/SKILL.md @@ -1,11 +1,11 @@ --- name: company-context -description: "Use Centaur's indexed company context together with direct Slack, Linear, Google Drive, or Calendar searches when answering internal company-history, prior-decision, project-context, meeting-context, roadmap/status, or cross-source memory questions. Use for questions like what was discussed, decided, planned, mentioned, or documented internally, especially when the user did not name one exact source." +description: "Use Centaur's indexed company context together with direct Slack, Linear, Google Docs, Drive, or Calendar searches when answering internal company-history, prior-decision, project-context, meeting-context, roadmap/status, or cross-source memory questions. Indexed context includes Slack channels, user-visible Slack DMs, Google Docs, Google Calendar, and Linear. Use for questions like what was discussed, decided, planned, mentioned, or documented internally, especially when the user did not name one exact source." --- # Company Context -Use `company_context` as the first retrieval step for internal historical context. Its `search` command queries indexed company memory across enabled sources such as Slack, Google Drive/Docs, Google Calendar, and Linear. Always pair indexed results with the relevant direct source tools, then reconcile and collate both evidence sets before answering. +Use `company_context` as the first retrieval step for internal historical context. Its `search` command queries indexed company memory across enabled sources such as Slack channels, Google Docs (`--source docs`), Google Calendar, and Linear. It also has dedicated commands for user-visible Slack DMs and DM conversations. Always pair indexed results with the relevant direct source tools, then reconcile and collate both evidence sets before answering. ## Default Workflow @@ -15,6 +15,13 @@ Use `company_context` as the first retrieval step for internal historical contex company_context search "QUERY" --limit 10 --json ``` +For DM-specific questions, use the dedicated DM search surface: + +```bash +company_context search-dms "QUERY" --limit 10 --json +company_context search-dm-conversations "PERSON OR QUERY" --limit 10 --json +``` + 2. Read promising documents before answering: ```bash @@ -35,18 +42,22 @@ Use the source-specific tools that match the question. For broad cross-source qu 4. For time-sensitive or "latest" asks, check index freshness: ```bash -company_context latest-date --json -company_context latest-date --source slack --json -company_context latest-date --source linear --json +company_context latest-date +company_context latest-date --source slack +company_context latest-date --source docs --source-type google_doc +company_context latest-date --source linear ``` +`latest-date` always outputs JSON, so it does not accept a `--json` flag. + 5. If results are weak, broaden or target source filters: ```bash company_context search "QUERY" --source slack --limit 10 --json -company_context search "QUERY" --source google_drive --limit 10 --json +company_context search "QUERY" --source docs --source-type google_doc --limit 10 --json company_context search "QUERY" --source google_calendar --limit 10 --json company_context search "QUERY" --source linear --limit 10 --json +company_context search-dms "QUERY" --limit 10 --json ``` 6. Collate indexed and live results before answering: diff --git a/.agents/skills/creating-tools/SKILL.md b/.agents/skills/creating-tools/SKILL.md index 889cc5172..e2df253ab 100644 --- a/.agents/skills/creating-tools/SKILL.md +++ b/.agents/skills/creating-tools/SKILL.md @@ -1,34 +1,28 @@ --- name: creating-tools -description: "Scaffold and build new Centaur tool integrations in tools/. Use when asked to create a new tool, add an API integration, or build a new client for an external service." +description: "Scaffold and build new tool integrations in tools/. Use when asked to create a new tool, add an API integration, or build a new client for an external service." --- # Creating Tools -Build tools for the current Centaur runtime model: API metadata plus sandbox -CLI shims. Agents use `centaur-tools list`, ` --help`, and direct tool -CLIs. Do not scaffold new tools around legacy `/tools/{name}/{method}` HTTP -routes. +Scaffold and implement new tool integrations following the established conventions. ## File Structure -Prefer the existing categorized layout under `tools///`. -Match nearby tools when choosing the category. - -```text -tools/// -|-- __init__.py -|-- .env.example -|-- client.py -|-- cli.py -|-- pyproject.toml -`-- tests/ +Every tool lives at `tools//` with exactly these files: + +``` +tools// +├── __init__.py # Empty file +├── .env.example # Document required secrets (one per line: KEY=description) +├── client.py # API client class + _client() factory function +├── cli.py # Typer CLI for standalone use +└── pyproject.toml # Package metadata + [tool.ai-v2] section ``` -## Metadata +## Step-by-Step -Every tool needs a `pyproject.toml` with `[project.scripts]` and -`[tool.centaur]`. +### 1. Create `pyproject.toml` ```toml [project] @@ -44,87 +38,108 @@ dependencies = [ ] [project.scripts] - = ".cli:app" + = ".cli:app" [build-system] requires = ["hatchling"] build-backend = "hatchling.build" -[tool.centaur] +[tool.ai-v2] module = "client.py" -secrets = [ - {type = "http", name = "_API_KEY", mode = "inject", inject_header = "Authorization", inject_formatter = "Bearer {{ .Value }}", hosts = ["api.example.com"]}, -] ``` -Use `optional_secrets` when a credential unlocks optional behavior but the -tool can still run without it. Use `type = "pg_dsn"` for Postgres access; set -`name` to the environment variable the sandbox should see and set `database` -to the upstream database name. +The `[tool.ai-v2] module = "client.py"` line is **required** — the tool manager uses it to discover and register the tool. -## Client +Add extra dependencies only if needed (e.g., `websockets`, `pydantic`). The base set (`httpx`, `typer`, `rich`, `python-dotenv`) covers most tools. -Rules: +### 2. Create `client.py` -- Do not call `load_dotenv()` in `client.py`. -- Import `secret` from `centaur_sdk.tool_sdk`. -- Keep a class-based client plus a `_client()` factory. -- Use `secret("KEY", default="")` for credentials. -- Public methods should have clear type hints and return JSON-serializable - values. -- Keep mutating methods explicit with names like `create_`, `update_`, and - `delete_`. +Rules: +- **NO `load_dotenv()`** — secrets come from `secret()` helper or env vars at runtime +- **Import `secret` from `shared.tool_sdk`** — never use `os.getenv()` for API keys +- **Class-based** — one main client class with public methods +- **`_client()` factory function** at module bottom — this is how the tool manager instantiates the client +- **Methods starting with `_` are excluded** from tool registration (use for internal helpers) +- **Lifecycle methods** (`close`, `__enter__`, `__exit__`) are also excluded +- **All imports at file top** — never inside functions +- **Type hints on all public methods** — the tool manager uses them to generate schemas ```python """ API client.""" -from __future__ import annotations - import httpx +from shared.tool_sdk import secret -from centaur_sdk.tool_sdk import secret +class Client: + """Client for API.""" -class NameClient: - def __init__(self, api_key: str | None = None, timeout: float = 30.0) -> None: + def __init__(self, api_key: str | None = None, timeout: float = 30.0): self._api_key = api_key - self._timeout = timeout - self._base_url = "https://api.example.com" - - def _api_key_or_raise(self) -> str: - api_key = self._api_key or secret("_API_KEY", default="") + self.base_url = "https://api.example.com" + self.timeout = timeout + self._client: httpx.Client | None = None + + @property + def client(self) -> httpx.Client: + if self._client is None: + self._client = httpx.Client(timeout=self.timeout) + return self._client + + def _get_api_key(self) -> str | None: + if self._api_key: + return self._api_key + return secret("_API_KEY", "") + + def _request(self, endpoint: str, params: dict | None = None) -> dict | list: + api_key = self._get_api_key() if not api_key: - raise RuntimeError("_API_KEY not set") - return api_key + raise RuntimeError("_API_KEY not set.") + url = f"{self.base_url}{endpoint}" + headers = {"Authorization": f"Bearer {api_key}"} + try: + response = self.client.get(url, params=params, headers=headers) + response.raise_for_status() + return response.json() + except httpx.HTTPStatusError as e: + raise RuntimeError(f"API error: {e.response.status_code} - {e.response.text}") + except httpx.RequestError as e: + raise RuntimeError(f"Request failed: {e}") def search(self, query: str, limit: int = 10) -> dict: - response = httpx.get( - f"{self._base_url}/search", - headers={"Authorization": f"Bearer {self._api_key_or_raise()}"}, - params={"q": query, "limit": limit}, - timeout=self._timeout, - ) - response.raise_for_status() - return response.json() + """Search for items.""" + return self._request("/search", params={"q": query, "limit": limit}) + + def close(self): + if self._client: + self._client.close() + self._client = None - def health(self) -> dict: - return {"status": "ok"} + def __enter__(self): + return self + def __exit__(self, *args): + self.close() -def _client() -> NameClient: - return NameClient() + +def _client() -> Client: + api_key = secret("_API_KEY", "") + if not api_key: + raise RuntimeError("_API_KEY not set.") + return Client(api_key=api_key) ``` -## CLI +### 3. Create `cli.py` -CLIs run inside agent sandboxes and locally. They may call `load_dotenv()` so -local development can use `.env`, but keep the implementation as a thin wrapper -around `client.py`. +Rules: +- **YES `load_dotenv()` at the very top** — CLIs run standalone and need to load `.env` +- Thin wrapper around the client — each CLI command calls one client method +- Use `typer` for the CLI framework +- Use `rich` or `shared.cli_tables` for formatted output +- Support `--json` and `--markdown` output flags on every command ```python -"""CLI for .""" - -from __future__ import annotations +"""CLI for API.""" from dotenv import load_dotenv @@ -133,75 +148,121 @@ load_dotenv() import json import typer - -from .client import _client +from rich.console import Console +from shared.cli_tables import Table app = typer.Typer(name="", help="") +console = Console() -@app.command() -def search(query: str, limit: int = 10) -> None: - print(json.dumps(_client().search(query, limit=limit), indent=2)) +def get_client(): + from .client import Client + return Client() @app.command() -def health() -> None: - print(json.dumps(_client().health())) +def search( + query: str = typer.Argument(..., help="Search query"), + limit: int = typer.Option(10, "--limit", "-n", help="Max results"), + json_output: bool = typer.Option(False, "--json", help="Output as JSON"), +): + """Search for items.""" + client = get_client() + data = client.search(query, limit=limit) + if json_output: + print(json.dumps(data, indent=2)) + return + # ... rich table output ... if __name__ == "__main__": app() ``` -Use a `health` command for credentialed tools whenever possible. It gives QA a -safe, non-mutating deployment check. +### 4. Create `__init__.py` -## Secrets +Empty file: +```python +``` -Document required secrets in `.env.example`: +### 5. Create `.env.example` -```text +``` NAME_API_KEY=your-api-key-here ``` -For production credentials, create a matching secret source and request rule -through the deployment's secret manager. The tool should keep using -`secret("KEY")`; iron-proxy and the sandbox runtime decide whether that becomes -a placeholder, injected header, OAuth token, brokered token, GCP auth token, or -local proxy DSN. +### 6. Add to 1Password (if needed) -## Tests +If this is a credentialed tool, add the secret to 1Password: +- Vault: use the vault configured for your deployment +- Account: use the 1Password account configured for your deployment +- Item title: use the exact `ENV_VAR` name (e.g., `COINGECKO_API_KEY`) -Add focused tests for client behavior and CLI output. Avoid tests that hit a -real third-party API unless they are explicitly marked as live smoke tests. +### 7. Update `tools/README.md` -Run the packaging validator before staging: +Add a row to the "Available Plugins" table with the tool name, description, and required secrets. -```bash -python3 scripts/validate_cli_packaging.py -``` +## Secrets Resolution Order -## Verification +1. Tool `.env` file (`tools//.env`) — per-tool overrides for local dev +2. Root `.env` file (repo root) — central file for all secrets +3. Environment variables — Docker, CI, 1Password secret manager +4. Secret manager sidecar (`http://secrets:8100`) — production (accessed via `secret()`) -From a fresh sandbox or dev shell with shims installed: +**Always use `secret("KEY")` in client.py** — it handles all resolution layers. Never use `os.getenv()` or `os.environ` for API keys. -```bash -centaur-tools list - --help - health +## Common Patterns + +### No-auth tools (public APIs) +Skip `_get_api_key()` and auth headers. The `_client()` factory can be simpler: +```python +def _client() -> DefillLlamaClient: + return DefiLlamaClient() ``` -Use direct CLI commands for normal agent use. Use -`centaur-tools call ''` only for workflow-host -compatibility or for a method that intentionally has no standalone CLI command. +### Multi-secret tools +Some tools need multiple credentials: +```python +def _client() -> CoinbaseClient: + return CoinbaseClient( + api_key=secret("COINBASE_API_KEY"), + api_secret=secret("COINBASE_API_SECRET"), + passphrase=secret("COINBASE_API_PASSPHRASE"), + ) +``` -If the tool is missing, inspect: +### Secret cleaning +1Password sometimes returns multi-line blobs. If your API is sensitive to whitespace: +```python +def _clean_secret(value: str) -> str: + return value.strip().split("\n")[0].strip() +``` + +### POST/mutation methods +Name methods clearly (`create_`, `delete_`, `update_`). The tool-qa skill skips these during automated testing, but they're still registered for agent use. + +## Testing + +After creating the tool: + +1. **Verify registration**: restart the API (or hit `POST /admin/reload-tools`) and check `GET /tools` includes your tool +2. **Run tool-qa**: use the `tool-qa` skill to systematically test all methods +3. **Test via curl**: +```bash +source .env +curl -s "http://localhost:8000/tools/" \ + -H "Authorization: Bearer $API_SECRET_KEY" | jq + +curl -s -X POST "http://localhost:8000/tools//search" \ + -H "Authorization: Bearer $API_SECRET_KEY" \ + -H "Content-Type: application/json" \ + -d '{"query": "test", "limit": 3}' | jq +``` -- repo-cache source/ref and `TOOL_DIRS` -- the tool directory name -- `[tool.centaur] module = "client.py"` -- `[project.scripts]` -- sandbox shim install logs +## Deployment -Do not use `GET /tools`, `POST /admin/reload-tools`, or `/tools/...` curl calls -as the verification path for new tools. +Tools are **hot-reloaded** — no container restart needed. On merge to `main`: +1. CI runs `git pull` on the server +2. The API's file watcher detects changes in `tools/` +3. Tool is auto-reloaded within seconds +4. Fallback: `POST /admin/reload-tools` diff --git a/.agents/skills/qa/SKILL.md b/.agents/skills/qa/SKILL.md index 07f9ca186..57c9dac2c 100644 --- a/.agents/skills/qa/SKILL.md +++ b/.agents/skills/qa/SKILL.md @@ -44,7 +44,7 @@ echo "SLACK_THREAD_TS=${SLACK_THREAD_TS:-}" echo "SLACK_CHANNEL_NAME=${SLACK_CHANNEL_NAME:-}" ``` -If `SLACK_CHANNEL_ID` or `SLACK_THREAD_TS` is missing, infer it from `CENTAUR_THREAD_KEY` when possible. New Slack thread keys are `slack:::`; older persisted threads may still use `slack::`. +If `SLACK_CHANNEL_ID` or `SLACK_THREAD_TS` is missing, infer it from `CENTAUR_THREAD_KEY` when possible. Slack thread keys are usually `slack::`. ### 1. Tool Loading @@ -127,7 +127,7 @@ Pass when the command succeeds and returns valid search output. Empty results ar Find another accessible Slack file from the current channel. Prefer a result outside the current thread, but do not use files from other channels. `search_files` filters by filename or title, so start broad with an empty query: ```bash -centaur-tools call slack search_files '{"query":"", "max_results":5}' +centaur-tools call slack search_files '{"channel_id":"'"${SLACK_CHANNEL_ID}"'","query":"","max_results":5}' ``` Pick a result whose `channels` includes `${SLACK_CHANNEL_ID}`. If possible, avoid the file uploaded earlier in this QA run so the check proves download-and-reupload of an existing channel file. Download it: @@ -157,7 +157,7 @@ company_context list --limit 3 --json company_context search "centaur" --limit 3 --json ``` -Pass when the tool returns a valid JSON payload with `status: ok`, even if no documents match. Fail on database connection errors, permission errors, missing `CENTAUR_POSTGRES_DSN`, or malformed results. +Pass when the tool returns a valid JSON payload with `status: ok`, even if no documents match. Fail on database connection errors, permission errors, missing `COMPANY_CONTEXT_DSN`, or malformed results. If company context returns `upstream connection failed`, use runtime evidence before suggesting a code fix: @@ -299,7 +299,7 @@ responses; Slack does not render them reliably. Use this exact shape: ```text *Setup* -- *Thread context:* PASS - C123:1712345678.000000, key slack:T123:C123:... +- *Thread context:* PASS - C123:1712345678.000000, key slack:C123:... - *Tool loading:* PASS - 72 tools; expected slack/company_context/vlogs/vmetrics present *Slack* diff --git a/.agents/skills/qa/scripts/integration-slackbot.sh b/.agents/skills/qa/scripts/integration-slackbot.sh index 0d11bb4ac..799270ba7 100755 --- a/.agents/skills/qa/scripts/integration-slackbot.sh +++ b/.agents/skills/qa/scripts/integration-slackbot.sh @@ -99,16 +99,6 @@ urlencode() { jq -rn --arg value "$1" '$value|@uri' } -slack_thread_key() { - local channel="$1" - local thread_ts="$2" - if [[ -n "$SLACK_SMOKE_TEAM_ID" ]]; then - printf 'slack:%s:%s:%s\n' "$SLACK_SMOKE_TEAM_ID" "$channel" "$thread_ts" - else - printf 'slack:%s:%s\n' "$channel" "$thread_ts" - fi -} - api_request() { local method="$1" local url="$2" @@ -144,13 +134,7 @@ api_request() { slack_tool_json() { local method="$1" local body="${2:-{}}" - - if ! command -v centaur-tools >/dev/null 2>&1; then - echo " centaur-tools is required for Slack tool smoke checks." >&2 - return 1 - fi - - centaur-tools call slack "$method" "$body" | jq -c '.' + api_request POST "${API_URL}/tools/slack/${method}" "$body" | jq -c '.result' } resolve_smoke_channel() { @@ -344,29 +328,33 @@ wait_for_thread_reply() { return 1 } -wait_for_session_events() { +wait_for_execution_id() { local thread_key="$1" - local out_file="$2" local encoded_thread encoded_thread=$(urlencode "$thread_key") for _ in $(seq 1 "$SMOKE_POLL_ATTEMPTS"); do - : > "$out_file" - capture_execution_events "$thread_key" "$out_file" || return 1 - if [[ -s "$out_file" ]]; then + local executions_json + executions_json=$(api_request GET "${API_URL}/agent/threads/${encoded_thread}/executions?limit=1") || return 1 + + local execution_id + execution_id=$(jq -r '.executions[0].execution_id // empty' <<<"$executions_json") + if [[ -n "$execution_id" ]]; then + printf '%s\n' "$execution_id" return 0 fi sleep "$SMOKE_POLL_SLEEP_SECONDS" done - echo " timed out waiting for session events for ${thread_key}" + echo " timed out waiting for an execution for ${thread_key}" return 1 } capture_execution_events() { local thread_key="$1" - local out_file="$2" + local execution_id="$2" + local out_file="$3" local encoded_thread encoded_thread=$(urlencode "$thread_key") @@ -374,7 +362,7 @@ capture_execution_events() { set +e curl -sS -N --max-time "$EVENT_STREAM_TIMEOUT_SECONDS" \ "${AUTH_ARGS[@]}" \ - "${API_URL}/api/session/${encoded_thread}/events?poll_ms=1000" \ + "${API_URL}/agent/threads/${encoded_thread}/events?execution_id=${execution_id}&poll_ms=1000" \ > "$out_file" curl_status=$? set -e @@ -384,6 +372,10 @@ capture_execution_events() { echo " execution event stream failed with curl status ${curl_status}" return 1 fi + if [[ ! -s "$out_file" ]]; then + echo " execution event stream produced no events" + return 1 + fi } sse_data_json() { @@ -422,10 +414,13 @@ assert_execution_upload_stream() { local thread_key="$1" local require_permalink="${2:-1}" + local execution_id + execution_id=$(wait_for_execution_id "$thread_key") || return 1 + local events_file events_file=$(mktemp) - wait_for_session_events "$thread_key" "$events_file" || { - echo " failed to capture session events for ${thread_key}" + capture_execution_events "$thread_key" "$execution_id" "$events_file" || { + echo " failed to capture execution events for ${execution_id}" rm -f "$events_file" return 1 } @@ -569,8 +564,7 @@ smoke_chart_case() { local thread_ts channel=$(jq -r '.channel' <<<"$seed_json") thread_ts=$(jq -r '.ts' <<<"$seed_json") - local thread_key - thread_key=$(slack_thread_key "$channel" "$thread_ts") + local thread_key="${channel}:${thread_ts}" local chart_body chart_body=$(build_event_body \ @@ -596,8 +590,7 @@ smoke_generated_media_case() { local thread_ts channel=$(jq -r '.channel' <<<"$seed_json") thread_ts=$(jq -r '.ts' <<<"$seed_json") - local thread_key - thread_key=$(slack_thread_key "$channel" "$thread_ts") + local thread_key="${channel}:${thread_ts}" local media_body media_body=$(build_event_body \ diff --git a/.github/landing-previews/console-integrations/integrations-dark.png b/.github/landing-previews/console-integrations/integrations-dark.png new file mode 100644 index 000000000..e3da7ae4b Binary files /dev/null and b/.github/landing-previews/console-integrations/integrations-dark.png differ diff --git a/.github/landing-previews/console-integrations/integrations-light.png b/.github/landing-previews/console-integrations/integrations-light.png new file mode 100644 index 000000000..9b65f4048 Binary files /dev/null and b/.github/landing-previews/console-integrations/integrations-light.png differ diff --git a/.github/scripts/audit_workflow_trust.py b/.github/scripts/audit_workflow_trust.py new file mode 100644 index 000000000..8c539d7c2 --- /dev/null +++ b/.github/scripts/audit_workflow_trust.py @@ -0,0 +1,151 @@ +#!/usr/bin/env python3 +"""Fail closed when PR validation regains publication credentials or secrets.""" + +from __future__ import annotations + +from pathlib import Path +import re + + +ROOT = Path(__file__).resolve().parents[2] +WORKFLOWS = ROOT / ".github" / "workflows" + + +def read(name: str) -> str: + return (WORKFLOWS / name).read_text() + + +def require(text: str, needle: str, source: str) -> None: + if needle not in text: + raise SystemExit(f"{source}: missing required trust-boundary marker: {needle}") + + +def reject(text: str, needle: str, source: str) -> None: + if needle in text: + raise SystemExit(f"{source}: forbidden in this trust lane: {needle}") + + +def require_count(text: str, needle: str, minimum: int, source: str) -> None: + count = text.count(needle) + if count < minimum: + raise SystemExit( + f"{source}: expected at least {minimum} occurrences of {needle}, found {count}" + ) + + +def audit_pull_request_workflows() -> None: + label_gated_secret_workflow = "pr-audit.yml" + for path in sorted(WORKFLOWS.glob("*.yml")): + text = path.read_text() + if not re.search(r"^\s{2}pull_request:\s*$", text, re.MULTILINE): + continue + if path.name == label_gated_secret_workflow: + require(text, "types: [labeled]", path.name) + require(text, "github.event.label.name == 'cyclops'", path.name) + reject(text, "actions/checkout", path.name) + continue + reject(text, "${{ secrets.", path.name) + for permission in ("contents", "packages", "deployments", "actions"): + if re.search(rf"^\s+{permission}:\s+write\s*$", text, re.MULTILINE): + raise SystemExit(f"{path.name}: PR workflow grants {permission}: write") + + +def audit_split_publication_lanes() -> None: + image_validation = read("validate-images.yml") + require(image_validation, "pull_request:", "validate-images.yml") + require(image_validation, "push: false", "validate-images.yml") + require(image_validation, "name: Image validation success", "validate-images.yml") + for helper in ( + ".github/scripts/resolve-runnable-image-digest.sh", + ".github/scripts/verify-registry-tag-absent.sh", + ".github/scripts/verify-reviewed-image-release.sh", + ): + require(image_validation, helper, "validate-images.yml") + reject(image_validation, "docker/login-action", "validate-images.yml") + reject(image_validation, "packages: write", "validate-images.yml") + + image_publish = read("publish-images.yml") + reject(image_publish, "pull_request:", "publish-images.yml") + require(image_publish, "'reviewed-images-publish-*'", "publish-images.yml") + require(image_publish, "group: publish-reviewed-centaur-images", "publish-images.yml") + require(image_publish, "cancel-in-progress: false", "publish-images.yml") + require(image_publish, "checks: read", "publish-images.yml") + require( + image_publish, + "verify-reviewed-image-release.sh", + "publish-images.yml", + ) + require( + image_publish, + "verify-registry-tag-absent.sh", + "publish-images.yml", + ) + require_count( + image_publish, + "verify-registry-tag-absent.sh", + 2, + "publish-images.yml", + ) + require(image_publish, "REVIEWED_TAG: reviewed-${{ github.sha }}", "publish-images.yml") + require(image_publish, 'tag="reviewed-${RELEASE_REVISION}"', "publish-images.yml") + require(image_publish, 'needs: tag-absence-gate', "publish-images.yml") + require( + image_publish, + 'if [[ "${#root_digest_files[@]}" -ne 2 ]]', + "publish-images.yml", + ) + require( + image_publish, + "pattern: runnable-child-digests-*-linux-arm64", + "publish-images.yml", + ) + require( + image_publish, + 'if [[ "$digest" != "$run_child_digest" ]]', + "publish-images.yml", + ) + reject(image_publish, 'tag="sha-${RELEASE_REVISION', "publish-images.yml") + reject(image_publish, "type=semver", "publish-images.yml") + reject(image_publish, "promote-fineas-infra", "publish-images.yml") + + for name in ("docs-deploy.yml", "release-chart-publish.yml"): + text = read(name) + reject(text, "pull_request:", name) + reject(text, "push:", name) + require(text, "workflow_dispatch:", name) + require(text, "confirm_reviewed_main", name) + require(text, "github.ref == 'refs/heads/main'", name) + + for name in ("docs.yml", "release-chart.yml"): + text = read(name) + require(text, "pull_request:", name) + reject(text, "${{ secrets.", name) + reject(text, "contents: write", name) + + +def audit_upstream_import_lane() -> None: + audit = read("upstream-sync.yml") + require(audit, 'UPSTREAM_OWNER: paradigmxyz', "upstream-sync.yml") + require(audit, '"${UPSTREAM_OWNER}:${UPSTREAM_BRANCH}"', "upstream-sync.yml") + require(audit, 'repos/${UPSTREAM_REPO}/commits/${commit}', "upstream-sync.yml") + require(audit, "draft: true", "upstream-sync.yml") + reject(audit, "SYNC_BRANCH", "upstream-sync.yml") + reject(audit, "git push", "upstream-sync.yml") + reject(audit, "permission-contents: write", "upstream-sync.yml") + + verifier = read("upstream-pr-verify.yml") + require(verifier, "moving upstream head", "upstream-pr-verify.yml") + require(verifier, "rev-list --reverse", "upstream-pr-verify.yml") + require(verifier, "repos/paradigmxyz/centaur/commits/${commit}", "upstream-pr-verify.yml") + reject(verifier, "${{ secrets.", "upstream-pr-verify.yml") + + +def main() -> None: + audit_pull_request_workflows() + audit_split_publication_lanes() + audit_upstream_import_lane() + print("workflow trust-boundary audit passed") + + +if __name__ == "__main__": + main() diff --git a/.github/scripts/check-migration-order.sh b/.github/scripts/check-migration-order.sh index 38bf2ca81..2ce6b4667 100755 --- a/.github/scripts/check-migration-order.sh +++ b/.github/scripts/check-migration-order.sh @@ -1,5 +1,6 @@ #!/usr/bin/env bash set -euo pipefail +IFS=$'\n\t' base_ref="${1:-origin/${GITHUB_BASE_REF:-main}}" failed=0 @@ -24,11 +25,75 @@ version_number() { echo $((10#${version})) } +check_checksum_manifest() { + local label="$1" + local dir="$2" + local extension="$3" + local algorithm="$4" + local manifest="$5" + local manifest_failed=0 + + if [[ ! -f "${manifest}" ]]; then + failed=1 + echo "::error title=${label} checksum manifest missing::Expected ${manifest}" + return + fi + + if ! shasum -a "${algorithm}" --check "${manifest}"; then + failed=1 + manifest_failed=1 + echo "::error title=${label} checksum mismatch::Release migrations must match ${manifest}" + fi + + local duplicate_paths + duplicate_paths="$(awk 'NF { print $2 }' "${manifest}" | sort | uniq -d)" + if [[ -n "${duplicate_paths}" ]]; then + failed=1 + manifest_failed=1 + echo "::error title=${label} duplicate checksum entries::Each migration path must appear once in ${manifest}" + printf '%s\n' "${duplicate_paths}" + fi + + local unlisted_paths + unlisted_paths="$({ + comm -23 \ + <(find "${dir}" -maxdepth 1 -type f -name "*.${extension}" -print | sort) \ + <(awk 'NF { print $2 }' "${manifest}" | sort) + })" + if [[ -n "${unlisted_paths}" ]]; then + failed=1 + manifest_failed=1 + echo "::error title=${label} migration missing checksum::Append checksums for every new migration to ${manifest}" + printf '%s\n' "${unlisted_paths}" + fi + + if git cat-file -e "${base_ref}:${manifest}" 2>/dev/null; then + while IFS= read -r applied_entry; do + [[ -n "${applied_entry}" ]] || continue + if ! grep -Fqx -- "${applied_entry}" "${manifest}"; then + failed=1 + manifest_failed=1 + echo "::error title=${label} applied migration changed::Base checksum entry must remain byte-for-byte: ${applied_entry}" + fi + done < <(git show "${base_ref}:${manifest}") + fi + + if [[ "${manifest_failed}" -eq 0 ]]; then + echo "${label}: migration checksums match the immutable release manifest." + fi +} + check_migrations() { local label="$1" local dir="$2" local regex="$3" + local manifest="$4" local dir_failed=0 + local bootstrapping_manifest=0 + + if ! git cat-file -e "${base_ref}:${manifest}" 2>/dev/null; then + bootstrapping_manifest=1 + fi local head_entries head_entries="$( @@ -75,7 +140,8 @@ check_migrations() { while IFS= read -r version; do [[ -n "${version}" ]] || continue - if (( $(version_number "${version}") <= $(version_number "${base_max}") )); then + if (( $(version_number "${version}") <= $(version_number "${base_max}") )) \ + && [[ "${bootstrapping_manifest}" -eq 0 ]]; then failed=1 dir_failed=1 echo "::error title=${label} non-monotonic migration::New migration version ${version} must be greater than ${base_max} from ${base_ref}" @@ -84,18 +150,38 @@ check_migrations() { done <<<"${added_versions}" if [[ "${dir_failed}" -eq 0 ]]; then - echo "${label}: migration versions are monotonic relative to ${base_ref}." + if [[ "${bootstrapping_manifest}" -eq 1 ]]; then + echo "${label}: bootstrapping the immutable migration lineage manifest." + else + echo "${label}: migration versions are monotonic relative to ${base_ref}." + fi fi } check_migrations \ "SQLx" \ "services/api-rs/crates/centaur-session-sqlx/migrations" \ - '^([0-9]+)_.+\.sql$' + '^([0-9]+)_.+\.sql$' \ + "services/api-rs/crates/centaur-session-sqlx/migrations/.checksums.sha384" check_migrations \ "Rails console" \ "services/console/db/migrate" \ - '^([0-9]+)_.+\.rb$' + '^([0-9]+)_.+\.rb$' \ + "services/console/db/migrate/.checksums.sha256" + +check_checksum_manifest \ + "SQLx" \ + "services/api-rs/crates/centaur-session-sqlx/migrations" \ + "sql" \ + "384" \ + "services/api-rs/crates/centaur-session-sqlx/migrations/.checksums.sha384" + +check_checksum_manifest \ + "Rails console" \ + "services/console/db/migrate" \ + "rb" \ + "256" \ + "services/console/db/migrate/.checksums.sha256" exit "${failed}" diff --git a/.github/scripts/promote_fineas_infra.py b/.github/scripts/promote_fineas_infra.py new file mode 100644 index 000000000..b5daa5acf --- /dev/null +++ b/.github/scripts/promote_fineas_infra.py @@ -0,0 +1,232 @@ +from __future__ import annotations + +import base64 +import json +import os +import re +import subprocess +import urllib.error +import urllib.parse +import urllib.request +from pathlib import Path + + +OWNER = "TipLink" +REPO = "fineas-centaur-infra" +BASE_BRANCH = "main" +ROOT = Path(os.environ.get("FINEAS_INFRA_ROOT", "fineas-centaur-infra")) +CENTAUR_SHA = os.environ["CENTAUR_SHA"] +CENTAUR_SHORT = CENTAUR_SHA[:7] +BRANCH = f"automation/promote-centaur-{CENTAUR_SHORT}" +MESSAGE = f"chore: stage centaur {CENTAUR_SHORT} console migration" +API = f"https://api.github.com/repos/{OWNER}/{REPO}" + +HEADERS = { + "Authorization": f"Bearer {os.environ['FINEAS_INFRA_TOKEN']}", + "Accept": "application/vnd.github+json", + "X-GitHub-Api-Version": "2022-11-28", + "User-Agent": "centaur-promote-fineas", +} +CENTAUR_HEADERS = { + **HEADERS, + "Authorization": f"Bearer {os.environ['CENTAUR_TOKEN']}", +} + + +def request( + method: str, + url: str, + payload: dict | None = None, + *, + ok: tuple[int, ...] = (200, 201, 204), + headers: dict[str, str] | None = None, +): + data = None + request_headers = dict(headers or HEADERS) + if payload is not None: + data = json.dumps(payload).encode() + request_headers["Content-Type"] = "application/json" + req = urllib.request.Request(url, data=data, headers=request_headers, method=method) + try: + with urllib.request.urlopen(req) as response: + body = response.read().decode() + if response.status not in ok: + raise RuntimeError(f"{method} {url} returned {response.status}: {body}") + return json.loads(body) if body else None + except urllib.error.HTTPError as error: + body = error.read().decode() + if error.code in ok: + if error.code == 404: + return None + return json.loads(body) if body else None + raise RuntimeError(f"{method} {url} returned {error.code}: {body}") from None + + +def git(*args: str) -> str: + return subprocess.check_output(["git", "-C", str(ROOT), *args], text=True) + + +def centaur_pr_url_for_commit() -> str: + repo = os.environ["CENTAUR_REPO"] + url = f"https://api.github.com/repos/{repo}/commits/{CENTAUR_SHA}/pulls" + try: + pulls = request("GET", url, headers=CENTAUR_HEADERS) + except RuntimeError as error: + print(f"warning: could not look up Centaur PR for {CENTAUR_SHA}: {error}") + return "" + merged = [pull for pull in pulls if pull.get("merged_at")] + pull = (merged or pulls or [None])[0] + return pull["html_url"] if pull else "" + + +def changed_paths() -> list[tuple[str, str]]: + changed: list[tuple[str, str]] = [] + for line in git("status", "--porcelain").splitlines(): + if not line: + continue + status = line[:2] + path = line[3:] + if " -> " in path: + path = path.split(" -> ", 1)[1] + kind = "A" if status == "??" else "D" if "D" in status else "M" + changed.append((kind, path)) + return changed + + +def ensure_branch() -> str: + encoded = urllib.parse.quote(BRANCH, safe="") + existing = request("GET", f"{API}/git/ref/heads/{encoded}", ok=(200, 404)) + if not existing: + base_ref = request("GET", f"{API}/git/ref/heads/{BASE_BRANCH}") + request( + "POST", + f"{API}/git/refs", + {"ref": f"refs/heads/{BRANCH}", "sha": base_ref["object"]["sha"]}, + ) + return encoded + + +def put_file(rel: str, encoded_branch: str) -> bool: + encoded_path = urllib.parse.quote(rel, safe="") + url = f"{API}/contents/{encoded_path}" + existing = request("GET", f"{url}?ref={encoded_branch}", ok=(200, 404)) + content = (ROOT / rel).read_bytes() + if existing: + current = base64.b64decode(existing["content"]).replace(b"\r\n", b"\n") + if current == content: + return False + payload = { + "message": MESSAGE, + "content": base64.b64encode(content).decode(), + "branch": BRANCH, + } + if existing: + payload["sha"] = existing["sha"] + request("PUT", url, payload) + return True + + +def delete_file(rel: str, encoded_branch: str) -> bool: + encoded_path = urllib.parse.quote(rel, safe="") + url = f"{API}/contents/{encoded_path}" + existing = request("GET", f"{url}?ref={encoded_branch}", ok=(200, 404)) + if not existing: + return False + request( + "DELETE", + url, + {"message": MESSAGE, "branch": BRANCH, "sha": existing["sha"]}, + ) + return True + + +def set_application_annotation(key: str, value: str) -> bool: + path = ROOT / "clusters/centaur-sandbox/argocd/applications/centaur-sandbox.yaml" + text = path.read_text(encoding="utf-8") + pattern = re.compile(rf"^ {re.escape(key)}: .*$", flags=re.MULTILINE) + if value: + line = f" {key}: {json.dumps(value)}" + if pattern.search(text): + updated = pattern.sub(line, text, count=1) + else: + marker = " argocd.argoproj.io/compare-options: ServerSideDiff=true\n" + if marker not in text: + raise RuntimeError(f"could not place Application annotation {key}") + updated = text.replace(marker, f"{marker}{line}\n", 1) + else: + updated = pattern.sub("", text) + updated = re.sub(r"\n{3,}", "\n\n", updated) + if updated == text: + return False + path.write_text(updated, encoding="utf-8") + return True + + +def pull_request_body(centaur_pr_url: str) -> str: + source = [ + f"- Centaur commit: https://github.com/{os.environ['CENTAUR_REPO']}/commit/{CENTAUR_SHA}", + f"- Image publish run: {os.environ['CENTAUR_RUN_URL']}", + ] + if centaur_pr_url: + source.append(f"- Centaur PR: {centaur_pr_url}") + return f"""## Summary +- stage the Fineas Console image and Centaur chart at `sha-{CENTAUR_SHORT}` +- keep api-rs, Slackbot, sandbox, and proxy runtime images on their previous pins +- keep `apiRs.runMigrations=false`; this is the Console-only migration stage + +## Source +{chr(10).join(source)} + +## Tests +- `bash scripts/bump-centaur-pins.sh --stage console {CENTAUR_SHA}` +- `scripts/audit-supply-chain.sh` +""" + + +def main() -> None: + paths = changed_paths() + if not paths: + print("Fineas infra pins already match; no promotion PR needed") + return + + encoded_branch = ensure_branch() + changed = False + for kind, rel in paths: + changed |= delete_file(rel, encoded_branch) if kind == "D" else put_file(rel, encoded_branch) + + centaur_pr_url = centaur_pr_url_for_commit() + title = f"Stage Centaur {CENTAUR_SHORT} Console migration in Fineas" + body = pull_request_body(centaur_pr_url) + head = urllib.parse.quote(f"{OWNER}:{BRANCH}", safe="") + pulls = request("GET", f"{API}/pulls?state=open&head={head}") + if pulls: + pull = request("PATCH", f"{API}/pulls/{pulls[0]['number']}", {"title": title, "body": body}) + elif changed: + pull = request( + "POST", + f"{API}/pulls", + {"title": title, "head": BRANCH, "base": BASE_BRANCH, "body": body}, + ) + else: + pull = None + + if not pull: + print("Promotion branch already matches and no open PR was found") + return + + annotations_changed = set_application_annotation( + "fineas.dev/deployment-pr-url", pull["html_url"] + ) + annotations_changed |= set_application_annotation( + "fineas.dev/centaur-pr-url", centaur_pr_url + ) + if annotations_changed: + put_file( + "clusters/centaur-sandbox/argocd/applications/centaur-sandbox.yaml", + encoded_branch, + ) + print(pull["html_url"]) + + +if __name__ == "__main__": + main() diff --git a/.github/scripts/resolve-runnable-image-digest.sh b/.github/scripts/resolve-runnable-image-digest.sh new file mode 100644 index 000000000..660288cde --- /dev/null +++ b/.github/scripts/resolve-runnable-image-digest.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +image_index_ref=${1:-} +platform_os=${2:-} +platform_architecture=${3:-} + +if [[ ! "$image_index_ref" =~ @sha256:[0-9a-f]{64}$ ]]; then + echo "image index reference must end in a full sha256 digest" >&2 + exit 1 +fi +if [[ ! "$platform_os" =~ ^[a-z0-9]+$ || + ! "$platform_architecture" =~ ^[a-z0-9_]+$ ]]; then + echo "platform OS and architecture must be non-empty lowercase identifiers" >&2 + exit 1 +fi + +index_json="$(docker buildx imagetools inspect "$image_index_ref" --raw)" +if ! jq -e ' + type == "object" and + ( + .mediaType == "application/vnd.oci.image.index.v1+json" or + .mediaType == "application/vnd.docker.distribution.manifest.list.v2+json" + ) and + (.manifests | type == "array") +' <<<"$index_json" >/dev/null; then + echo "build output is not a parseable OCI/Docker image index: $image_index_ref" >&2 + exit 1 +fi + +runnable_digests=() +while IFS= read -r digest; do + runnable_digests+=("$digest") +done < <( + jq -r \ + --arg os "$platform_os" \ + --arg architecture "$platform_architecture" ' + .manifests[] + | select( + .platform.os == $os and + .platform.architecture == $architecture and + ( + .mediaType == "application/vnd.oci.image.manifest.v1+json" or + .mediaType == "application/vnd.docker.distribution.manifest.v2+json" + ) + ) + | .digest + ' <<<"$index_json" +) + +if [[ "${#runnable_digests[@]}" -ne 1 ]]; then + echo "expected exactly one runnable ${platform_os}/${platform_architecture} child in $image_index_ref; found ${#runnable_digests[@]}" >&2 + exit 1 +fi + +runnable_digest=${runnable_digests[0]} +if [[ ! "$runnable_digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "index contains an invalid runnable child digest: $runnable_digest" >&2 + exit 1 +fi + +repository=${image_index_ref%@*} +docker buildx imagetools inspect "${repository}@${runnable_digest}" --raw >/dev/null + +printf '%s\n' "$runnable_digest" diff --git a/.github/scripts/test-resolve-runnable-image-digest.sh b/.github/scripts/test-resolve-runnable-image-digest.sh new file mode 100644 index 000000000..a0adf2f08 --- /dev/null +++ b/.github/scripts/test-resolve-runnable-image-digest.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +resolver=.github/scripts/resolve-runnable-image-digest.sh +scratch="$(mktemp -d -t runnable-image-digest.XXXXXXXXXX)" +trap 'rm -rf "$scratch"' EXIT +mkdir -p "$scratch/bin" + +cat >"$scratch/bin/docker" <<'MOCK_DOCKER' +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +if [[ "$#" -ne 5 || "$1" != "buildx" || "$2" != "imagetools" || + "$3" != "inspect" || "$5" != "--raw" ]]; then + echo "unexpected mocked docker invocation: $*" >&2 + exit 97 +fi + +case "$4" in + "$MOCK_INDEX_REF") printf '%s\n' "$MOCK_INDEX_JSON" ;; + "$MOCK_CHILD_REF") + if [[ "$MOCK_CHILD_PULLABLE" != "true" ]]; then + exit 98 + fi + printf '{"mediaType":"application/vnd.oci.image.manifest.v1+json"}\n' + ;; + *) + echo "unexpected mocked image reference: $4" >&2 + exit 99 + ;; +esac +MOCK_DOCKER +chmod +x "$scratch/bin/docker" + +digest_arm64="sha256:$(printf 'a%.0s' {1..64})" +digest_amd64="sha256:$(printf 'b%.0s' {1..64})" +digest_attestation="sha256:$(printf 'c%.0s' {1..64})" +MOCK_INDEX_REF="ghcr.io/tiplink/centaur/example@sha256:$(printf 'd%.0s' {1..64})" +export MOCK_INDEX_REF +export MOCK_CHILD_REF="ghcr.io/tiplink/centaur/example@${digest_arm64}" +export MOCK_CHILD_PULLABLE=true +export PATH="$scratch/bin:$PATH" + +make_index() { + jq -cn \ + --arg arm64 "$digest_arm64" \ + --arg amd64 "$digest_amd64" \ + --arg attestation "$digest_attestation" '{ + mediaType: "application/vnd.oci.image.index.v1+json", + manifests: [ + { + mediaType: "application/vnd.oci.image.manifest.v1+json", + digest: $arm64, + platform: {os: "linux", architecture: "arm64"} + }, + { + mediaType: "application/vnd.oci.image.manifest.v1+json", + digest: $amd64, + platform: {os: "linux", architecture: "amd64"} + }, + { + mediaType: "application/vnd.oci.image.manifest.v1+json", + digest: $attestation, + platform: {os: "unknown", architecture: "unknown"}, + annotations: {"vnd.docker.reference.type": "attestation-manifest"} + } + ] + }' +} + +expect_reject() { + local label=$1 + shift + if "$@" >"$scratch/${label}.out" 2>&1; then + echo "runnable digest resolver unexpectedly accepted: $label" >&2 + exit 1 + fi +} + +MOCK_INDEX_JSON="$(make_index)" +export MOCK_INDEX_JSON +actual="$(bash "$resolver" "$MOCK_INDEX_REF" linux arm64)" +[[ "$actual" == "$digest_arm64" ]] || { + echo "resolver returned $actual instead of $digest_arm64" >&2 + exit 1 +} + +MOCK_INDEX_JSON="$(jq --argjson duplicate "$(jq -c '.manifests[0]' <<<"$MOCK_INDEX_JSON")" \ + '.manifests += [$duplicate]' <<<"$MOCK_INDEX_JSON")" +export MOCK_INDEX_JSON +expect_reject duplicate-platform-child bash "$resolver" "$MOCK_INDEX_REF" linux arm64 + +MOCK_INDEX_JSON="$(jq '.manifests |= map(select(.platform.architecture != "arm64"))' <<<"$(make_index)")" +export MOCK_INDEX_JSON +expect_reject missing-platform-child bash "$resolver" "$MOCK_INDEX_REF" linux arm64 + +MOCK_INDEX_JSON="$(jq '.manifests[0].digest = "sha256:not-a-digest"' <<<"$(make_index)")" +export MOCK_INDEX_JSON +expect_reject invalid-child-digest bash "$resolver" "$MOCK_INDEX_REF" linux arm64 + +MOCK_INDEX_JSON="$(jq -cn '{mediaType:"application/vnd.oci.image.manifest.v1+json"}')" +export MOCK_INDEX_JSON +expect_reject direct-manifest bash "$resolver" "$MOCK_INDEX_REF" linux arm64 + +MOCK_INDEX_JSON="$(make_index)" +export MOCK_INDEX_JSON +export MOCK_CHILD_PULLABLE=false +expect_reject inaccessible-child bash "$resolver" "$MOCK_INDEX_REF" linux arm64 +export MOCK_CHILD_PULLABLE=true + +expect_reject unpinned-index bash "$resolver" ghcr.io/tiplink/centaur/example:latest linux arm64 +expect_reject invalid-platform bash "$resolver" "$MOCK_INDEX_REF" 'Linux!' arm64 + +echo "runnable image digest resolver tests passed" diff --git a/.github/scripts/test-verify-registry-tag-absent.sh b/.github/scripts/test-verify-registry-tag-absent.sh new file mode 100644 index 000000000..c027926b4 --- /dev/null +++ b/.github/scripts/test-verify-registry-tag-absent.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +verifier=.github/scripts/verify-registry-tag-absent.sh +scratch="$(mktemp -d -t registry-tag-absent.XXXXXXXXXX)" +trap 'rm -rf "$scratch"' EXIT +mkdir -p "$scratch/bin" + +cat >"$scratch/bin/curl" <<'MOCK_CURL' +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +url=${!#} +if [[ "$url" == "https://ghcr.io/token" ]]; then + printf '{"token":"mock-registry-token"}\n' + exit 0 +fi +if [[ "$url" != "https://ghcr.io/v2/tiplink/centaur/example/manifests/${MOCK_TAG:?}" ]]; then + echo "unexpected mocked curl URL: $url" >&2 + exit 97 +fi + +output='' +previous='' +for argument in "$@"; do + if [[ "$previous" == "--output" ]]; then + output="$argument" + break + fi + previous="$argument" +done +if [[ -z "$output" ]]; then + echo "mocked manifest request omitted --output" >&2 + exit 98 +fi + +case "${MOCK_STATUS:?}" in + 404-known) + printf '{"errors":[{"code":"MANIFEST_UNKNOWN"}]}\n' >"$output" + printf '404' + ;; + 404-unknown) + printf '{"errors":[{"code":"DENIED"}]}\n' >"$output" + printf '404' + ;; + 200) + printf '{}\n' >"$output" + printf '200' + ;; + 403) + printf '{}\n' >"$output" + printf '403' + ;; + *) exit 99 ;; +esac +MOCK_CURL +chmod +x "$scratch/bin/curl" + +export PATH="$scratch/bin:$PATH" +export GITHUB_ACTOR=fineas-bot +export GHCR_TOKEN=not-a-real-token +MOCK_TAG="reviewed-$(printf 'a%.0s' {1..40})" +export MOCK_TAG + +expect_reject() { + local label=$1 + shift + if "$@" >"$scratch/${label}.out" 2>&1; then + echo "registry absence verifier unexpectedly accepted: $label" >&2 + exit 1 + fi +} + +export MOCK_STATUS=404-known +bash "$verifier" tiplink/centaur/example "$MOCK_TAG" >"$scratch/safe.out" +grep -qF 'OK reviewed tag is absent' "$scratch/safe.out" + +export MOCK_STATUS=200 +expect_reject existing-tag bash "$verifier" tiplink/centaur/example "$MOCK_TAG" +grep -qF 'refusing to overwrite immutable reviewed tag' "$scratch/existing-tag.out" + +export MOCK_STATUS=404-unknown +expect_reject ambiguous-404 bash "$verifier" tiplink/centaur/example "$MOCK_TAG" + +export MOCK_STATUS=403 +expect_reject forbidden bash "$verifier" tiplink/centaur/example "$MOCK_TAG" + +expect_reject shortened-tag bash "$verifier" tiplink/centaur/example reviewed-aaaaaaa + +echo "registry reviewed-tag absence tests passed" diff --git a/.github/scripts/test-verify-reviewed-image-release.sh b/.github/scripts/test-verify-reviewed-image-release.sh new file mode 100644 index 000000000..400092f58 --- /dev/null +++ b/.github/scripts/test-verify-reviewed-image-release.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +verifier=.github/scripts/verify-reviewed-image-release.sh +scratch="$(mktemp -d -t reviewed-image-release.XXXXXXXXXX)" +trap 'rm -rf "$scratch"' EXIT +mkdir -p "$scratch/bin" + +MOCK_SHA="$(printf 'a%.0s' {1..40})" +export MOCK_SHA +export MOCK_MODE=valid +export MOCK_TAG_NAME='' + +cat >"$scratch/bin/git" <<'MOCK_GIT' +#!/usr/bin/env bash +set -euo pipefail +if [[ "$*" == "rev-parse HEAD" ]]; then + printf '%s\n' "${MOCK_SHA:?}" +else + echo "unexpected mocked git invocation: $*" >&2 + exit 97 +fi +MOCK_GIT + +cat >"$scratch/bin/curl" <<'MOCK_CURL' +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' +url=${!#} +base='https://api.github.test/repos/TipLink/centaur' + +case "$url" in + "$base/git/ref/tags/"*) + jq -cn --arg ref "refs/tags/${MOCK_TAG_NAME:?}" --arg sha "${MOCK_SHA:?}" \ + '{ref:$ref,object:{type:"commit",sha:$sha}}' + ;; + "$base/git/commits/${MOCK_SHA}") + if [[ "${MOCK_MODE:?}" == "bad-signature" ]]; then + jq -cn --arg sha "$MOCK_SHA" '{sha:$sha,verification:{verified:false,reason:"unsigned"}}' + else + jq -cn --arg sha "$MOCK_SHA" '{sha:$sha,verification:{verified:true,reason:"valid"}}' + fi + ;; + "$base/commits/${MOCK_SHA}/pulls") + draft=false + if [[ "$MOCK_MODE" == "draft" ]]; then draft=true; fi + head_repo='TipLink/centaur' + if [[ "$MOCK_MODE" == "fork-pr" ]]; then head_repo='untrusted/centaur'; fi + jq -cn --arg sha "$MOCK_SHA" --argjson draft "$draft" --arg head_repo "$head_repo" \ + '[{number:76,state:"open",merged_at:null,draft:$draft,base:{ref:"main",repo:{full_name:"TipLink/centaur"}},head:{sha:$sha,repo:{full_name:$head_repo}}}]' + ;; + "$base/commits/${MOCK_SHA}/check-runs?filter=latest&per_page=100") + ci_conclusion=success + if [[ "$MOCK_MODE" == "failed-check" ]]; then ci_conclusion=failure; fi + jq -cn --arg sha "$MOCK_SHA" --arg ci "$ci_conclusion" '{check_runs:[ + {name:"CI success",head_sha:$sha,status:"completed",conclusion:$ci,completed_at:"2026-07-12T00:00:03Z",details_url:"https://github.com/TipLink/centaur/actions/runs/11/job/101",app:{slug:"github-actions"}}, + {name:"Console CI success",head_sha:$sha,status:"completed",conclusion:"success",completed_at:"2026-07-12T00:00:02Z",details_url:"https://github.com/TipLink/centaur/actions/runs/12/job/102",app:{slug:"github-actions"}}, + {name:"Image validation success",head_sha:$sha,status:"completed",conclusion:"success",completed_at:"2026-07-12T00:00:01Z",details_url:"https://github.com/TipLink/centaur/actions/runs/13/job/103",app:{slug:"github-actions"}} + ]}' + ;; + "$base/actions/runs/11") + pr=76 + if [[ "$MOCK_MODE" == "wrong-pr-run" ]]; then pr=75; fi + jq -cn --arg sha "$MOCK_SHA" --argjson pr "$pr" '{head_sha:$sha,event:"pull_request",status:"completed",conclusion:"success",path:".github/workflows/ci.yml",pull_requests:[{number:$pr,head:{sha:$sha}}]}' + ;; + "$base/actions/runs/12") + path='.github/workflows/console-ci.yml' + if [[ "$MOCK_MODE" == "wrong-workflow" ]]; then path='.github/workflows/codeql.yml'; fi + jq -cn --arg sha "$MOCK_SHA" --arg path "$path" '{head_sha:$sha,event:"pull_request",status:"completed",conclusion:"success",path:$path,pull_requests:[{number:76,head:{sha:$sha}}]}' + ;; + "$base/actions/runs/13") + jq -cn --arg sha "$MOCK_SHA" '{head_sha:$sha,event:"pull_request",status:"completed",conclusion:"success",path:".github/workflows/validate-images.yml",pull_requests:[{number:76,head:{sha:$sha}}]}' + ;; + *) + echo "unexpected mocked curl URL: $url" >&2 + exit 98 + ;; +esac +MOCK_CURL +chmod +x "$scratch/bin/git" "$scratch/bin/curl" + +export PATH="$scratch/bin:$PATH" +export GITHUB_API_TOKEN=not-a-real-token +export TRIGGER_API_URL=https://api.github.test +export TRIGGER_EVENT_NAME=workflow_dispatch +export TRIGGER_REF=refs/heads/reviewed +export TRIGGER_REF_NAME=reviewed +export TRIGGER_REF_TYPE=branch +export TRIGGER_REPOSITORY=TipLink/centaur +export TRIGGER_SHA="$MOCK_SHA" +export DISPATCH_REVIEWED_COMMIT="$MOCK_SHA" + +expect_reject() { + local label=$1 + shift + if "$@" >"$scratch/${label}.out" 2>&1; then + echo "reviewed release verifier unexpectedly accepted: $label" >&2 + exit 1 + fi +} + +bash "$verifier" >"$scratch/valid.out" +grep -qF 'OK reviewed signed PR #76' "$scratch/valid.out" + +export MOCK_MODE=bad-signature +expect_reject bad-signature bash "$verifier" +export MOCK_MODE=draft +expect_reject draft-pr bash "$verifier" +export MOCK_MODE=failed-check +expect_reject failed-check bash "$verifier" +export MOCK_MODE=wrong-workflow +expect_reject wrong-workflow bash "$verifier" +export MOCK_MODE=wrong-pr-run +expect_reject wrong-pr-run bash "$verifier" +export MOCK_MODE=fork-pr +expect_reject fork-pr bash "$verifier" + +export MOCK_MODE=valid +DISPATCH_REVIEWED_COMMIT="$(printf 'b%.0s' {1..40})" +export DISPATCH_REVIEWED_COMMIT +expect_reject mismatched-dispatch-commit bash "$verifier" + +export DISPATCH_REVIEWED_COMMIT="$MOCK_SHA" +export TRIGGER_EVENT_NAME=push +export TRIGGER_REF_TYPE=tag +export TRIGGER_REF_CREATED=true +attested_at="$(date +%s)" +export MOCK_TAG_NAME="reviewed-images-publish-${MOCK_SHA}-at-${attested_at}" +export TRIGGER_REF_NAME="$MOCK_TAG_NAME" +export TRIGGER_REF="refs/tags/${MOCK_TAG_NAME}" +bash "$verifier" >"$scratch/valid-tag.out" +grep -qF 'OK reviewed signed PR #76' "$scratch/valid-tag.out" + +export TRIGGER_REF_CREATED=false +expect_reject moved-tag bash "$verifier" + +echo "reviewed image release gate tests passed" diff --git a/.github/scripts/verify-registry-tag-absent.sh b/.github/scripts/verify-registry-tag-absent.sh new file mode 100644 index 000000000..c91ba4a58 --- /dev/null +++ b/.github/scripts/verify-registry-tag-absent.sh @@ -0,0 +1,59 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +repository=${1:-} +tag=${2:-} +registry=${REGISTRY:-ghcr.io} + +if [[ ! "$repository" =~ ^[a-z0-9._-]+/[a-z0-9._/-]+$ ]]; then + echo "repository must be a lowercase registry namespace/path" >&2 + exit 2 +fi +if [[ ! "$tag" =~ ^reviewed-[0-9a-f]{40}$ ]]; then + echo "tag must be reviewed- followed by the exact 40-character commit SHA" >&2 + exit 2 +fi +if [[ -z "${GITHUB_ACTOR:-}" || -z "${GHCR_TOKEN:-}" ]]; then + echo "GITHUB_ACTOR and GHCR_TOKEN are required" >&2 + exit 2 +fi + +token_json="$(curl --fail --silent --show-error \ + --user "${GITHUB_ACTOR}:${GHCR_TOKEN}" \ + --get \ + --data-urlencode "scope=repository:${repository}:pull" \ + --data-urlencode "service=${registry}" \ + "https://${registry}/token")" +registry_token="$(jq -er '.token' <<<"$token_json")" +response_body="$(mktemp -t reviewed-tag-response.XXXXXXXXXX)" +trap 'rm -f "$response_body"' EXIT + +if ! status="$(curl --silent --show-error \ + --output "$response_body" \ + --write-out '%{http_code}' \ + --header "Authorization: Bearer ${registry_token}" \ + --header 'Accept: application/vnd.oci.image.index.v1+json, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.docker.distribution.manifest.v2+json' \ + "https://${registry}/v2/${repository}/manifests/${tag}")"; then + echo "registry request failed while checking ${repository}:${tag}" >&2 + exit 1 +fi + +case "$status" in + 404) + if ! jq -e 'any(.errors[]?; .code == "MANIFEST_UNKNOWN")' "$response_body" >/dev/null; then + echo "registry returned HTTP 404 without MANIFEST_UNKNOWN for ${repository}:${tag}" >&2 + exit 1 + fi + ;; + 200) + echo "refusing to overwrite immutable reviewed tag: ${repository}:${tag}" >&2 + exit 1 + ;; + *) + echo "registry returned HTTP $status while checking ${repository}:${tag}" >&2 + exit 1 + ;; +esac + +echo "OK reviewed tag is absent: ${repository}:${tag}" diff --git a/.github/scripts/verify-reviewed-image-release.sh b/.github/scripts/verify-reviewed-image-release.sh new file mode 100644 index 000000000..7b017e831 --- /dev/null +++ b/.github/scripts/verify-reviewed-image-release.sh @@ -0,0 +1,145 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +required_env=( + GITHUB_API_TOKEN + TRIGGER_API_URL + TRIGGER_EVENT_NAME + TRIGGER_REF + TRIGGER_REF_NAME + TRIGGER_REF_TYPE + TRIGGER_REPOSITORY + TRIGGER_SHA +) +for name in "${required_env[@]}"; do + if [[ -z "${!name:-}" ]]; then + echo "missing required environment variable: $name" >&2 + exit 2 + fi +done +unset name required_env + +if [[ ! "$TRIGGER_REPOSITORY" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ || + ! "$TRIGGER_SHA" =~ ^[0-9a-f]{40}$ ]]; then + echo "invalid trigger repository or commit SHA" >&2 + exit 2 +fi +checked_out_sha="$(git rev-parse HEAD)" +if [[ "$checked_out_sha" != "$TRIGGER_SHA" ]]; then + echo "publication trigger SHA does not match the checked-out commit" >&2 + exit 1 +fi + +api_get() { + curl --fail --silent --show-error \ + --header "Authorization: Bearer ${GITHUB_API_TOKEN}" \ + --header 'Accept: application/vnd.github+json' \ + --header 'X-GitHub-Api-Version: 2022-11-28' \ + "$1" +} + +case "$TRIGGER_EVENT_NAME" in + workflow_dispatch) + if [[ "${DISPATCH_REVIEWED_COMMIT:-}" != "$TRIGGER_SHA" ]]; then + echo "workflow dispatch reviewed_commit must match the exact checked-out commit" >&2 + exit 1 + fi + ;; + push) + if [[ "$TRIGGER_REF_TYPE" != "tag" || "${TRIGGER_REF_CREATED:-}" != "true" || + "$TRIGGER_REF" != "refs/tags/${TRIGGER_REF_NAME}" ]]; then + echo "publication must be triggered by a newly created tag ref" >&2 + exit 1 + fi + tag_pattern='^reviewed-images-publish-([0-9a-f]{40})-at-([1-9][0-9]{9})$' + if [[ ! "$TRIGGER_REF_NAME" =~ $tag_pattern || + "${BASH_REMATCH[1]:-}" != "$TRIGGER_SHA" ]]; then + echo "publication tag must encode the exact reviewed commit" >&2 + exit 1 + fi + attested_at="${BASH_REMATCH[2]}" + now="$(date +%s)" + if ((attested_at > now + 120 || now - attested_at > 900)); then + echo "publication tag timestamp must be within the 900-second admission window" >&2 + exit 1 + fi + encoded_ref="$(jq -nr --arg value "$TRIGGER_REF_NAME" '$value|@uri')" + ref_json="$(api_get "${TRIGGER_API_URL}/repos/${TRIGGER_REPOSITORY}/git/ref/tags/${encoded_ref}")" + if [[ "$(jq -er '.ref' <<<"$ref_json")" != "$TRIGGER_REF" || + "$(jq -er '.object.type' <<<"$ref_json")" != "commit" || + "$(jq -er '.object.sha' <<<"$ref_json")" != "$TRIGGER_SHA" ]]; then + echo "publication tag must be a lightweight ref directly targeting the reviewed commit" >&2 + exit 1 + fi + ;; + *) + echo "unsupported publication trigger: $TRIGGER_EVENT_NAME" >&2 + exit 1 + ;; +esac + +commit_json="$(api_get "${TRIGGER_API_URL}/repos/${TRIGGER_REPOSITORY}/git/commits/${TRIGGER_SHA}")" +if [[ "$(jq -r '.sha' <<<"$commit_json")" != "$TRIGGER_SHA" || + "$(jq -r '.verification.verified' <<<"$commit_json")" != "true" || + "$(jq -r '.verification.reason' <<<"$commit_json")" != "valid" ]]; then + echo "reviewed publication commit is not GitHub-signature verified" >&2 + exit 1 +fi + +pulls_json="$(api_get "${TRIGGER_API_URL}/repos/${TRIGGER_REPOSITORY}/commits/${TRIGGER_SHA}/pulls")" +reviewed_pr="$(jq -cer --arg sha "$TRIGGER_SHA" --arg repo "$TRIGGER_REPOSITORY" ' + [ .[] + | select(.base.ref == "main") + | select(.base.repo.full_name == $repo) + | select(.head.sha == $sha) + | select(.head.repo.full_name == $repo) + | select(.draft == false) + | select(.state == "open" or .merged_at != null) + ] + | if length == 1 then .[0] else error("expected exactly one ready or merged main PR at the trigger SHA") end +' <<<"$pulls_json")" +reviewed_pr_number="$(jq -er '.number' <<<"$reviewed_pr")" + +checks_json="$(api_get "${TRIGGER_API_URL}/repos/${TRIGGER_REPOSITORY}/commits/${TRIGGER_SHA}/check-runs?filter=latest&per_page=100")" + +require_workflow_check() { + local check_name="$1" + local workflow_path="$2" + local check_json details_url run_id run_json + check_json="$(jq -cer --arg name "$check_name" --arg sha "$TRIGGER_SHA" ' + [.check_runs[] + | select(.name == $name and .head_sha == $sha and .app.slug == "github-actions")] + | sort_by(.completed_at // .started_at // "") + | if length > 0 then .[-1] else error("missing required GitHub Actions check") end + ' <<<"$checks_json")" + if [[ "$(jq -r '.status' <<<"$check_json")" != "completed" || + "$(jq -r '.conclusion' <<<"$check_json")" != "success" ]]; then + echo "required check is not successful: $check_name" >&2 + exit 1 + fi + details_url="$(jq -er '.details_url' <<<"$check_json")" + if [[ ! "$details_url" =~ /actions/runs/([0-9]+)/job/ ]]; then + echo "required check is not bound to an Actions workflow run: $check_name" >&2 + exit 1 + fi + run_id="${BASH_REMATCH[1]}" + run_json="$(api_get "${TRIGGER_API_URL}/repos/${TRIGGER_REPOSITORY}/actions/runs/${run_id}")" + if [[ "$(jq -r '.head_sha' <<<"$run_json")" != "$TRIGGER_SHA" || + "$(jq -r '.event' <<<"$run_json")" != "pull_request" || + "$(jq -r '.status' <<<"$run_json")" != "completed" || + "$(jq -r '.conclusion' <<<"$run_json")" != "success" || + "$(jq -r '.path' <<<"$run_json")" != "$workflow_path" || + "$(jq -r --argjson pr "$reviewed_pr_number" --arg sha "$TRIGGER_SHA" ' + any(.pull_requests[]?; .number == $pr and .head.sha == $sha) + ' <<<"$run_json")" != "true" ]]; then + echo "required check is not a successful exact-head run of $workflow_path: $check_name" >&2 + exit 1 + fi +} + +require_workflow_check "CI success" ".github/workflows/ci.yml" +require_workflow_check "Console CI success" ".github/workflows/console-ci.yml" +require_workflow_check "Image validation success" ".github/workflows/validate-images.yml" + +echo "OK reviewed signed PR #${reviewed_pr_number} and exact-head non-CodeQL checks authorize image publication" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2f8c25b8c..b7d024a5c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -19,15 +19,26 @@ jobs: outputs: migration_order: ${{ steps.filter.outputs.migration_order }} rust_api: ${{ steps.filter.outputs.rust_api }} - sandbox_config_tests: ${{ steps.filter.outputs.sandbox_config_tests }} + slack_archive_privacy: ${{ steps.filter.outputs.slack_archive_privacy }} slackbotv2_tests: ${{ steps.filter.outputs.slackbotv2_tests }} - linearbot_tests: ${{ steps.filter.outputs.linearbot_tests }} discordbot_checks: ${{ steps.filter.outputs.discordbot_checks }} + githubbot_checks: ${{ steps.filter.outputs.githubbot_checks }} teamsbot_checks: ${{ steps.filter.outputs.teamsbot_checks }} + shared_node_packages: ${{ steps.filter.outputs.shared_node_packages }} + python_surfaces: ${{ steps.filter.outputs.python_surfaces }} steps: - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 with: fetch-depth: 0 + persist-credentials: false + + - name: Audit workflow trust boundaries + run: | + set -euo pipefail + python .github/scripts/audit_workflow_trust.py + bash .github/scripts/test-resolve-runnable-image-digest.sh + bash .github/scripts/test-verify-registry-tag-absent.sh + bash .github/scripts/test-verify-reviewed-image-release.sh - id: filter name: Detect changed paths @@ -37,17 +48,9 @@ jobs: changed_files="$(git diff --name-only "${base}...HEAD")" elif [[ "${{ github.event_name }}" == "push" ]]; then changed_files="$(git diff --name-only "${{ github.event.before }}...HEAD")" - elif [[ "${{ github.event_name }}" == "merge_group" ]]; then - base="${{ github.event.merge_group.base_sha }}" - changed_files="$(git diff --name-only "${base}...HEAD")" else - base_ref="${{ github.base_ref || 'main' }}" - if git rev-parse --verify --quiet "origin/${base_ref}" >/dev/null; then - changed_files="$(git diff --name-only "origin/${base_ref}...HEAD")" - else - git fetch --no-tags --depth=1 origin "${base_ref}" - changed_files="$(git diff --name-only FETCH_HEAD...HEAD)" - fi + git fetch --no-tags --depth=1 origin main + changed_files="$(git diff --name-only origin/main...HEAD)" fi matches() { @@ -78,13 +81,9 @@ jobs: set_output rust_api \ '^services/api-rs/' \ '^\.github/workflows/ci\.yml$' - set_output sandbox_config_tests \ - '^services/sandbox/configure_codex_config\.py$' \ - '^services/sandbox/test_configure_codex_config\.py$' \ - '^services/sandbox/install_tool_shims\.py$' \ - '^services/sandbox/test_install_tool_shims\.py$' \ - '^services/sandbox/Dockerfile$' \ - '^services/sandbox/entrypoint\.sh$' \ + set_output slack_archive_privacy \ + '^workflows/slack/archive_import\.py$' \ + '^workflows/slack/tests/test_archive_import\.py$' \ '^\.github/workflows/ci\.yml$' set_output slackbotv2_tests \ '^services/slackbotv2/' \ @@ -92,14 +91,14 @@ jobs: '^pnpm-lock\.yaml$' \ '^pnpm-workspace\.yaml$' \ '^\.github/workflows/ci\.yml$' - set_output linearbot_tests \ - '^services/linearbot/' \ + set_output discordbot_checks \ + '^services/discordbot/' \ '^package\.json$' \ '^pnpm-lock\.yaml$' \ '^pnpm-workspace\.yaml$' \ '^\.github/workflows/ci\.yml$' - set_output discordbot_checks \ - '^services/discordbot/' \ + set_output githubbot_checks \ + '^services/githubbot/' \ '^package\.json$' \ '^pnpm-lock\.yaml$' \ '^pnpm-workspace\.yaml$' \ @@ -110,22 +109,18 @@ jobs: '^pnpm-lock\.yaml$' \ '^pnpm-workspace\.yaml$' \ '^\.github/workflows/ci\.yml$' - - sandbox-config-tests: - name: Sandbox Codex config tests - runs-on: ubuntu-latest - needs: ci_changes - if: needs.ci_changes.outputs.sandbox_config_tests == 'true' - steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 - with: - persist-credentials: false - - - name: Test sandbox Codex config generation - run: python3 services/sandbox/test_configure_codex_config.py - - - name: Test sandbox tool shim installation - run: python3 services/sandbox/test_install_tool_shims.py + set_output shared_node_packages \ + '^packages/(api-client|harness-events|rendering)/' \ + '^package\.json$' \ + '^pnpm-lock\.yaml$' \ + '^pnpm-workspace\.yaml$' \ + '^\.github/workflows/ci\.yml$' + set_output python_surfaces \ + '^services/sandbox/' \ + '^services/workflow-python/' \ + '^workflows/' \ + '^tools/productivity/slack/' \ + '^\.github/workflows/ci\.yml$' migration-order: name: Migration order @@ -146,19 +141,6 @@ jobs: runs-on: ubuntu-latest needs: ci_changes if: needs.ci_changes.outputs.rust_api == 'true' - services: - postgres: - image: postgres:17 - env: - POSTGRES_USER: postgres - POSTGRES_PASSWORD: postgres - ports: - - 5432:5432 - options: >- - --health-cmd="pg_isready -U postgres" - --health-interval=10s - --health-timeout=5s - --health-retries=5 steps: - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 with: @@ -181,10 +163,32 @@ jobs: working-directory: services/api-rs run: cargo clippy --workspace --all-targets -- -D warnings + - name: Start Rust test database + run: | + docker run --detach --name centaur-rust-test-postgres \ + --publish 127.0.0.1:5432:5432 \ + --env POSTGRES_USER=postgres \ + --env POSTGRES_PASSWORD=postgres \ + paradedb/paradedb:0.23.0-pg16@sha256:e41e0c742ef91ece4fc7c08dda7f24e5a8f818563b164bbfea3a4364941d75f7 \ + -c shared_preload_libraries=pg_search,pg_cron \ + -c max_connections=500 + for _attempt in {1..60}; do + if docker logs centaur-rust-test-postgres 2>&1 \ + | grep -q 'PostgreSQL init process complete' \ + && docker exec centaur-rust-test-postgres \ + psql -U postgres -d postgres -tAc 'select 1' >/dev/null; then + exit 0 + fi + sleep 1 + done + docker logs centaur-rust-test-postgres + exit 1 + - name: Test working-directory: services/api-rs env: SESSION_SQLX_TEST_DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres + SESSION_RUNTIME_TEST_DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres run: cargo test --workspace - name: Start API integration test dependencies @@ -194,10 +198,10 @@ jobs: --env POSTGRES_USER=postgres \ --env POSTGRES_PASSWORD=postgres \ --env POSTGRES_DB=centaur \ - paradedb/paradedb:0.23.0-pg16 \ + paradedb/paradedb:0.23.0-pg16@sha256:e41e0c742ef91ece4fc7c08dda7f24e5a8f818563b164bbfea3a4364941d75f7 \ -c shared_preload_libraries=pg_search,pg_cron \ -c max_connections=500 - for _ in {1..60}; do + for _attempt in {1..60}; do if docker logs centaur-api-integration-test-postgres 2>&1 \ | grep -q 'PostgreSQL init process complete' \ && docker exec centaur-api-integration-test-postgres \ @@ -218,8 +222,12 @@ jobs: DATABASE_URL: postgres://postgres:postgres@127.0.0.1:15432/centaur?sslmode=disable RUN_MIGRATIONS: "true" RUST_LOG: info + CENTAUR_CONTROL_API_KEY: api-integration-control-0000000000000001 + SLACK_FEEDBACK_API_KEY: api-integration-feedback-0000000000000002 + SLACKBOT_API_KEY: api-integration-slackbot-0000000000000003 WORKFLOW_DIRS: ${{ runner.temp }}/api-integration-workflows WORKFLOW_HOST_SANDBOX: "false" + WORKFLOW_API_KEY: api-integration-workflow-0000000000000004 WORKFLOW_REAP_REMOVED_AFTER_TICKS: "1" WORKFLOW_RECONCILE_INTERVAL_SECS: "1" run: | @@ -247,6 +255,26 @@ jobs: fi docker rm -f centaur-api-integration-test-postgres || true + slack-archive-privacy: + name: Slack archive privacy tests + runs-on: ubuntu-latest + needs: ci_changes + if: needs.ci_changes.outputs.slack_archive_privacy == 'true' + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + + - name: Install pytest + run: python -m pip install pytest==8.4.2 + + - name: Verify archive import privacy boundaries + run: python -m pytest workflows/slack/tests/test_archive_import.py + slackbotv2-tests: name: Slackbot v2 tests runs-on: ubuntu-latest @@ -278,8 +306,6 @@ jobs: linearbot-tests: name: Linearbot typecheck and tests runs-on: ubuntu-latest - needs: ci_changes - if: needs.ci_changes.outputs.linearbot_tests == 'true' steps: - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 with: @@ -341,6 +367,40 @@ jobs: working-directory: services/discordbot run: bun run test + githubbot-checks: + name: Githubbot typecheck and tests + runs-on: ubuntu-latest + needs: ci_changes + if: needs.ci_changes.outputs.githubbot_checks == 'true' + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "24" + + - uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4 + with: + version: 10.28.1 + run_install: false + + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + + # Workspace-root install: githubbot links @centaur/* workspace packages + # and relies on the root pnpm patches. + - name: Install Node dependencies + run: pnpm install --frozen-lockfile + + - name: Typecheck Githubbot + working-directory: services/githubbot + run: bun run check:types + + - name: Githubbot tests + working-directory: services/githubbot + run: bun run test + teamsbot-checks: name: Teamsbot typecheck and tests runs-on: ubuntu-latest @@ -373,23 +433,96 @@ jobs: working-directory: services/teamsbot run: bun run test + shared-node-packages: + name: Shared API client and rendering checks + runs-on: ubuntu-latest + needs: ci_changes + if: needs.ci_changes.outputs.shared_node_packages == 'true' + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "24" + + - uses: pnpm/action-setup@b906affcce14559ad1aafd4ab0e942779e9f58b1 # v4 + with: + version: 10.28.1 + run_install: false + + - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + + - name: Install Node dependencies + run: pnpm install --frozen-lockfile + + - name: Typecheck shared packages + run: | + pnpm --filter @centaur/api-client exec tsc --noEmit + pnpm --filter @centaur/rendering typecheck + + - name: Test shared packages + run: | + pnpm --filter @centaur/api-client test + pnpm --filter @centaur/rendering test + + python-surfaces: + name: Sandbox, workflow, and Slack Python tests + runs-on: ubuntu-latest + needs: ci_changes + if: needs.ci_changes.outputs.python_surfaces == 'true' + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + + - name: Install Python test dependencies + run: | + python -m pip install pytest==8.4.2 tomli-w==1.2.0 uv==0.11.6 ./tools/productivity/slack + uv sync --project services/workflow-python --locked + + - name: Test sandbox bootstrap + run: python -m pytest services/sandbox/test_*.py + + - name: Test workflow library + env: + PYTHONPATH: workflows + run: python -m pytest workflows/tests workflows/slack/tests + + - name: Test workflow host + run: | + uv run --project services/workflow-python python -m unittest discover \ + -s services/workflow-python/tests -p 'test_*.py' + + - name: Test Slack tool + env: + PYTHONPATH: tools/productivity + run: python -m pytest tools/productivity/slack/tests + ci-success: name: CI success runs-on: ubuntu-latest if: always() needs: - ci_changes - - sandbox-config-tests - migration-order - rust-api + - slack-archive-privacy - slackbotv2-tests - - linearbot-tests - discordbot-checks + - githubbot-checks - teamsbot-checks + - shared-node-packages + - python-surfaces timeout-minutes: 30 steps: - name: Decide whether the needed jobs succeeded or failed uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe # release/v1 with: - allowed-skips: sandbox-config-tests, migration-order, rust-api, slackbotv2-tests, linearbot-tests, discordbot-checks, teamsbot-checks + allowed-skips: migration-order, rust-api, slack-archive-privacy, slackbotv2-tests, discordbot-checks, githubbot-checks, teamsbot-checks, shared-node-packages, python-surfaces jobs: ${{ toJSON(needs) }} diff --git a/.github/workflows/console-ci.yml b/.github/workflows/console-ci.yml index 844cf269b..00d15102c 100644 --- a/.github/workflows/console-ci.yml +++ b/.github/workflows/console-ci.yml @@ -5,13 +5,13 @@ on: push: branches: [ main ] -permissions: - contents: read - defaults: run: working-directory: services/console +permissions: + contents: read + jobs: console_changes: name: Detect Console CI changes @@ -149,7 +149,6 @@ jobs: IRON_CONTROL_DB_PORT: 5432 IRON_CONTROL_DB_USERNAME: postgres IRON_CONTROL_DB_PASSWORD: postgres - # RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }} run: bin/rails db:test:prepare test console-ci-success: diff --git a/.github/workflows/docs-deploy.yml b/.github/workflows/docs-deploy.yml new file mode 100644 index 000000000..34d9b632c --- /dev/null +++ b/.github/workflows/docs-deploy.yml @@ -0,0 +1,55 @@ +name: Deploy Docs + +on: + workflow_dispatch: + inputs: + confirm_reviewed_main: + description: Confirm this deploy is from reviewed main + required: true + type: boolean + +concurrency: + group: docs-deploy + cancel-in-progress: false + +permissions: + contents: read + +jobs: + deploy: + name: Deploy reviewed docs + if: github.ref == 'refs/heads/main' && inputs.confirm_reviewed_main + runs-on: ubuntu-latest + timeout-minutes: 20 + defaults: + run: + working-directory: docs + steps: + - name: Checkout + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - name: Set up Node + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "24" + cache: npm + cache-dependency-path: docs/package-lock.json + + - name: Set up Bun + uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2 + + - name: Install dependencies + run: npm ci + + - name: Build docs + run: npm run build + + - name: Deploy + uses: cloudflare/wrangler-action@9acf94ace14e7dc412b076f2c5c20b8ce93c79cd # v3.15.0 + with: + apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} + accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + workingDirectory: docs + command: deploy --config wrangler.ci.jsonc diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index d978d17f7..86f330027 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -19,12 +19,10 @@ concurrency: permissions: contents: read - deployments: write - issues: write - pull-requests: write jobs: build-and-deploy: + # Preserve the historical check context while PR execution is now build-only. name: Build and deploy docs runs-on: ubuntu-latest timeout-minutes: 20 @@ -33,9 +31,6 @@ jobs: worker: - name: docs directory: docs - env: - CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} defaults: run: working-directory: docs @@ -60,68 +55,3 @@ jobs: - name: Build docs run: npm run build - - - name: Skip Cloudflare deploy - if: ${{ env.CLOUDFLARE_ACCOUNT_ID == '' || env.CLOUDFLARE_API_TOKEN == '' }} - run: echo "Cloudflare credentials are not configured; docs build completed without deployment." - - - name: Deploy - if: github.event_name != 'pull_request' - uses: cloudflare/wrangler-action@9acf94ace14e7dc412b076f2c5c20b8ce93c79cd # v3.15.0 - with: - apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }} - accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - workingDirectory: ${{ matrix.worker.directory }} - command: deploy --config wrangler.ci.jsonc - - - name: Preview - id: preview - if: ${{ github.event_name == 'pull_request' && env.CLOUDFLARE_ACCOUNT_ID != '' && env.CLOUDFLARE_API_TOKEN != '' }} - env: - CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} - CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} - PREVIEW_ALIAS: pr-${{ github.event.pull_request.number }} - run: | - set -o pipefail - npx wrangler versions upload --preview-alias "$PREVIEW_ALIAS" 2>&1 | tee "$RUNNER_TEMP/wrangler-preview.log" - deployment_url="$(grep -Eo 'https://[^[:space:]]+workers.dev[^[:space:]]*' "$RUNNER_TEMP/wrangler-preview.log" | tail -n 1)" - if [ -z "$deployment_url" ]; then - echo "No Cloudflare preview URL found in Wrangler output." >&2 - exit 1 - fi - echo "deployment-url=$deployment_url" >> "$GITHUB_OUTPUT" - - - name: Publish preview summary - if: ${{ github.event_name == 'pull_request' && steps.preview.outputs.deployment-url != '' }} - env: - DEPLOYMENT_URL: ${{ steps.preview.outputs.deployment-url }} - run: | - { - echo "### Cloudflare Workers docs preview" - echo - echo "$DEPLOYMENT_URL" - } >> "$GITHUB_STEP_SUMMARY" - - - name: Publish preview comment - if: ${{ github.event_name == 'pull_request' && steps.preview.outputs.deployment-url != '' }} - env: - DEPLOYMENT_URL: ${{ steps.preview.outputs.deployment-url }} - GH_TOKEN: ${{ github.token }} - PR_NUMBER: ${{ github.event.pull_request.number }} - run: | - marker="" - body="$(cat </dev/null | tail -n 1 || true)" - if [ -n "$comment_id" ]; then - gh api --method PATCH "repos/${GITHUB_REPOSITORY}/issues/comments/${comment_id}" -f body="$body" - else - gh api --method POST "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" -f body="$body" - fi diff --git a/.github/workflows/publish-images.yml b/.github/workflows/publish-images.yml index 89f75fc74..f2de3ebe3 100644 --- a/.github/workflows/publish-images.yml +++ b/.github/workflows/publish-images.yml @@ -1,29 +1,25 @@ -name: Publish Images +name: Publish Reviewed Images on: push: - branches: [main] - tags: [v*] - paths: - - .github/workflows/publish-images.yml - - services/** - - crates/harness-server/** - - centaur_sdk/** - - packages/** - - tools/** - - scripts/bootstrap-k8s-secrets.sh - - package.json - - pnpm-lock.yaml - - pnpm-workspace.yaml - - .agents/skills/** - pull_request: + tags: + - 'reviewed-images-publish-*' workflow_dispatch: + inputs: + reviewed_commit: + description: Exact signed, ready-for-review PR head to publish + required: true + type: string concurrency: - group: publish-images-${{ github.ref }} - cancel-in-progress: true + # Reviewed package tags are global state. Serialize every branch, tag, and + # dispatch so a later run cannot race or cancel a partially published run. + group: publish-reviewed-centaur-images + cancel-in-progress: false permissions: + actions: read + checks: read contents: read packages: write pull-requests: read @@ -32,24 +28,76 @@ env: REGISTRY: ghcr.io IMAGE_NAMESPACE: tiplink/centaur IMAGE_SOURCE: https://github.com/TipLink/centaur - # Main/tags keep optimized release images. PRs and manual non-release - # branch publishes use debug builds so staging/dev iteration does not spend - # minutes optimizing Rust binaries that are immediately replaced by the next - # test build. - RUST_BUILD_PROFILE: ${{ (github.event_name == 'pull_request' || (github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' && !startsWith(github.ref, 'refs/tags/v'))) && 'debug' || 'release' }} + RUST_BUILD_PROFILE: release jobs: + release-gate: + name: Bind publication to the reviewed signed PR head + runs-on: ubuntu-latest + env: + DISPATCH_REVIEWED_COMMIT: ${{ inputs.reviewed_commit }} + GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }} + TRIGGER_API_URL: ${{ github.api_url }} + TRIGGER_EVENT_NAME: ${{ github.event_name }} + TRIGGER_REF: ${{ github.ref }} + TRIGGER_REF_CREATED: ${{ github.event.created }} + TRIGGER_REF_NAME: ${{ github.ref_name }} + TRIGGER_REF_TYPE: ${{ github.ref_type }} + TRIGGER_REPOSITORY: ${{ github.repository }} + TRIGGER_SHA: ${{ github.sha }} + steps: + - name: Checkout the exact publication commit + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - name: Require signed PR head and exact-head non-CodeQL checks + run: bash .github/scripts/verify-reviewed-image-release.sh + + tag-absence-gate: + name: Prove every reviewed image tag is unassigned + runs-on: ubuntu-latest + needs: release-gate + env: + GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }} + REVIEWED_TAG: reviewed-${{ github.sha }} + steps: + - name: Checkout immutable-tag verifier + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - name: Reject existing or unprovably absent tags + run: | + set -euo pipefail + IFS=$'\n\t' + for image in \ + centaur-api-rs \ + centaur-slackbotv2 \ + centaur-linearbot \ + centaur-discordbot \ + centaur-githubbot \ + centaur-teamsbot \ + centaur-agent \ + centaur-iron-proxy \ + centaur-console; do + bash .github/scripts/verify-registry-tag-absent.sh \ + "${IMAGE_NAMESPACE}/${image}" "$REVIEWED_TAG" + done + # Build each image natively per platform (amd64 on x64 runners, arm64 on - # arm runners), push by digest, and hand the digests to the merge job below - # which assembles the multi-arch manifest. PR and manual non-release branch - # builds stay amd64-only to keep iteration fast. + # arm runners), push by digest, and retain two distinct identities: the + # attested per-platform root used for the final merge and its runnable child + # manifest. Pull requests are validated by the separate read-only workflow; + # only the signed, exact checked PR head reaches this package-write lane. build: + needs: tag-absence-gate runs-on: ${{ matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }} strategy: fail-fast: false matrix: - service: [api-rs, slackbotv2, linearbot, discordbot, teamsbot, agent, iron-proxy, console] - platform: ${{ (github.event_name == 'push' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')) && fromJSON('["linux/amd64", "linux/arm64"]') || fromJSON('["linux/amd64"]') }} + service: [api-rs, slackbotv2, linearbot, discordbot, githubbot, teamsbot, agent, iron-proxy, console] + platform: [linux/amd64, linux/arm64] include: - service: api-rs image: centaur-api-rs @@ -71,6 +119,11 @@ jobs: context: . dockerfile: services/discordbot/Dockerfile target: "" + - service: githubbot + image: centaur-githubbot + context: . + dockerfile: services/githubbot/Dockerfile + target: "" - service: teamsbot image: centaur-teamsbot context: . @@ -100,6 +153,7 @@ jobs: - name: Derive platform slug run: | + set -euo pipefail platform="${{ matrix.platform }}" echo "PLATFORM_SLUG=${platform//\//-}" >> "$GITHUB_ENV" @@ -134,40 +188,65 @@ jobs: platforms: ${{ matrix.platform }} labels: ${{ steps.meta.outputs.labels }} # Tags are applied by the merge job on the multi-arch manifest; - # per-arch builds are pushed by digest only. Fork PRs run with a - # read-only GITHUB_TOKEN, so they build without pushing. - outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }},push-by-digest=true,name-canonical=true,push=${{ !github.event.pull_request.head.repo.fork }} + # per-arch release builds are pushed by digest only. + outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }},push-by-digest=true,name-canonical=true,push=true + provenance: true + sbom: true build-args: | RUST_BUILD_PROFILE=${{ env.RUST_BUILD_PROFILE }} cache-from: | type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }}:buildcache-${{ env.PLATFORM_SLUG }} type=gha,scope=${{ matrix.image }}-${{ env.PLATFORM_SLUG }} - # Fork PRs run with a read-only GITHUB_TOKEN: exporting the registry - # cache would fail the build, so only export it when we can push. cache-to: | - ${{ !github.event.pull_request.head.repo.fork && format('type=registry,ref={0}/{1}/{2}:buildcache-{3},mode=max', env.REGISTRY, env.IMAGE_NAMESPACE, matrix.image, env.PLATFORM_SLUG) || '' }} + type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }}:buildcache-${{ env.PLATFORM_SLUG }},mode=max type=gha,mode=max,scope=${{ matrix.image }}-${{ env.PLATFORM_SLUG }} - - name: Export digest - if: ${{ !github.event.pull_request.head.repo.fork }} + - name: Export attested root and runnable platform child digests + env: + BUILD_ROOT_DIGEST: ${{ steps.build.outputs.digest }} + BUILD_IMAGE: ${{ matrix.image }} + BUILD_PLATFORM: ${{ matrix.platform }} run: | - mkdir -p ${{ runner.temp }}/digests - digest="${{ steps.build.outputs.digest }}" - touch "${{ runner.temp }}/digests/${digest#sha256:}" + set -euo pipefail + IFS=$'\n\t' + if [[ ! "$BUILD_ROOT_DIGEST" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "invalid attested build root digest for $BUILD_IMAGE: $BUILD_ROOT_DIGEST" >&2 + exit 1 + fi + mkdir -p "${{ runner.temp }}/merge-root-digests" + touch "${{ runner.temp }}/merge-root-digests/${BUILD_ROOT_DIGEST#sha256:}" + + platform_os=${BUILD_PLATFORM%%/*} + platform_architecture=${BUILD_PLATFORM##*/} + runnable_digest="$(bash .github/scripts/resolve-runnable-image-digest.sh \ + "${REGISTRY}/${IMAGE_NAMESPACE}/${BUILD_IMAGE}@${BUILD_ROOT_DIGEST}" \ + "$platform_os" "$platform_architecture")" + if [[ ! "$runnable_digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "invalid runnable child digest for $BUILD_IMAGE ($BUILD_PLATFORM): $runnable_digest" >&2 + exit 1 + fi + mkdir -p "${{ runner.temp }}/runnable-child-digests" + touch "${{ runner.temp }}/runnable-child-digests/${runnable_digest#sha256:}" + + - name: Upload attested platform root digest + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: merge-root-digests-${{ matrix.image }}-${{ env.PLATFORM_SLUG }} + path: ${{ runner.temp }}/merge-root-digests/* + if-no-files-found: error + retention-days: 1 - - name: Upload digest - if: ${{ !github.event.pull_request.head.repo.fork }} + - name: Upload runnable platform child digest uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: - name: digests-${{ matrix.image }}-${{ env.PLATFORM_SLUG }} - path: ${{ runner.temp }}/digests/* + name: runnable-child-digests-${{ matrix.image }}-${{ env.PLATFORM_SLUG }} + path: ${{ runner.temp }}/runnable-child-digests/* if-no-files-found: error retention-days: 1 merge: runs-on: ubuntu-latest needs: build - if: ${{ !github.event.pull_request.head.repo.fork }} strategy: fail-fast: false matrix: @@ -179,14 +258,20 @@ jobs: - image: centaur-iron-proxy - image: centaur-console - image: centaur-discordbot + - image: centaur-githubbot - image: centaur-teamsbot steps: - - name: Download digests + - name: Checkout immutable-tag verifier + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - name: Download attested platform root digests uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: - pattern: digests-${{ matrix.image }}-* - path: ${{ runner.temp }}/digests + pattern: merge-root-digests-${{ matrix.image }}-* + path: ${{ runner.temp }}/merge-root-digests merge-multiple: true - name: Set up Docker Buildx @@ -199,318 +284,128 @@ jobs: username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - - name: Docker metadata for ${{ matrix.image }} - id: meta - uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5 - with: - images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }} - flavor: | - latest=false - tags: | - type=raw,value=latest,enable={{is_default_branch}} - type=raw,value=main,enable={{is_default_branch}} - type=raw,value=edge,enable={{is_default_branch}} - type=sha,prefix=main-sha-,enable={{is_default_branch}} - type=semver,pattern={{raw}} - type=ref,event=pr - type=sha - labels: | - org.opencontainers.image.title=${{ matrix.image }} - org.opencontainers.image.source=${{ env.IMAGE_SOURCE }} + - name: Recheck absence and create immutable reviewed manifest + working-directory: ${{ runner.temp }}/merge-root-digests env: - DOCKER_METADATA_PR_HEAD_SHA: true - - - name: Create multi-arch manifest for ${{ matrix.image }} - working-directory: ${{ runner.temp }}/digests + GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }} + REVIEWED_TAG: reviewed-${{ github.sha }} run: | - mapfile -t tags < <(jq -r '.tags[]' <<< "$DOCKER_METADATA_OUTPUT_JSON") - tag_args=() - for tag in "${tags[@]}"; do - tag_args+=("-t" "$tag") - done - + set -euo pipefail + IFS=$'\n\t' + repository="${IMAGE_NAMESPACE}/${{ matrix.image }}" + mapfile -t root_digest_files < <(find . -maxdepth 1 -type f -print) + if [[ "${#root_digest_files[@]}" -ne 2 ]]; then + echo "expected exactly two attested platform roots for ${{ matrix.image }}; found ${#root_digest_files[@]}" >&2 + exit 1 + fi image_refs=() - for digest in *; do - image_refs+=("${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }}@sha256:${digest}") + for digest_file in "${root_digest_files[@]}"; do + digest="sha256:$(basename "$digest_file")" + if [[ ! "$digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "invalid attested platform root for ${{ matrix.image }}: $digest" >&2 + exit 1 + fi + image_refs+=("${REGISTRY}/${repository}@${digest}") done - docker buildx imagetools create "${tag_args[@]}" "${image_refs[@]}" + # Keep this single-assignment proof immediately adjacent to the only + # command that can create the final reviewed tag. + bash "$GITHUB_WORKSPACE/.github/scripts/verify-registry-tag-absent.sh" \ + "$repository" "$REVIEWED_TAG" + docker buildx imagetools create \ + -t "${REGISTRY}/${repository}:${REVIEWED_TAG}" \ + "${image_refs[@]}" - - name: Inspect manifest + - name: Inspect immutable reviewed manifest run: | - docker buildx imagetools inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${{ matrix.image }}:${{ steps.meta.outputs.version }} + set -euo pipefail + docker buildx imagetools inspect \ + "${REGISTRY}/${IMAGE_NAMESPACE}/${{ matrix.image }}:reviewed-${GITHUB_SHA}" - promote-fineas-infra: - name: Open Fineas infra promotion PR + release-descriptor: + name: Publish reviewed linux/arm64 release descriptor runs-on: ubuntu-latest needs: merge - if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/main' steps: - - name: Create Fineas infra GitHub App token - id: fineas_token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 - with: - app-id: ${{ vars.FINEAS_GITHUB_APP_ID }} - private-key: ${{ secrets.FINEAS_GITHUB_APP_PRIVATE_KEY }} - owner: TipLink - repositories: fineas-centaur-infra - permission-contents: write - permission-pull-requests: write - - - name: Checkout Fineas infra - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 + - name: Download this run's runnable linux/arm64 child digests + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: - repository: TipLink/fineas-centaur-infra - token: ${{ steps.fineas_token.outputs.token }} - path: fineas-centaur-infra - persist-credentials: false + pattern: runnable-child-digests-*-linux-arm64 + path: ${{ runner.temp }}/arm64-runnable-child-digests - - name: Update Fineas Centaur pins - working-directory: fineas-centaur-infra - run: | - set -euo pipefail - bash scripts/bump-centaur-pins.sh "${GITHUB_SHA}" - scripts/audit-supply-chain.sh + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3 - - name: Open or update Fineas infra PR - id: promote + - name: Log in to GHCR + uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Write release descriptor env: - FINEAS_INFRA_TOKEN: ${{ steps.fineas_token.outputs.token }} - CENTAUR_TOKEN: ${{ github.token }} - CENTAUR_SHA: ${{ github.sha }} - CENTAUR_REPO: ${{ github.repository }} - CENTAUR_RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + RELEASE_REVISION: ${{ github.sha }} + ARM64_RUNNABLE_DIGEST_ROOT: ${{ runner.temp }}/arm64-runnable-child-digests run: | - python3 <<'PY' - from __future__ import annotations - - import base64 - import json - import os - import re - import subprocess - import urllib.error - import urllib.parse - import urllib.request - from pathlib import Path - - owner = "TipLink" - repo = "fineas-centaur-infra" - base_branch = "main" - root = Path("fineas-centaur-infra") - centaur_sha = os.environ["CENTAUR_SHA"] - centaur_short = centaur_sha[:7] - centaur_tag = f"sha-{centaur_short}" - branch = f"automation/promote-centaur-{centaur_short}" - message = f"chore: promote centaur {centaur_short}" - title = f"Promote Centaur {centaur_short} to Fineas" - - headers = { - "Authorization": f"Bearer {os.environ['FINEAS_INFRA_TOKEN']}", - "Accept": "application/vnd.github+json", - "X-GitHub-Api-Version": "2022-11-28", - "User-Agent": "centaur-promote-fineas", - } - api = f"https://api.github.com/repos/{owner}/{repo}" - centaur_headers = dict(headers) - centaur_headers["Authorization"] = f"Bearer {os.environ['CENTAUR_TOKEN']}" - - def request( - method: str, - url: str, - payload: dict | None = None, - *, - ok: tuple[int, ...] = (200, 201, 204), - request_headers: dict[str, str] | None = None, - ): - data = None - req_headers = dict(request_headers or headers) - if payload is not None: - data = json.dumps(payload).encode() - req_headers["Content-Type"] = "application/json" - req = urllib.request.Request(url, data=data, headers=req_headers, method=method) - try: - with urllib.request.urlopen(req) as resp: - body_text = resp.read().decode() - if resp.status not in ok: - raise RuntimeError(f"{method} {url} returned {resp.status}: {body_text}") - return json.loads(body_text) if body_text else None - except urllib.error.HTTPError as exc: - body_text = exc.read().decode() - if exc.code in ok: - if exc.code == 404: - return None - return json.loads(body_text) if body_text else None - raise RuntimeError(f"{method} {url} returned {exc.code}: {body_text}") from None - - def centaur_pr_url_for_commit() -> str: - repo_full = os.environ["CENTAUR_REPO"] - url = f"https://api.github.com/repos/{repo_full}/commits/{centaur_sha}/pulls" - try: - pulls = request("GET", url, request_headers=centaur_headers) - except RuntimeError as exc: - print(f"warning: could not look up Centaur PR for {centaur_sha}: {exc}") - return "" - - merged_pulls = [pull for pull in pulls if pull.get("merged_at")] - pull = (merged_pulls or pulls or [None])[0] - return pull["html_url"] if pull else "" - - def git(*args: str) -> str: - return subprocess.check_output(["git", "-C", str(root), *args], text=True) - - def set_application_annotation(key: str, value: str) -> bool: - application_path = root / "clusters/centaur-sandbox/argocd/applications/centaur-sandbox.yaml" - text = application_path.read_text(encoding="utf-8") - pattern = re.compile(rf"^ {re.escape(key)}: .*$", flags=re.MULTILINE) - - if value: - line = f" {key}: {json.dumps(value)}" - if pattern.search(text): - new_text = pattern.sub(line, text, count=1) - else: - marker = " argocd.argoproj.io/compare-options: ServerSideDiff=true\n" - if marker not in text: - raise RuntimeError(f"could not place Application annotation {key}") - new_text = text.replace(marker, f"{marker}{line}\n", 1) - else: - new_text = pattern.sub("", text) - new_text = re.sub(r"\n{3,}", "\n\n", new_text) - - if new_text == text: - return False - application_path.write_text(new_text, encoding="utf-8") - return True - - def put_file_to_branch(rel: str, encoded_branch: str) -> bool: - encoded_path = urllib.parse.quote(rel, safe="") - existing = request("GET", f"{api}/contents/{encoded_path}?ref={encoded_branch}", ok=(200, 404)) - content_bytes = (root / rel).read_bytes() - if existing: - current = base64.b64decode(existing["content"]).replace(b"\r\n", b"\n") - if current == content_bytes: - return False - - payload = { - "message": message, - "content": base64.b64encode(content_bytes).decode(), - "branch": branch, - } - if existing: - payload["sha"] = existing["sha"] - request("PUT", f"{api}/contents/{encoded_path}", payload) - return True - - centaur_pr_url = centaur_pr_url_for_commit() - source_lines = [ - f"- Centaur commit: https://github.com/{os.environ['CENTAUR_REPO']}/commit/{centaur_sha}", - f"- Image publish run: {os.environ['CENTAUR_RUN_URL']}", - ] - if centaur_pr_url: - source_lines.append(f"- Centaur PR: {centaur_pr_url}") - - body = f"""## Summary - - update Fineas Centaur base image pins to `{centaur_tag}` - - pin the Centaur chart source to `{centaur_sha}` - - ## Source - {chr(10).join(source_lines)} - - ## Tests - - `bash scripts/bump-centaur-pins.sh {centaur_sha}` - - `scripts/audit-supply-chain.sh` - """ - - status_lines = [line for line in git("status", "--porcelain").splitlines() if line] - changed_paths: list[tuple[str, str]] = [] - for line in status_lines: - status = line[:2] - path = line[3:] - if " -> " in path: - path = path.split(" -> ", 1)[1] - if status == "??": - kind = "A" - elif "D" in status: - kind = "D" - else: - kind = "M" - changed_paths.append((kind, path)) - - output_path = os.environ["GITHUB_OUTPUT"] - if not changed_paths: - with open(output_path, "a", encoding="utf-8") as output: - output.write("changed=false\n") - output.write(f"branch={branch}\n") - output.write("pr_url=\n") - raise SystemExit(0) - - base_ref = request("GET", f"{api}/git/ref/heads/{base_branch}") - base_sha = base_ref["object"]["sha"] - encoded_branch = urllib.parse.quote(branch, safe="") - try: - request("POST", f"{api}/git/refs", {"ref": f"refs/heads/{branch}", "sha": base_sha}) - except RuntimeError as exc: - if "Reference already exists" not in str(exc): - raise - - changed = False - for kind, rel in changed_paths: - encoded_path = urllib.parse.quote(rel, safe="") - existing = request("GET", f"{api}/contents/{encoded_path}?ref={encoded_branch}", ok=(200, 404)) - if kind == "D": - if existing: - request( - "DELETE", - f"{api}/contents/{encoded_path}", - {"message": message, "branch": branch, "sha": existing["sha"]}, - ) - changed = True - continue - - content_bytes = (root / rel).read_bytes() - if existing: - current = base64.b64decode(existing["content"]).replace(b"\r\n", b"\n") - if current == content_bytes: - continue - - payload = { - "message": message, - "content": base64.b64encode(content_bytes).decode(), - "branch": branch, - } - if existing: - payload["sha"] = existing["sha"] - request("PUT", f"{api}/contents/{encoded_path}", payload) - changed = True - - pulls = request( - "GET", - f"{api}/pulls?state=open&head={urllib.parse.quote(f'{owner}:{branch}', safe='')}", - ) - if pulls: - pr = request("PATCH", f"{api}/pulls/{pulls[0]['number']}", {"title": title, "body": body}) - elif changed: - pr = request("POST", f"{api}/pulls", {"title": title, "head": branch, "base": base_branch, "body": body}) - else: - pr = None - - if pr: - annotations_changed = False - annotations_changed |= set_application_annotation("fineas.dev/deployment-pr-url", pr["html_url"]) - annotations_changed |= set_application_annotation("fineas.dev/centaur-pr-url", centaur_pr_url) - if annotations_changed: - put_file_to_branch( - "clusters/centaur-sandbox/argocd/applications/centaur-sandbox.yaml", - encoded_branch, - ) - - with open(output_path, "a", encoding="utf-8") as output: - output.write(f"changed={'true' if changed else 'false'}\n") - output.write(f"branch={branch}\n") - output.write(f"centaur_short={centaur_short}\n") - if pr: - output.write(f"pr_url={pr['html_url']}\n") - output.write(f"pr_number={pr['number']}\n") - else: - output.write("pr_url=\n") - PY + set -euo pipefail + IFS=$'\n\t' + if [[ ! "$RELEASE_REVISION" =~ ^[0-9a-f]{40}$ ]]; then + echo "invalid release revision: $RELEASE_REVISION" >&2 + exit 1 + fi + tag="reviewed-${RELEASE_REVISION}" + printf 'component\trepository\ttag\tdigest\trevision\n' > centaur-release.tsv + while IFS=$'\t' read -r component image; do + repository="${{ env.REGISTRY }}/${{ env.IMAGE_NAMESPACE }}/${image}" + index_json="$(docker buildx imagetools inspect "${repository}:${tag}" --raw)" + mapfile -t arm64_digests < <(jq -r ' + [.manifests[] + | select(.platform.os == "linux" and .platform.architecture == "arm64") + | .digest] | .[]' <<<"$index_json") + if [[ "${#arm64_digests[@]}" -ne 1 ]]; then + echo "expected one runnable linux/arm64 manifest for $image; found ${#arm64_digests[@]}" >&2 + exit 1 + fi + digest="${arm64_digests[0]}" + if [[ ! "$digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "invalid arm64 digest for $image: $digest" >&2 + exit 1 + fi + artifact_dir="${ARM64_RUNNABLE_DIGEST_ROOT}/runnable-child-digests-${image}-linux-arm64" + mapfile -t run_child_digest_files < <(find "$artifact_dir" -maxdepth 1 -type f -print) + if [[ "${#run_child_digest_files[@]}" -ne 1 ]]; then + echo "expected one runnable arm64 child artifact for $image; found ${#run_child_digest_files[@]}" >&2 + exit 1 + fi + run_child_digest="sha256:$(basename "${run_child_digest_files[0]}")" + if [[ ! "$run_child_digest" =~ ^sha256:[0-9a-f]{64}$ ]]; then + echo "invalid runnable arm64 child artifact for $image: $run_child_digest" >&2 + exit 1 + fi + if [[ "$digest" != "$run_child_digest" ]]; then + echo "reviewed tag arm64 child for $image does not match this run: tag=$digest run_child=$run_child_digest" >&2 + exit 1 + fi + # Prove the platform child itself is pullable. The multi-arch tag + # points at an OCI index; Fineas deploys tag@child-digest and its + # live verifier compares that child imageID, never the tag index. + docker buildx imagetools inspect "${repository}@${digest}" >/dev/null + printf '%s\tghcr.io/tiplink/centaur/%s\t%s\t%s\t%s\n' \ + "$component" "$image" "$tag" "$digest" "$RELEASE_REVISION" \ + >> centaur-release.tsv + done <<'COMPONENTS' + api-rs centaur-api-rs + slackbotv2 centaur-slackbotv2 + sandbox centaur-agent + iron-proxy centaur-iron-proxy + console centaur-console + COMPONENTS + + - name: Upload release descriptor + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: centaur-linux-arm64-release-${{ github.sha }} + path: centaur-release.tsv + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/release-chart-publish.yml b/.github/workflows/release-chart-publish.yml new file mode 100644 index 000000000..a7bc8f363 --- /dev/null +++ b/.github/workflows/release-chart-publish.yml @@ -0,0 +1,70 @@ +name: Publish Chart + +"on": + workflow_dispatch: + inputs: + confirm_reviewed_main: + description: Confirm this publish is from reviewed main + required: true + type: boolean + +permissions: {} + +jobs: + release: + name: Publish reviewed chart + if: github.ref == 'refs/heads/main' && inputs.confirm_reviewed_main + permissions: + contents: write + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + fetch-depth: 0 + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Configure Git + run: | + git config user.name "$GITHUB_ACTOR" + git config user.email "$GITHUB_ACTOR@users.noreply.github.com" + + - name: Bootstrap gh-pages branch + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + if git ls-remote --exit-code --heads origin gh-pages >/dev/null 2>&1; then + echo "gh-pages branch already exists" + else + echo "Creating empty gh-pages branch" + git switch --orphan gh-pages + git commit --allow-empty -m "chore: initialize gh-pages" + git push origin gh-pages + git switch - + fi + + - name: Set up Helm + uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4 + with: + version: v4.1.1 + + - name: Add chart repositories + run: helm repo add onepassword https://1password.github.io/connect-helm-charts + + - name: Build chart dependencies + run: helm dependency build contrib/chart + + - name: Lint chart + run: helm lint contrib/chart + + - name: Test staged compatibility rendering + run: bash contrib/chart/tests/test_overlay_image_compat.sh + + - name: Run chart-releaser + uses: helm/chart-releaser-action@cae68fefc6b5f367a0275617c9f83181ba54714f # v1.7.0 + with: + skip_existing: true + charts_dir: contrib + env: + CR_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release-chart.yml b/.github/workflows/release-chart.yml index 76ed0e5e6..234b9ce8a 100644 --- a/.github/workflows/release-chart.yml +++ b/.github/workflows/release-chart.yml @@ -2,52 +2,28 @@ name: Release Chart "on": pull_request: - branches: - - main + branches: [main] paths: - - "contrib/chart/**" - - ".github/workflows/release-chart.yml" + - contrib/chart/** + - .github/workflows/release-chart.yml push: - branches: - - main + branches: [main] paths: - - "contrib/chart/**" - - ".github/workflows/release-chart.yml" + - contrib/chart/** + - .github/workflows/release-chart.yml -permissions: {} +permissions: + contents: read jobs: release: - permissions: - contents: write runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 with: fetch-depth: 0 - token: ${{ secrets.GITHUB_TOKEN }} - - - name: Configure Git - run: | - git config user.name "$GITHUB_ACTOR" - git config user.email "$GITHUB_ACTOR@users.noreply.github.com" - - - name: Bootstrap gh-pages branch - if: github.event_name == 'push' - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - set -e - if git ls-remote --exit-code --heads origin gh-pages >/dev/null 2>&1; then - echo "gh-pages branch already exists" - else - echo "Creating empty gh-pages branch" - git switch --orphan gh-pages - git commit --allow-empty -m "chore: initialize gh-pages" - git push origin gh-pages - git switch - - fi + persist-credentials: false - name: Set up Helm uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4 @@ -63,45 +39,32 @@ jobs: - name: Lint chart run: helm lint contrib/chart + - name: Test staged compatibility rendering + run: bash contrib/chart/tests/test_overlay_image_compat.sh + - name: Install yq uses: mikefarah/yq@5a7e72a743649b1b3a47d1a1d8214f3453173c51 # v4 - name: Validate chart version bump run: | - set -e - - CHART_YAML="contrib/chart/Chart.yaml" - - if [ "${GITHUB_EVENT_NAME}" = "pull_request" ]; then + set -euo pipefail + chart_yaml="contrib/chart/Chart.yaml" + if [ "$GITHUB_EVENT_NAME" = "pull_request" ]; then git fetch --no-tags origin "${GITHUB_BASE_REF}:refs/remotes/origin/${GITHUB_BASE_REF}" - BASE_REF="origin/${GITHUB_BASE_REF}" - CHANGED_FILES=$(git diff --name-only "${BASE_REF}...HEAD" 2>/dev/null || echo "") - PREVIOUS_VERSION=$(git show "${BASE_REF}:${CHART_YAML}" 2>/dev/null | yq '.version' 2>/dev/null || echo "") + base_ref="origin/${GITHUB_BASE_REF}" else - CHANGED_FILES=$(git diff --name-only HEAD~1 HEAD 2>/dev/null || echo "") - PREVIOUS_VERSION=$(git show HEAD~1:"$CHART_YAML" 2>/dev/null | yq '.version' 2>/dev/null || echo "") + base_ref="HEAD~1" fi - - if ! echo "$CHANGED_FILES" | grep -q "^contrib/chart/"; then + changed_files="$(git diff --name-only "${base_ref}...HEAD" 2>/dev/null || true)" + previous_version="$(git show "${base_ref}:${chart_yaml}" 2>/dev/null | yq '.version' 2>/dev/null || true)" + if ! grep -q '^contrib/chart/' <<<"$changed_files"; then echo "No chart changes detected" exit 0 fi - - CURRENT_VERSION=$(yq '.version' "$CHART_YAML") - - if [ -n "$PREVIOUS_VERSION" ] && [ "$CURRENT_VERSION" != "$PREVIOUS_VERSION" ]; then - echo "Version already bumped ($PREVIOUS_VERSION -> $CURRENT_VERSION)" + current_version="$(yq '.version' "$chart_yaml")" + if [ -n "$previous_version" ] && [ "$current_version" != "$previous_version" ]; then + echo "Version already bumped ($previous_version -> $current_version)" exit 0 fi - - echo "::error file=${CHART_YAML}::contrib/chart changed but chart version stayed at ${CURRENT_VERSION}. Bump .version in the PR." + echo "::error file=${chart_yaml}::contrib/chart changed but chart version stayed at ${current_version}. Bump .version in the PR." exit 1 - - - name: Run chart-releaser - if: github.event_name == 'push' - uses: helm/chart-releaser-action@cae68fefc6b5f367a0275617c9f83181ba54714f # v1.7.0 - with: - skip_existing: true - charts_dir: contrib - env: - CR_TOKEN: "${{ secrets.GITHUB_TOKEN }}" diff --git a/.github/workflows/upstream-pr-verify.yml b/.github/workflows/upstream-pr-verify.yml new file mode 100644 index 000000000..59353afe0 --- /dev/null +++ b/.github/workflows/upstream-pr-verify.yml @@ -0,0 +1,91 @@ +name: Verify Upstream Audit Head + +on: + pull_request: + types: [opened, reopened, synchronize, edited, ready_for_review] + +permissions: + contents: read + pull-requests: read + +jobs: + verify: + name: Verify recorded upstream SHA and signatures + if: >- + github.event.pull_request.base.repo.full_name == 'TipLink/centaur' && + github.event.pull_request.base.ref == 'main' && + github.event.pull_request.head.repo.full_name == 'paradigmxyz/centaur' && + github.event.pull_request.head.ref == 'main' + runs-on: ubuntu-latest + steps: + - name: Match the moving head to the audited body + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + PR_BODY: ${{ github.event.pull_request.body }} + run: | + set -euo pipefail + python3 <<'PY' + import os + import re + + body = os.environ.get("PR_BODY", "") + base = os.environ["BASE_SHA"] + head = os.environ["HEAD_SHA"] + sha_match = re.search(r"^- upstream HEAD: `([0-9a-f]{40})`$", body, re.MULTILINE) + base_match = re.search( + r"^- base HEAD at generation: `([0-9a-f]{40})`$", body, re.MULTILINE + ) + count_match = re.search( + r"^- GitHub-verified upstream-only commits: ([0-9]+)$", body, re.MULTILINE + ) + if sha_match is None or base_match is None or count_match is None: + raise SystemExit("audit PR body is missing a recorded SHA or verified count") + if base_match.group(1) != base: + raise SystemExit( + f"moving base {base} does not match audited body SHA {base_match.group(1)}" + ) + if sha_match.group(1) != head: + raise SystemExit( + f"moving upstream head {head} does not match audited body SHA {sha_match.group(1)}" + ) + print(f"EXPECTED_COUNT={count_match.group(1)}", file=open(os.environ["GITHUB_ENV"], "a")) + PY + + - name: Re-verify every current upstream-only commit + env: + BASE_SHA: ${{ github.event.pull_request.base.sha }} + GH_TOKEN: ${{ github.token }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + run: | + set -euo pipefail + repo="$RUNNER_TEMP/upstream-head-verification" + git init --quiet "$repo" + git -C "$repo" remote add base "https://github.com/${GITHUB_REPOSITORY}.git" + git -C "$repo" remote add upstream "https://github.com/paradigmxyz/centaur.git" + git -C "$repo" fetch --quiet --filter=blob:none --no-tags base "$BASE_SHA" + git -C "$repo" fetch --quiet --filter=blob:none --no-tags upstream "$HEAD_SHA" + mapfile -t commits < <(git -C "$repo" rev-list --reverse "${BASE_SHA}..${HEAD_SHA}") + if [[ "${#commits[@]}" -ne "$EXPECTED_COUNT" ]]; then + echo "PR commit count ${#commits[@]} does not match audited count $EXPECTED_COUNT" >&2 + exit 1 + fi + if [[ "${#commits[@]}" -eq 0 || "${commits[-1]}" != "$HEAD_SHA" ]]; then + echo "PR commit list does not terminate at the current upstream head" >&2 + exit 1 + fi + failed=() + for commit in "${commits[@]}"; do + verification="$(gh api "repos/paradigmxyz/centaur/commits/${commit}" \ + --jq '[.commit.verification.verified, .commit.verification.reason] | @tsv')" + verified="${verification%%$'\t'*}" + reason="${verification#*$'\t'}" + if [[ "$verified" != "true" ]]; then + failed+=("${commit}:${reason:-unknown}") + fi + done + if [[ "${#failed[@]}" -ne 0 ]]; then + printf 'unverified upstream commit(s):\n' >&2 + printf ' %s\n' "${failed[@]}" >&2 + exit 1 + fi diff --git a/.github/workflows/upstream-sync.yml b/.github/workflows/upstream-sync.yml index b85f12bd7..ec26180d1 100644 --- a/.github/workflows/upstream-sync.yml +++ b/.github/workflows/upstream-sync.yml @@ -1,4 +1,4 @@ -name: Weekly Upstream Sync +name: Weekly Upstream Audit on: schedule: @@ -7,7 +7,7 @@ on: workflow_dispatch: concurrency: - group: upstream-sync + group: upstream-audit cancel-in-progress: false permissions: @@ -15,16 +15,16 @@ permissions: env: BASE_BRANCH: main - SYNC_BRANCH: automation/upstream-sync UPSTREAM_BRANCH: main + UPSTREAM_OWNER: paradigmxyz UPSTREAM_REPO: paradigmxyz/centaur jobs: open-pr: - name: Open upstream sync PR + name: Open untrusted upstream audit PR runs-on: ubuntu-latest steps: - - name: Create Centaur GitHub App token + - name: Create pull-request-only GitHub App token id: app_token uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 with: @@ -32,34 +32,34 @@ jobs: private-key: ${{ secrets.FINEAS_GITHUB_APP_PRIVATE_KEY }} owner: TipLink repositories: centaur - permission-contents: write permission-pull-requests: write - - name: Checkout + - name: Checkout trusted base uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 with: - token: ${{ steps.app_token.outputs.token }} fetch-depth: 0 persist-credentials: false - - name: Update upstream sync branch - id: sync + - name: Audit upstream ancestry and signatures + id: audit env: - APP_TOKEN: ${{ steps.app_token.outputs.token }} - GH_TOKEN: ${{ steps.app_token.outputs.token }} + GH_TOKEN: ${{ github.token }} run: | set -euo pipefail + parent="$(gh api "repos/${GITHUB_REPOSITORY}" --jq '.parent.full_name // empty')" + if [[ "$parent" != "$UPSTREAM_REPO" ]]; then + echo "expected $GITHUB_REPOSITORY to remain a fork of $UPSTREAM_REPO; found ${parent:-none}" >&2 + exit 1 + fi + git remote add upstream "https://github.com/${UPSTREAM_REPO}.git" - git fetch --no-tags \ - "https://x-access-token:${APP_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \ - "${BASE_BRANCH}:refs/remotes/origin/${BASE_BRANCH}" - git fetch --no-tags upstream "${UPSTREAM_BRANCH}" + git fetch --no-tags origin "${BASE_BRANCH}:refs/remotes/origin/${BASE_BRANCH}" + git fetch --no-tags upstream "${UPSTREAM_BRANCH}:refs/remotes/upstream/${UPSTREAM_BRANCH}" base_sha="$(git rev-parse "origin/${BASE_BRANCH}")" upstream_sha="$(git rev-parse "upstream/${UPSTREAM_BRANCH}")" merge_base="$(git merge-base "$base_sha" "$upstream_sha")" - { echo "base_sha=$base_sha" echo "upstream_sha=$upstream_sha" @@ -68,33 +68,55 @@ jobs: if git merge-base --is-ancestor "$upstream_sha" "$base_sha"; then echo "changed=false" >> "$GITHUB_OUTPUT" - existing_pr="$(gh pr list \ - --repo "$GITHUB_REPOSITORY" \ - --head "$SYNC_BRANCH" \ - --base "$BASE_BRANCH" \ - --state open \ - --json number \ - --jq '.[0].number // empty')" - if [ -n "$existing_pr" ]; then - gh pr close "$existing_pr" \ - --repo "$GITHUB_REPOSITORY" \ - --comment "Closing because ${UPSTREAM_REPO}/${UPSTREAM_BRANCH} is already contained in ${BASE_BRANCH}." - fi + echo "verified_commit_count=0" >> "$GITHUB_OUTPUT" exit 0 fi - git push --force \ - "https://x-access-token:${APP_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" \ - "$upstream_sha:refs/heads/${SYNC_BRANCH}" + mapfile -t commits < <(git rev-list --reverse "${base_sha}..${upstream_sha}") + if [[ "${#commits[@]}" -eq 0 ]]; then + echo "upstream is not contained, but no upstream-only commits were found" >&2 + exit 1 + fi + failed=() + for commit in "${commits[@]}"; do + verification="$(gh api "repos/${UPSTREAM_REPO}/commits/${commit}" \ + --jq '[.commit.verification.verified, .commit.verification.reason] | @tsv')" + verified="${verification%%$'\t'*}" + reason="${verification#*$'\t'}" + if [[ "$verified" != "true" ]]; then + failed+=("${commit}:${reason:-unknown}") + fi + done + if [[ "${#failed[@]}" -ne 0 ]]; then + printf 'refusing unverified upstream commit(s):\n' >&2 + printf ' %s\n' "${failed[@]}" >&2 + exit 1 + fi echo "changed=true" >> "$GITHUB_OUTPUT" + echo "verified_commit_count=${#commits[@]}" >> "$GITHUB_OUTPUT" - - name: Build PR body - if: steps.sync.outputs.changed == 'true' + - name: Close stale audit PR when already contained + if: steps.audit.outputs.changed == 'false' env: - BASE_SHA: ${{ steps.sync.outputs.base_sha }} - MERGE_BASE: ${{ steps.sync.outputs.merge_base }} - UPSTREAM_SHA: ${{ steps.sync.outputs.upstream_sha }} + GH_TOKEN: ${{ steps.app_token.outputs.token }} + run: | + set -euo pipefail + existing_pr="$(gh api \ + "repos/${GITHUB_REPOSITORY}/pulls?state=open&base=${BASE_BRANCH}&head=${UPSTREAM_OWNER}:${UPSTREAM_BRANCH}&per_page=100" \ + --jq '.[0].number // empty')" + if [[ -n "$existing_pr" ]]; then + gh pr close "$existing_pr" --repo "$GITHUB_REPOSITORY" \ + --comment "Closing because ${UPSTREAM_REPO}/${UPSTREAM_BRANCH} is already contained in ${BASE_BRANCH}." + fi + + - name: Build audit PR body + if: steps.audit.outputs.changed == 'true' + env: + BASE_SHA: ${{ steps.audit.outputs.base_sha }} + MERGE_BASE: ${{ steps.audit.outputs.merge_base }} + UPSTREAM_SHA: ${{ steps.audit.outputs.upstream_sha }} + VERIFIED_COMMIT_COUNT: ${{ steps.audit.outputs.verified_commit_count }} run: | set -euo pipefail @@ -110,22 +132,19 @@ jobs: base_sha = os.environ["BASE_SHA"] merge_base = os.environ["MERGE_BASE"] upstream_sha = os.environ["UPSTREAM_SHA"] + verified_count = int(os.environ["VERIFIED_COMMIT_COUNT"]) base_branch = os.environ["BASE_BRANCH"] - sync_branch = os.environ["SYNC_BRANCH"] upstream_branch = os.environ["UPSTREAM_BRANCH"] + upstream_owner = os.environ["UPSTREAM_OWNER"] upstream_repo = os.environ["UPSTREAM_REPO"] runner_temp = Path(os.environ["RUNNER_TEMP"]) - body_path = runner_temp / "upstream-sync-body.md" + body_path = runner_temp / "upstream-audit-body.md" def git(*args: str, cwd: str | Path = ".") -> str: - return subprocess.check_output( - ["git", *args], - cwd=cwd, - text=True, - ).strip() + return subprocess.check_output(["git", *args], cwd=cwd, text=True).strip() - def lines(command: list[str]) -> list[str]: - output = git(*command) + def lines(args: list[str]) -> list[str]: + output = git(*args) return [line for line in output.splitlines() if line.strip()] commit_lines = lines( @@ -133,7 +152,6 @@ jobs: ) file_rows = lines(["diff", "--name-status", f"{merge_base}..{upstream_sha}"]) file_names = lines(["diff", "--name-only", f"{merge_base}..{upstream_sha}"]) - risk_prefixes = ( ".github/workflows/", "contrib/chart/", @@ -148,7 +166,7 @@ jobs: if path.startswith(risk_prefixes) or any(term in path.lower() for term in risk_terms) ] - probe_dir = Path(tempfile.mkdtemp(prefix="upstream-sync-merge-", dir=runner_temp)) + probe_dir = Path(tempfile.mkdtemp(prefix="upstream-audit-merge-", dir=runner_temp)) conflict_files: list[str] = [] merge_failed = False try: @@ -193,7 +211,6 @@ jobs: risky = [f"- `{path}`" for path in risk_files] conflicts = [f"- `{path}`" for path in conflict_files] compare_url = f"https://github.com/{upstream_repo}/compare/{merge_base}...{upstream_sha}" - if conflict_files: merge_probe = f"Local merge probe found {len(conflict_files)} conflicting file(s)." elif merge_failed: @@ -202,14 +219,20 @@ jobs: merge_probe = "No local merge conflicts detected." body = f"""## Summary - - sync `{upstream_repo}/{upstream_branch}` into `{base_branch}` + - compare `{upstream_repo}/{upstream_branch}` with `{base_branch}` - upstream HEAD: `{upstream_sha}` - base HEAD at generation: `{base_sha}` - - upstream commits not yet in `{base_branch}`: {len(commit_lines)} + - GitHub-verified upstream-only commits: {verified_count} - files changed upstream since merge-base: {len(file_rows)} - merge probe: {merge_probe} - This PR is intentionally human-in-the-loop. Merge it with a normal merge commit so TipLink-specific history stays intact. + This is a draft, untrusted cross-repository audit PR directly from + `{upstream_owner}:{upstream_branch}`. It deliberately does not copy upstream code + into a same-repository branch, so pull-request workflows receive the external-head + token/secret boundary before human review. Do not merge it directly when compatibility + work is required; create a reviewed integration branch pinned to the recorded upstream + SHA and preserve both histories. The moving PR head is not an integration input unless + the `Verify recorded upstream SHA and signatures` check matches this body and is green. ## Upstream Compare {compare_url} @@ -226,51 +249,45 @@ jobs: ## Conflicts {limited(conflicts, limit=40)} - ## After Merge - - `publish-images.yml` should publish new `ghcr.io/tiplink/centaur/*` images. - - The existing promotion job should open or update the Fineas infra PR that bumps deployed image/chart pins. - - Merge the infra PR after reviewing the pins, then let Argo deploy. + ## Required integration audit + - Treat this PR as an upstream-delta signal, never deployment authorization. + - Classify every fork patch as retained, superseded upstream, migrated, or dropped with evidence. + - Re-run migration lineage, security-boundary, overlay, rollback-bridge, and immutable-descriptor gates. + - Roll out only through the reviewed Fineas infra PR DAG and operator acceptance stages. - Generated by the weekly upstream sync workflow from `{sync_branch}`. + Generated by the weekly upstream audit workflow. """ body_path.write_text("\n".join(line.strip() for line in body.splitlines()) + "\n") - print(body_path) PY - - name: Open or update PR - if: steps.sync.outputs.changed == 'true' + - name: Open or update untrusted audit PR + if: steps.audit.outputs.changed == 'true' env: GH_TOKEN: ${{ steps.app_token.outputs.token }} run: | set -euo pipefail - - title="chore: sync upstream Centaur" - body_file="${RUNNER_TEMP}/upstream-sync-body.md" - existing_pr="$(gh pr list \ - --repo "$GITHUB_REPOSITORY" \ - --head "$SYNC_BRANCH" \ - --base "$BASE_BRANCH" \ - --state open \ - --json number \ + body_file="${RUNNER_TEMP}/upstream-audit-body.md" + existing_pr="$(gh api \ + "repos/${GITHUB_REPOSITORY}/pulls?state=open&base=${BASE_BRANCH}&head=${UPSTREAM_OWNER}:${UPSTREAM_BRANCH}&per_page=100" \ --jq '.[0].number // empty')" - - if [ -n "$existing_pr" ]; then - gh pr edit "$existing_pr" \ - --repo "$GITHUB_REPOSITORY" \ - --title "$title" \ + if [[ -n "$existing_pr" ]]; then + gh pr edit "$existing_pr" --repo "$GITHUB_REPOSITORY" \ + --title "chore: audit current upstream Centaur" \ --body-file "$body_file" pr_url="$(gh pr view "$existing_pr" --repo "$GITHUB_REPOSITORY" --json url --jq .url)" else - pr_url="$(gh pr create \ - --repo "$GITHUB_REPOSITORY" \ - --base "$BASE_BRANCH" \ - --head "$SYNC_BRANCH" \ - --title "$title" \ - --body-file "$body_file")" + jq -n \ + --arg title "chore: audit current upstream Centaur" \ + --arg head "${UPSTREAM_OWNER}:${UPSTREAM_BRANCH}" \ + --arg base "$BASE_BRANCH" \ + --rawfile body "$body_file" \ + '{title: $title, head: $head, base: $base, body: $body, draft: true}' \ + >"${RUNNER_TEMP}/create-upstream-pr.json" + pr_url="$(gh api --method POST "repos/${GITHUB_REPOSITORY}/pulls" \ + --input "${RUNNER_TEMP}/create-upstream-pr.json" --jq .html_url)" fi - { - echo "### Upstream sync PR" + echo "### Upstream audit PR" echo echo "$pr_url" } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/validate-images.yml b/.github/workflows/validate-images.yml new file mode 100644 index 000000000..4b20df160 --- /dev/null +++ b/.github/workflows/validate-images.yml @@ -0,0 +1,150 @@ +name: Publish Images + +on: + pull_request: + branches: [main] + paths: + - .github/workflows/validate-images.yml + - .github/workflows/publish-images.yml + - .github/scripts/resolve-runnable-image-digest.sh + - .github/scripts/verify-registry-tag-absent.sh + - .github/scripts/verify-reviewed-image-release.sh + - .github/scripts/test-resolve-runnable-image-digest.sh + - .github/scripts/test-verify-registry-tag-absent.sh + - .github/scripts/test-verify-reviewed-image-release.sh + - services/** + - crates/harness-server/** + - harness/** + - centaur_sdk/** + - packages/** + - tools/** + - workflows/** + - scripts/bootstrap-k8s-secrets.sh + - scripts/probe-agent-harness-image.sh + - package.json + - pnpm-lock.yaml + - pnpm-workspace.yaml + - .agents/skills/** + +concurrency: + group: validate-images-${{ github.ref }} + cancel-in-progress: true + +permissions: + contents: read + +env: + RUST_BUILD_PROFILE: debug + +jobs: + build: + runs-on: ${{ matrix.platform == 'linux/arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }} + strategy: + fail-fast: false + matrix: + service: [api-rs, slackbotv2, linearbot, discordbot, githubbot, teamsbot, agent, iron-proxy, console] + platform: [linux/amd64] + include: + - service: api-rs + image: centaur-api-rs + context: . + dockerfile: services/api-rs/Dockerfile + target: "" + - service: slackbotv2 + image: centaur-slackbotv2 + context: . + dockerfile: services/slackbotv2/Dockerfile + target: "" + - service: linearbot + image: centaur-linearbot + context: . + dockerfile: services/linearbot/Dockerfile + target: "" + - service: discordbot + image: centaur-discordbot + context: . + dockerfile: services/discordbot/Dockerfile + target: "" + - service: githubbot + image: centaur-githubbot + context: . + dockerfile: services/githubbot/Dockerfile + target: "" + - service: teamsbot + image: centaur-teamsbot + context: . + dockerfile: services/teamsbot/Dockerfile + target: "" + - service: agent + image: centaur-agent + context: . + dockerfile: services/sandbox/Dockerfile + target: sandbox + - service: agent + platform: linux/arm64 + image: centaur-agent + context: . + dockerfile: services/sandbox/Dockerfile + target: sandbox + - service: iron-proxy + image: centaur-iron-proxy + context: . + dockerfile: services/iron-proxy/Dockerfile + target: "" + - service: console + image: centaur-console + context: services/console + dockerfile: services/console/Dockerfile + target: "" + + steps: + - name: Checkout + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + persist-credentials: false + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3 + + - name: Derive platform slug + run: | + set -euo pipefail + platform="${{ matrix.platform }}" + echo "PLATFORM_SLUG=${platform//\//-}" >> "$GITHUB_ENV" + + - name: Build ${{ matrix.image }} without registry credentials + uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6 + with: + context: ${{ matrix.context }} + file: ${{ matrix.dockerfile }} + target: ${{ matrix.target }} + platforms: ${{ matrix.platform }} + push: false + load: ${{ matrix.service == 'agent' }} + tags: ${{ matrix.service == 'agent' && format('{0}:validate-{1}', matrix.image, env.PLATFORM_SLUG) || '' }} + build-args: | + RUST_BUILD_PROFILE=${{ env.RUST_BUILD_PROFILE }} + cache-from: type=gha,scope=validate-${{ matrix.image }}-${{ env.PLATFORM_SLUG }} + cache-to: type=gha,mode=max,scope=validate-${{ matrix.image }}-${{ env.PLATFORM_SLUG }} + + - name: Probe packaged agent harness + if: matrix.service == 'agent' + env: + AGENT_IMAGE: ${{ matrix.image }}:validate-${{ env.PLATFORM_SLUG }} + run: bash scripts/probe-agent-harness-image.sh "$AGENT_IMAGE" + + image-validation-success: + name: Image validation success + runs-on: ubuntu-latest + if: always() + needs: build + steps: + - name: Require every image validation build to succeed + env: + BUILD_RESULT: ${{ needs.build.result }} + run: | + set -euo pipefail + if [[ "$BUILD_RESULT" != "success" ]]; then + echo "image validation matrix result is $BUILD_RESULT; expected success" >&2 + exit 1 + fi diff --git a/.gitignore b/.gitignore index 0c7a9429e..dc9537c0d 100644 --- a/.gitignore +++ b/.gitignore @@ -13,6 +13,7 @@ venv/ # uv uv.lock +!services/workflow-python/uv.lock # Testing .pytest_cache/ diff --git a/AGENTS.md b/AGENTS.md index b508ea59d..03b5514fe 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,832 +1,172 @@ -# Centaur — Developer Guide - -## Quick Start - -### 1. Clone and configure - -```bash -git clone -cd centaur -brew install just -``` - -Centaur runs locally on Kubernetes through the Helm chart. Infra secrets are required as pre-created Kubernetes Secrets. For local development, `just bootstrap-secrets` creates them from your shell environment: - -```bash -export OP_SERVICE_ACCOUNT_TOKEN=... -export OP_VAULT=... -export SLACK_BOT_TOKEN=... -export SLACK_SIGNING_SECRET=... -export SLACKBOT_API_KEY=... -``` - -Application-level LLM/tool secrets such as OpenAI and Anthropic tokens stay in 1Password and are loaded by the secrets service. - -### 2. Boot the stack - -```bash -just up -``` - -### Database migrations - -api-rs embeds SQLx migrations from -`services/api-rs/crates/centaur-session-sqlx/migrations`. To add schema, create -the next numbered SQL file in that directory and keep it compatible with the -embedded migrator. The api-rs binary applies those migrations on startup when -the chart enables migration running, and the Rust tests use the same migration -set for database-backed coverage. - -### 3. Test - -From inside the API deployment (localhost bypass — no key needed): - -```bash -THREAD_KEY=cli:test-e2e-1 -THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') - -SESSION=$(kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}" \ - -H "Content-Type: application/json" \ - -d '{"harness_type":"codex","on_harness_conflict":"restart"}') - -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/messages" \ - -H "Content-Type: application/json" \ - -d '{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG and nothing else."}]}]}' - -EXECUTE=$(kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/execute" \ - -H "Content-Type: application/json" \ - -d '{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG and nothing else.\"}]}}"]}') -EXECUTION_ID=$(printf '%s' "$EXECUTE" | jq -r '.execution_id') - -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -N \ - "http://localhost:8080/api/session/${THREAD_PATH}/events?execution_id=${EXECUTION_ID}&after_event_id=0" -``` - -Or use the deployment's configured service bearer token path for external -clients (see [API Key Management](#api-key-management)). - -## Architecture - -See the [architecture diagram in the README](README.md#architecture). - -### End-to-End Request Flow - -1. User mentions bot in Slack → webhook → slackbotv2 → api-rs -2. api-rs spawns/reuses a Kubernetes sandbox pod (`centaur-agent:latest`) for that thread -3. Executes harness (amp/claude-code/codex) through the sandbox backend -4. Harness calls local tool CLI shims installed by `centaur-tools` (NOT MCP) -5. LLM/API calls route through per-sandbox iron-proxy which injects real credentials -6. Results stream as JSON events → posted to Slack - -### Service Interface Contracts - -Centaur is a modular service architecture. Each service communicates through well-defined interfaces. As long as you implement these interfaces, you can swap or extend any layer independently. - -**Client → API** (durable control-plane protocol): - -Clients (slackbotv2, CLI, external integrations) should stay thin. They create -or reuse a session, append durable messages, execute the session, and stream or -replay output from the durable event endpoint. api-rs owns runtime assignment, -execution serialization, cancellation/recovery, and final delivery; Postgres is -the source of truth. - -Thread keys are path parameters on the api-rs session routes, so callers must -URL-encode values such as `slack:T123:C456:1773364194.179929`. - -**Step 1: Assign or reuse a session** (`POST /api/session/{thread_key}`) - -Creates a session for the thread, or returns the current one. - -``` -POST /api/session/slack%3AT123%3AC456%3A1773364194.179929 -{ - "harness_type": "codex", - "persona_id": "incident-responder", - "metadata": {"platform": "slack"}, - "on_harness_conflict": "reject" -} - -← { - "thread_key": "slack:T123:C456:1773364194.179929", - "sandbox_id": "sbx_123", - "harness_type": "codex", - "status": "active", - "harness_switched": false - } -``` - -**Step 2: Persist the user turn** (`POST /api/session/{thread_key}/messages`) - -Writes one or more durable transcript messages. Parts use the same -Anthropic-style content block shape the sandbox adapter understands. - -``` -POST /api/session/slack%3AT123%3AC456%3A1773364194.179929/messages -{ - "messages": [ - { - "client_message_id": "slack-evt-123", - "role": "user", - "parts": [{"type": "text", "text": "analyze this"}], - "metadata": {"user_name": "alice", "platform": "slack"} - } - ] -} - -← {"ok": true, "message_ids": ["msg_123"]} -``` - -**Step 3: Execute the session** (`POST /api/session/{thread_key}/execute`) - -Creates a durable execution row and drives the attached sandbox. `input_lines` -are NDJSON strings sent to the harness adapter for this execution. - -``` -POST /api/session/slack%3AT123%3AC456%3A1773364194.179929/execute -{ - "idempotency_key": "slack-delivery-123", - "metadata": {"platform": "slack"}, - "input_lines": [ - "{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"analyze this\"}]}}" - ] -} - -← { - "ok": true, - "execution_id": "exe_123", - "thread_key": "slack:T123:C456:1773364194.179929", - "status": "queued" - } -``` - -**Step 4: Stream or replay output** (`GET /api/session/{thread_key}/events`) - -Consumers tail durable events for one execution. On disconnect, reconnect with -the last seen event id. - -``` -GET /api/session/slack%3AT123%3AC456%3A1773364194.179929/events?execution_id=exe_123&after_event_id=0 - -← SSE event: session.output.line -← data: {"type":"assistant","message":{...}} -← SSE event: session.execution_completed -← data: {"status":"completed","result_text":"..."} -``` - -**Inspect the active session context** (`GET /api/session/{thread_key}`) - -Returns the normalized session context for a thread, including Slack channel -and thread timestamp information when the thread key is Slack-shaped. - -**Durable state written for one turn:** - -| Table | What | -|-------|------| -| `sessions` | Thread-to-sandbox assignment, harness, persona, and status | -| `session_messages` | Durable transcript messages | -| `session_executions` | Queued/running/terminal execution rows | -| `session_events` | Replayable execution, output, and status events | -| `session_warm_sandboxes` | SQL-backed warm-pool inventory and claims | - -**API → Sandbox** (stdin/stdout, NDJSON): - -api-rs communicates with sandbox Pods through the active sandbox backend's -attach stream. Execution `input_lines` are opaque newline-delimited strings at -the session API layer; api-rs validates that each item is one line, adds -session/trace context to JSON objects, writes them to sandbox stdin, and stores -each stdout line as a durable `session.output.line` event. Current chat clients -send Codex-compatible user lines shaped like: - -``` -→ stdin: {"type":"user", - "thread_key":"slack:T123:C456:1773364194.179929", - "message":{ - "role":"user", - "content":[ - {"type":"text","text":"what is this?"}, - {"type":"image","source":{"type":"base64","media_type":"image/png","data":"..."}} - ] - ]} - -← stdout: {"type":"system","subtype":"init","session_id":"T-..."} -← stdout: {"type":"assistant","message":{"role":"assistant","content":[...]}} -← stdout: {"type":"result","subtype":"success","result":"..."} -← stdout: {"type":"turn.completed","turn_id":"turn-1","result":"..."} -``` - -**Harness adapter behavior**: - -The sandbox/runtime layer translates the user input content into whatever each -harness CLI actually accepts: - -| Harness | Translation | -|---------|-------------| -| **claude-code** | Pass through directly (native Anthropic format) | -| **amp** | Materialize image/document blocks to files on disk, replace with `@/path` text mentions (Amp stdin only accepts text blocks) | -| **codex / pi-mono** | Extract text from content blocks, pass as CLI argument | - -This means clients and api-rs avoid most harness-specific quirks. Clients send -durable messages plus the execution input lines they want delivered; the -sandbox/runtime adapter handles the target harness. - -**Sandbox tools and API callbacks**: - -Agent sandboxes do not use legacy HTTP tool-method routes as a registry. -Startup runs `services/sandbox/install_tool_shims.py`, which scans `TOOL_DIRS` for -`pyproject.toml [project.scripts]`, installs each script with `uvx`, and emits -the local `centaur-tools` catalog. Agents call tool CLIs directly; Python -workflow hosts can use the generated `centaur-tools call` bridge for -`ctx.call_tool(...)` compatibility. - -### Network Isolation - -The Helm chart installs deny-by-default NetworkPolicies, then explicitly allows -the service paths the stack needs: chat ingress services to api-rs, api-rs to -Postgres/iron-control/Kubernetes, sandbox Pods to api-rs/iron-proxy, DNS, and -configured egress. - -## Directory Structure - -``` -centaur/ -├── services/ -│ ├── api-rs/ # Rust control plane, sessions, workflows, auth, metrics -│ │ ├── crates/centaur-api-server/ -│ │ ├── crates/centaur-session-runtime/ -│ │ ├── crates/centaur-session-sqlx/ -│ │ ├── crates/centaur-workflows/ -│ │ └── crates/centaur-perms/ -│ ├── workflow-python/ # Python workflow host compatibility runtime -│ ├── iron-proxy/ # Credential injection proxy -│ ├── sandbox/ # Agent container image (Ubuntu 24.04 + uv + gh + node + bun + amp) -│ ├── slackbotv2/ # Slack event handling and Slack delivery -│ ├── teamsbot/ # Teams ingress -│ ├── discordbot/ # Discord ingress -│ ├── linearbot/ # Linear ingress -│ └── console/ # Admin/operator console -├── centaur_sdk/ # Standalone SDK (pip install centaur-sdk) -├── tools/ # Open-source tool plugins (auto-discovered) -│ ├── alchemy/ # One directory per tool — each has client.py + pyproject.toml -│ ├── websearch/ -│ ├── telegram/ -│ └── … # 60+ tool plugins (crypto, research, productivity, infra, …) -├── workflows/ # External workflow definitions (auto-discovered) -│ ├── agent_loop.py # Recurring agent polling/monitoring loop -│ └── multi_step_demo.py # Demo: branching, loops, conditionals -├── scripts/ # Operational scripts -└── Justfile # Local Helm/Kubernetes workflow -``` - -## Terminology - -- **Chat SDK** always refers to the [Vercel Chat SDK](https://github.com/vercel/chat) (`~/github/vercel/chat`). When you need to understand how the Chat SDK or `@chat-adapter/*` packages work, **always read the source at `~/github/vercel/chat`** — never dig through `node_modules`. - -## Testing Before Pushing - -**NEVER push changes without testing them locally first.** Testing means actually running the affected service and proving the change works end-to-end — not just linting or reasoning about it. - -1. **Build the affected service:** `just build-one ` -2. **Bring it up:** `just deploy` -3. **Make a real request** that exercises the change and show the output -4. **Only then** commit and push - -For tool changes: verify from a real sandbox with `centaur-tools list`, -` --help`, and a command that exercises the changed behavior. If the -change is only for workflow `ctx.call_tool(...)`, run a small workflow-host -workflow that calls it. For Dockerfile/infra changes: rebuild, redeploy, and -verify the binary/service is present and functional. For proxy changes: test -from inside a sandbox pod through iron-proxy. - -## Local-First Testing — Never Touch the Deploy Box - -**All testing and E2E validation MUST happen on the local Kubernetes stack** (`just up` on this machine). -The deploy box is **production**. Changes reach it via `git push` → GitHub Actions auto-deploy. The only reasons to SSH into it are: -- Checking logs (`kubectl logs`, VictoriaLogs queries) for debugging production issues -- Emergency manual intervention — **only when the user explicitly asks** - -For E2E testing, always: -1. `just build-one ` locally -2. `just deploy` locally -3. Run curl commands against `localhost` through `kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl ...` -4. Verify results locally -5. Only then commit, push, and let CI/CD handle production - -## Code Conventions - -- Python 3.11+, `uv` for deps, `ruff` for lint/format (line-length=100) -- `services/slackbotv2` uses `pnpm` only (single lockfile: `pnpm-lock.yaml`) -- All imports at top of file, never inside functions -- Absolute imports only: `from api.X`, `from centaur_sdk.X` -- All secrets via env vars or secret manager, never hardcode -- `asyncpg` for Postgres, `pgvector` for embeddings -- Conventional commits: `feat:`, `fix:`, `docs:`, `refactor:`, `test:`, `chore:` - -## Lint & Test - -Each service has its own `pyproject.toml` and `ruff.toml`. From the repo root: - -```bash -uv run ruff check . # lint -uv run ruff format . # auto-fix -uv run pytest # tests -``` - -## Plugin System — Tools & Workflows - -Centaur has two plugin types that are auto-discovered at startup and hot-reloaded on file changes — no core code changes required to extend the system. - -### Tool Plugins - -Tools live in directories listed by `TOOL_DIRS` and ordered overlay sources. -Each tool is a directory with `client.py` (class + `_client()` factory), -`pyproject.toml`, and a CLI entry point exposed through `[project.scripts]`. -api-rs discovers tool metadata for secret grants; sandboxes install the -scripts as local CLI shims and list them with `centaur-tools list`. - -- `client.py`: NO `load_dotenv()`. Secrets via `secret()` from `centaur_sdk.tool_sdk`. -- `cli.py`: YES `load_dotenv()` at top. Thin typer wrapper for standalone use. -- Methods starting with `_` are excluded from registration. -- Tool dependencies declared in `pyproject.toml` are installed by the shim - runner when the script is installed. -- `[project.scripts]` is required for an agent-visible runtime tool. - -Example: - -```python -# tools/my-tool/client.py -import httpx - -class MyToolClient: - def search(self, query: str, limit: int = 10) -> dict: - """Search for something.""" - resp = httpx.get(f"https://api.example.com/search?q={query}&limit={limit}") - return resp.json() - -def _client(): - return MyToolClient() -``` - -### Workflow Plugins - -Workflows live in directories listed in the `WORKFLOW_DIRS` env var -(colon-separated paths). api-rs discovers workflow metadata through the Python -workflow host, and workflow-host sandboxes receive the same ordered list -translated to sandbox mount paths. Each workflow is a Python file exporting -`WORKFLOW_NAME`, an async `handler(params, ctx)`, and an optional `Input` -dataclass. See [Durable Workflows](#durable-workflows) for the full programming -model. - -Built-in workflows ship in the top-level `workflows/` tree. External workflows -are loaded identically — add their directories to `WORKFLOW_DIRS` through the -ordered overlay configuration. - -### Ordered Overlays - -Centaur supports a first-class ordered repo-cache overlay model, so -organizations can extend the base repo without forking or relying on filesystem -overlayfs. A common deployment keeps the base repo and an external overlay -checkout side by side: - -``` -your-deployment/ -├── centaur/ # This repo -└── centaur-overlay/ # Org-specific tools, workflows, skills, personas, prompt overlay -``` - -The Helm chart supports ordered overlays through `overlays.sources`. -repo-cache checks out each source; api-rs reads tools and workflows from -`/var/lib/centaur/repos/...`, and sandboxes read the same revisions from -`/home/agent/github/...`. Use chart-level `overlay.systemPrompt` only for small -prompt additions. Existing deployments can continue using `overlay.image.*` -while they migrate remaining prompt, harness, and persona assets onto -repo-cache-backed overlay sources. - -Later overlay entries win cleanly when names collide, so the base repo stays generic while deployments can layer in org-specific behavior from outside the checkout. - -## Durable Workflows - -api-rs owns durable workflow state through Absurd queues, while -`services/workflow-python` runs Python workflow handlers in a workflow-host -sandbox. The Python compatibility layer exposes `WorkflowContext`, so the -handler function is still the workflow: steps are runtime-discovered via -`ctx.step(name, fn)`, checkpointed to Postgres, and skipped on replay after a -restart. Dynamic branching, loops, and conditional logic work naturally because -the handler remains Python. - -### WorkflowContext API - -Every handler receives `(params, ctx)` where `ctx: WorkflowContext` provides: - -| Primitive | Purpose | -|-----------|---------| -| `ctx.step(name, fn)` | Execute *fn* exactly once; return cached result on replay. | -| `ctx.sleep_for(name, seconds)` | Suspend the run for a duration; checkpoint + resume automatically. | -| `ctx.sleep_until(name, when)` | Suspend until a specific datetime. | -| `ctx.agent_turn(text, **kwargs)` / `ctx.run_agent(text, **kwargs)` | Start an agent turn and wait for the result. | -| `ctx.call_tool(tool, method, args)` | Call a tool through the workflow-host `centaur-tools call` bridge. | -| `ctx.post_to_slack(channel, text, **kwargs)` | Post to Slack through the api-rs Slack context path. | -| `ctx._pool` | Access the workflow database pool when the workflow-host sandbox receives `DATABASE_URL`. | -| `ctx.log(msg, **kwargs)` | Structured log, suppressed during replay. | - -### Writing a workflow - -```python -# workflows/my_workflow.py -from dataclasses import dataclass -from api.workflow_engine import WorkflowContext - -WORKFLOW_NAME = "my_workflow" - -@dataclass -class Input: - message: str = "hello" - -async def handler(inp: Input, ctx: WorkflowContext) -> dict[str, Any]: - greeting = await ctx.step("gather", lambda: {"msg": inp.message}) - await ctx.sleep_for("pause", 300) - result = await ctx.run_agent(f"Summarize: {greeting['msg']}") - return {"greeting": greeting, "agent_result": result} -``` - -### Workflow lifecycle - -Runs go through: `queued → running → sleeping/waiting → running → … → completed/failed/cancelled`. - -- **Worker pool**: `WORKFLOW_WORKER_CONCURRENCY` workers (default 2) poll for claimable runs. -- **Lease-based fencing**: Each worker holds a lease on its run, extended by a heartbeat. If the worker dies, the lease expires and another worker reclaims the run. -- **Schedules**: Cron-based or interval-based schedules are discovered from workflow metadata by `api-rs`. The scheduler stores tick tasks in the Absurd `centaur_workflow_schedules` queue. -- **External events and child workflows**: the API has run/event surfaces, but - the staged Python host should not advertise unsupported v1 context helpers. - -### Workflow REST API - -| Endpoint | Purpose | -|----------|---------| -| `POST /api/workflows/runs` | Create a workflow run (`workflow_name`, `input`, optional idempotency fields, `eager_start`) | -| `GET /api/workflows/runs` | List recent runs. | -| `GET /api/workflows/runs/{run_id}` | Get run details. | -| `POST /api/workflows/runs/{run_id}/cancel` | Cancel a run. | -| `POST /api/workflows/events` | Deliver an external event (`event_name`, `payload`). | -| `GET /api/workflows/schedules` | Inspect registered workflow schedules. | - -### Built-in workflows - -| Workflow | Description | -|----------|-------------| -| `echo` | Minimal smoke workflow. | -| `slack_sync`, `slack_backfill` | Slack ETL sync and backfill jobs. | -| `company_context_documents` | Projection from synced sources into retrieval documents. | -| `google_drive_sync`, `google_calendar_sync`, `linear_sync` | Optional connector sync workflows. | -| `github_issue_triage` | Example webhook-triggered triage flow. | - -### Durable state - -| Table | What | -|-------|------| -| `absurd.queues` | Registered workflow queues, including standard, ETL, backfill, Slack-live, and schedule queues | -| `absurd.t_centaur_workflows*` | Workflow task metadata, state, input params, idempotency keys, and completed payloads | -| `absurd.r_centaur_workflows*` | Per-attempt run state, leases, timing, result payloads, and failures | -| `absurd.c_centaur_workflows*` | Per-step checkpoint state | -| `absurd.e_centaur_workflows*` | Emitted workflow events for event-driven resumes | -| `absurd.w_centaur_workflows*` | Wait registrations for sleeps and external events | -| `absurd.t_centaur_workflow_schedules` | Scheduler tick tasks for registered cron and interval schedules | - -## Agent Sandbox - -### Overview - -1 conversation = 1 Kubernetes sandbox Pod. api-rs spawns Pods running harness -CLIs (amp, claude-code, codex), streams messages over the sandbox attach -channel, and records output in durable session events. - -### How the System Prompt Works - -The sandbox image bakes `services/sandbox/SYSTEM_PROMPT.md` into `~/AGENTS.md` at build time. On container startup, `entrypoint.sh` copies it into the workspace root as `workspace/AGENTS.md` — this is the file that AI harnesses (Amp, Claude Code, Codex) read as their system instructions. - -The system prompt tells the agent: -- **Identity**: it's running inside a Kubernetes sandbox pod managed by api-rs -- **Tools**: three kinds — harness built-ins (Read, Bash, etc.), tool plugins exposed as shell CLI shims, and a headless browser -- **Tool CLIs**: each tool is installed as a shell command at container startup by `services/sandbox/install_tool_shims.py`, which scans `TOOL_DIRS` for `pyproject.toml [project.scripts]` and `uvx`-installs each. Agents invoke tool CLIs directly (`slack get_channel_history '{"channel":"general"}'`, ` --help` to discover). -- **Slack messaging**: the agent's stdout IS the Slack reply — never call `send_message` on the active thread -- **Rules**: never display secrets, show your work, lead with the answer - -`centaur-tools` is the generated catalog CLI emitted by the same installer: -- `centaur-tools list` → list available tool CLIs -- `centaur-tools run [args]` → run a tool CLI -- `centaur-tools call [json]` → internal compatibility for the Python workflow host's `ctx.call_tool(...)` -- ` --help` → discover one tool's direct CLI - -### Persona System - -The entrypoint supports persona overlays via `AGENT_PERSONA`. Persona prompts are discovered from the loaded tool directories (including overlays such as `~/centaur-overlay`) and appended after the base + org overlay system prompts at container startup. - -### Sandbox Pod Config - -- Runs under Kubernetes NetworkPolicies with API reachable through the in-cluster service URL -- Entrypoint injects the runtime URLs and tool catalog environment needed by the sandbox -- Stub API keys so harnesses init in API-key mode (not browser login) -- `HTTPS_PROXY` routes LLM and tool egress through iron-proxy -- Resource limits: 4GB memory, 2 CPUs -- Image tagged `centaur-agent:latest` -- Labels identify Centaur-managed sandboxes and carry thread/harness metadata for discovery/recovery - -### Credential Injection (iron-proxy) - -Sandbox Pods never see real API keys. Per-sandbox `iron-proxy` pods inject -credentials from the configured secret source and iron-control grants: - -| Target host | Header | Format | -|-------------|--------|--------| -| `api.anthropic.com` | `x-api-key` | raw | -| `api.openai.com` | `authorization` | bearer | -| `openrouter.ai` | `authorization` | bearer | -| `ampcode.com` | `authorization` | bearer | -| `api.github.com` | `authorization` | token | -| `github.com` | `authorization` | basic auth | -| `bedrock-mantle..api.aws` | `authorization` + `x-amz-*` | AWS SigV4 re-sign (opt-in, codex `amazon-bedrock`) | - -### Session Persistence - -- **`sessions`** table: tracks thread key, sandbox ID, harness, persona, and state -- **`session_messages`** table: stores persisted user/assistant messages -- **`session_executions`** and **`session_events`** tables: store durable run state and replayable output -- On api-rs restart, sandbox ownership is re-read from Postgres; process-local attach pipes are rebuilt lazily per sandbox -- Pods are still discoverable via Kubernetes labels even if DB state needs reconciliation - -## Security Model - -- **API auth**: Chat ingress services use deployment-scoped bearer tokens such as `SLACKBOT_API_KEY`, `TEAMSBOT_API_KEY`, `DISCORDBOT_API_KEY`, or `LINEARBOT_API_KEY` when configured. Local in-cluster service calls use the internal api-rs service URL. -- **Sandbox auth**: Sandbox Pods use the runtime's tool and workflow surfaces; agents should not depend on a user-visible Centaur API key. -- **Slack**: HMAC-SHA256 signature verification on all webhooks -- **Public edge**: The Helm chart exposes public routes only when configured through Ingress, HTTPRoute, or service settings. -- **Sandbox isolation**: Pods get stub keys only; real keys are injected by iron-proxy in-flight -- **Filesystem**: Host repos mounted read-only by default; only working repo is read-write -- **Kubernetes API**: The API service account is scoped to the Pod, Secret, exec, attach, and log operations needed to manage sandboxes. - -## API Key Management - -Chat ingress services send bearer tokens from the local infra Secret when the -deployment configures them. The current api-rs control plane does not use the -legacy DB-backed API-key table or legacy key prefix for the session routes. - -### Key types - -| Type | Prefix | Issued by | Used by | Scopes | -|------|--------|-----------|---------|--------| -| Service bearer | deployment-specific | Kubernetes Secret / bootstrap | Slackbotv2, Teamsbot, Discordbot, Linearbot | Service-to-api-rs calls | - -### How services get their keys - -- **Slackbotv2**: `SLACK_BOT_TOKEN`, `SLACK_SIGNING_SECRET`, and `SLACKBOT_API_KEY` are injected from the local infra Secret. -- **Sandbox containers**: Use runtime-provided tool CLIs and workflow context rather than a direct Centaur API key -- **Local testing**: Exec into the api-rs deployment and use `localhost:8080`, or use a bot/service token path configured for that deployment - -## Secrets - -Tool credentials (e.g., `ANTHROPIC_API_KEY`, `AMP_API_KEY`) are never materialized inside sandboxes or the API service. Tools declare which keys they need in their `pyproject.toml` and call `secret("KEY")` to receive a placeholder. Outbound HTTPS traffic is routed through iron-proxy, which substitutes the real credential based on the host/key injection map and iron-control grants. iron-proxy resolves `op://...` references directly against 1Password when that source is configured. - -For local development, infra secrets are stored in Kubernetes Secrets created by `just bootstrap-secrets`; application secrets continue to come from 1Password. - -### iron-control - -[iron-control](https://github.com/ironsh/iron-control) is an optional Rails control plane for permissioning and encrypted secret storage. It is off by default; enable it with `--set ironControl.enabled=true` (or set `ironControl.enabled: true` in a values file). When enabled, it runs against a dedicated `iron_control_production` database on the bundled Postgres (a separate logical DB so its Rails `schema_migrations` table never collides with api-rs SQLx migrations), created by an idempotent init container. - -`just bootstrap-secrets` seeds the required keys into `centaur-infra-env`: the three ActiveRecord encryption keys, `SECRET_KEY_BASE`, and the initial admin password/API key are auto-generated (only when absent, never rotated in place). `IRON_CONTROL_DATABASE_URL` defaults to the bundled Postgres server with no database path (so Rails resolves each connection's database name from the image's `database.yml`); export it before running `just bootstrap-secrets` to point at an external server. Override the admin email with `IRON_CONTROL_INITIAL_USER_EMAIL` (default `admin@centaur.local`). - -### centaur-perms - -`centaur-perms` is the operator CLI for iron-control permissions: it controls which chat principals (Slack users/channels, Discord channels, and Teams users/conversations) and which roles hold which tool roles and secrets. It lives at `services/api-rs/crates/centaur-perms` and reuses iron-control's canonical mappings (`derive_principal`, `RoleSpec::tool`), so every principal and role `foreign_id` it writes matches exactly what `api-rs` registers at session start. It is the supported way to inspect and edit grants by hand; the API writes the same resources at runtime. - -#### Concepts - -- **Principal** — the chat identity an agent session runs as. `foreign_id`s are derived canonically: `slack-channel--` for a Slack channel, `slack-user--` for a Slack DM, `discord-channel--` for Discord, `teams-conversation--` for Teams conversations, and `teams-user--` for Teams user-scoped runs. Each session binds to one derived principal; grant the channel/conversation principal for shared contexts and the user principal for DM or user-scoped contexts. -- **Role** — a named bundle of secret grants assignable to principals. Canonical roles: `infra` (shared infra secrets), `tools` (shared harness/tool secrets), and one `tool-` per tool (e.g. `tool-github`). -- **Secret** — a typed iron-control resource (static `ssr_`, OAuth token `ots_`, GCP auth `gas_`, Postgres DSN `pgs_`, HMAC signing `hms_`). iron-control never returns credential values, only the source each resolves from. Each `tool-` secret keeps a canonical `tool--…` id so the same object is shared no matter which role grants it. -- **Grant** — binds a secret to a grantee (a principal or a role). `centaur-perms` resources carry the label `managed-by=centaur`. - -A principal's *effective* access is the union of its directly granted secrets and the secrets carried by every role assigned to it. - -#### Setup - -The CLI talks to the iron-control admin API. Provide the connection via flags or env vars (iron-control must be enabled — see above): - -```bash -export IRON_CONTROL_URL=http://localhost:3000 # admin API base URL -export IRON_CONTROL_API_KEY=iak_… # admin API key -export IRON_CONTROL_NAMESPACE=default # optional, defaults to "default" -``` - -For `--tool` lookups, point the CLI at the same tool directories the API uses, via repeatable `--tools-dir` flags or the colon-separated `TOOL_DIRS` env var (explicit dirs first, then env; later dirs shadow earlier ones, matching the overlay order). Build and run from `services/api-rs`: - -```bash -cd services/api-rs -cargo run -p centaur-perms -- # or: cargo build -p centaur-perms; ./target/debug/centaur-perms -``` - -The `--tool` flag parses a tool's `pyproject.toml` `[tool.centaur]` secrets and registers them in iron-control before granting. How each secret's `secret_ref` resolves to a source is set by `--source-policy` (`env` default, `onepassword`, or `onepassword-connect`); the 1Password policies also require `--op-vault` (and accept `--op-ttl`, default `10m`). - -#### Command surface - -Commands are resource-first — `centaur-perms `: - -| Command | What it does | -|---------|--------------| -| `principals list [--filter S] [--label k=v] [--managed]` | List principals. `--filter` is a case-insensitive substring on `foreign_id`/name; `--managed` is shorthand for `--label managed-by=centaur`. | -| `principals show [--slack-user U]` | Show a principal's roles (with each role's grants), direct grants, and effective replace-secret placeholders. | -| `principals grant [--slack-user U] [--tool N] [--role F] [--secret OID]` | Grant access. `--tool` registers its `tool-` role + secrets then assigns it; `--role` assigns an existing role; `--secret` grants a secret OID directly. All repeatable; creates the principal if absent. | -| `principals revoke [--slack-user U] [--tool N] [--role F] [--secret OID] [--grant-id OID]` | Reverse of grant. `--tool`/`--role` unassign the role; `--secret` deletes the direct grant for that secret; `--grant-id` deletes a grant by its `grant_…` id. | -| `roles list / show ` | List roles, or show the secrets granted to one role. | -| `roles grant [--secret OID] [--tool N [--secret-name NAME]]` | Grant secrets to a role by OID, or register+grant a tool's declared secrets. `--secret-name` (repeatable, requires `--tool`) selects specific declared secrets instead of all. | -| `roles revoke --secret OID` | Revoke one or more secrets from a role (`--secret` required, repeatable). | -| `secrets list [--filter S] [--label k=v] [--managed]` | List secrets across every type, one row per secret. | -| `secrets show ` | Show one secret's full config by OID or `foreign_id` (values are never shown — only the source). | -| `broker create --foreign-id F --token-endpoint URL --client-id ID [--client-secret S] [--refresh-token SEED] [--scope SC]…` | Create or update an iron-control broker credential. Values are passed literally; iron-control owns the OAuth refresh loop. Re-supplying `--refresh-token` re-bootstraps it. | -| `broker list / show / delete ` | List broker credentials, show one (status/expiry; secret material is never returned), or delete one (by `bcr_` OID or `foreign_id`). | - -A `` argument is treated as a chat thread key when it contains `:` (for example `slack:T123:C456:1700000000.0001`, `discord:111:222:333`, or `teams::`) and run through `derive_principal`. Pass `--slack-user` so a Slack DM thread keys to the user. Any value without a `:` is used verbatim as a `foreign_id` (e.g. `slack-channel-t123-c456` or `teams-conversation-19-abc123-thread-tacv2`) or an OID. Grant/revoke operations are idempotent: re-granting an assigned role or revoking a missing grant is a no-op, reported as such. - -A tool's `brokered_token` secret registers the *consumer* side — a static secret that injects the access token from a `token_broker` source. The broker credential itself (the managed OAuth refresh loop) is provisioned out of band with `broker create`; the tool's `brokered_token` references it by `foreign_id` (its `credential`, defaulting to the secret `name`). - -#### Common workflows - -Give a channel access to a tool (registers the tool's role + secrets from its `pyproject.toml`, then assigns the role to the channel): - -```bash -centaur-perms principals grant slack-channel-t123-c456 --tool github --tools-dir tools -``` - -Inspect what a principal can actually do (resolve a live thread key, then list roles, direct grants, and effective secrets): - -```bash -centaur-perms principals show slack:T123:C456:1700000000.0001 -``` - -Give an individual user a tool only in their DMs: - -```bash -centaur-perms principals grant slack:D9999999:1700000000.0001 --slack-user U07ABC --tool github --tools-dir tools -``` - -Register a tool's secrets once on the shared `tools` role, then assign that role to many principals: - -```bash -centaur-perms roles grant tools --tool github --tools-dir tools -centaur-perms principals grant slack-channel-t123-c456 --role tools -``` - -Register only a single named secret from a tool onto a role: - -```bash -centaur-perms roles grant infra --tool slackbot --secret-name SLACK_BOT_TOKEN --tools-dir tools -``` - -Revoke a tool from a channel (unassigns the `tool-` role; shared secrets on other roles are untouched): - -```bash -centaur-perms principals revoke slack-channel-t123-c456 --tool github -``` - -Provision a managed broker credential a `brokered_token` secret (or a harness fragment) references — e.g. the Codex/Claude Code access-token harnesses reference `openai-codex` / `anthropic-claude`: - -```bash -centaur-perms broker create --foreign-id openai-codex \ - --token-endpoint https://auth.openai.com/oauth/token \ - --client-id "$OPENAI_CODEX_CLIENT_ID" --refresh-token "$OPENAI_CODEX_REFRESH_TOKEN" -``` - -Audit Centaur-managed secrets and inspect one: - -```bash -centaur-perms secrets list --managed -centaur-perms secrets show tool-github-github_token -``` - -## Observability & Audit Logs - -### Architecture - -All services write structured JSON logs to **stdout**. Kubernetes captures pod logs, and optional observability deployments can forward them to VictoriaLogs. api-rs exposes Prometheus metrics at `/metrics` when scraping is enabled. - -``` -Service → stdout (JSON) → Kubernetes pod logs → optional log collector → VictoriaLogs/Grafana -``` - -This design keeps the local Helm stack minimal while preserving structured logs for collectors. - -### Components - -| Component | Role | Config | -|-----------|------|--------| -| **VictoriaLogs** | Optional log storage + query engine | External/overlay deployment | -| **VictoriaMetrics** | Optional metrics storage + query engine | Push-based when enabled | -| **Grafana** | Optional dashboards + log explorer | External/overlay deployment | - -### Querying logs - -Via Grafana: navigate to **Explore → VictoriaLogs** and use [LogsQL](https://docs.victoriametrics.com/victorialogs/logsql/). - -Via CLI (from inside the Kubernetes network): - -```bash -# All logs for a specific thread -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s "http://victorialogs:9428/select/logsql/query" \ - --data-urlencode "query=thread_key:C042WDDP89Y" --data-urlencode "limit=50" - -# API errors in the last hour -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s "http://victorialogs:9428/select/logsql/query" \ - --data-urlencode "query=_stream:{service=\"api-rs\"} AND level:error" --data-urlencode "limit=20" - -# Firewall audit trail for a time range -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s "http://victorialogs:9428/select/logsql/query" \ - --data-urlencode "query=_stream:{service=\"iron-proxy\"} AND event:proxy_audit" \ - --data-urlencode "start=2026-03-10T00:00:00Z" --data-urlencode "end=2026-03-11T00:00:00Z" -``` - -### Audit logging - -**iron-proxy** emits structured audit events for outbound requests from sandbox -containers: method, host, path, status code, request/response bytes, duration, -and source container IP. These are searchable via `event:proxy_audit` in -VictoriaLogs. - -**api-rs** logs session lifecycle, workflow, sandbox, proxy, and HTTP request -events with thread context. - -### Logging contract - -Services must write single-line JSON to stdout with these fields: - -| Field | Required | Description | -|-------|----------|-------------| -| `timestamp` | Yes | ISO 8601 timestamp | -| `level` | Yes | `debug`, `info`, `warning`, `error` | -| `service` | Yes | Service name (`api-rs`, `iron-proxy`, `slackbotv2`, etc.) | -| `event` | Yes | Machine-readable event name | -| `msg` | No | Human-readable message | -| `thread_key` | No | Thread identifier (when applicable) | - -> **Never log secret values, auth headers, or raw tokens.** - -## E2E Testing (without Slack) - -### 1. Bring up the stack - -```bash -just up -``` - -All E2E curl commands below use `kubectl exec` for localhost bypass (no API key needed). -To test from outside the container, create a DB-backed key via the [admin API](#api-key-management). - -### 2. Create or reuse a session - -```bash -THREAD_KEY=cli:test-e2e-1 -THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') - -SESSION=$(kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}" \ - -H "Content-Type: application/json" \ - -d '{"harness_type":"codex","on_harness_conflict":"restart"}') -``` - -### 3. Persist a message - -```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/messages" \ - -H "Content-Type: application/json" \ - -d '{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG and nothing else."}]}]}' -``` - -### 4. Execute the session - -```bash -EXECUTE=$(kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/execute" \ - -H "Content-Type: application/json" \ - -d '{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG and nothing else.\"}]}}"]}') -EXECUTION_ID=$(printf '%s' "$EXECUTE" | jq -r '.execution_id') -``` - -### 5. Tail durable events (or reconnect later) - -```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -N \ - "http://localhost:8080/api/session/${THREAD_PATH}/events?execution_id=${EXECUTION_ID}&after_event_id=0" -``` - -If this stream disconnects, reconnect with the last seen SSE `id` as -`after_event_id`. - -### 6. Inspect the session - -```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/session/${THREAD_PATH}" | jq -``` - -### Debugging - -```bash -kubectl get pods -n centaur -l centaur.ai/managed=true -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s http://localhost:8080/healthz -kubectl exec -n centaur -- centaur-tools list -``` +# Centaur Agent Guide + +## Scope and instruction hierarchy + +This file applies to the whole repository. Read the nearest `AGENTS.md` before +changing files below it; service-local instructions extend this file and take +precedence for that service. + +Keep new and rewritten service guidance deployment-neutral. Do not add new +company names, private domains, cluster names, chat workspace identifiers, +private repository names, absolute user paths, or private overlay procedures. +Use neutral placeholders for new examples. Do not remove or rewrite existing +product-specific defaults solely to make them neutral unless the user asks. + +## How to work here + +- Inspect `git status` before editing. Preserve unrelated work and never format, + stage, or revert files outside the task. +- For a focused PR when the current checkout has unrelated changes, use an + isolated worktree based on the intended branch. Check the base branch and any + dependent PRs before implementing instead of rebuilding work that already + exists elsewhere. +- Establish the requested boundary before acting: explanation, review, + diagnosis, implementation, local validation, and remote operation are + different scopes. Do not turn a read-only request into a change. +- Read the current implementation, manifests, and tests before relying on prose + documentation. Prefer an existing name, setting, abstraction, or protocol to + a parallel one. +- Keep clients thin and changes focused. If a contract changes, update every + producer, consumer, test fixture, and chart value affected by it. +- Never expose credentials in output, logs, fixtures, commits, or command-line + arguments. Use placeholders and configured secret paths. +- Do not mutate a remote environment unless the user explicitly asks. Local + testing does not authorize committing, pushing, deploying, or restarting. +- Before any Kubernetes operation, verify the current context and namespace. + Pass an explicit `--context` for non-local or destructive work; never rely on + an ambient context when a mistake could affect another environment. +- When the user explicitly requests an artifact such as a PR, CI repair, or + deployment, carry the authorized workflow through to that artifact and its + relevant verification instead of stopping after the code edit. +- Concretely, a PR request means validate, commit, push the branch, open or + update the PR, and return its link. If the user also asks for CI, rollout, or + dependent-PR follow-through, monitor and repair that requested boundary too. +- Use conventional commit prefixes when a commit is requested: `feat:`, `fix:`, + `docs:`, `refactor:`, `test:`, or `chore:`. + +## Architecture boundaries + +The durable request path is: + +1. A chat ingress verifies and normalizes a platform event. +2. The ingress creates or reuses a session, appends the durable user message, + starts an execution, and consumes replayable events. +3. `api-rs` owns session assignment, execution serialization, recovery, + workflow state, and persistence in Postgres. +4. The sandbox runtime translates neutral content into the selected harness and + exposes tool CLIs. Harness and tool traffic reaches upstreams through + `iron-proxy` without materializing real credentials in the sandbox. +5. The ingress renders durable output back to the originating platform. + +Ownership by tree: + +- `services/api-rs/`: Rust control plane, durable sessions, sandbox backends, + workflows, auth integration, and telemetry. +- `services/slackbotv2/`, `discordbot/`, `githubbot/`, `linearbot/`, + `teamsbot/`: platform transport, policy gates, session forwarding, and + platform rendering. +- `services/sandbox/`: agent image, startup composition, tool installation, + repo-cache helpers, and runtime prompt. Harness protocol normalization lives + in `crates/harness-server/`, which is built into the image. +- `services/iron-proxy/`: credential-injecting proxy image and startup config. +- `services/workflow-python/`: Python workflow compatibility host; durability + remains in `api-rs`. +- `services/console/`: operator UI and credential-control API. +- `tools/`: independently packaged agent-facing CLI plugins. +- `workflows/`: discoverable workflow definitions. +- `contrib/chart/`: Helm wiring, policies, probes, and service configuration. +- `packages/`: shared TypeScript event and rendering contracts used by ingress + services. + +Do not reintroduce legacy control paths alongside the durable session API. +Modern investigations should start with `sessions`, `session_messages`, +`session_executions`, `session_events`, and workflow state, then follow the +final platform-delivery boundary. + +## Local development and validation + +Centaur is validated on the local Kubernetes stack. Start with the narrowest +relevant unit or integration test, then prove cross-service behavior when a +boundary changed. Run `kubectl config current-context` before the local stack +commands below; `just deploy` uses the ambient Helm/Kubernetes context. + +```bash +just up # build and start the local stack +just deploy # update the local Helm release +just status +just logs +``` + +For a runtime change requested for publication, local proof means: + +1. run the service's format, type, lint, and unit checks; +2. build the affected runtime artifact with the repository's build recipes; +3. deploy it to the local stack; +4. make a real request through the changed path and inspect the durable result; +5. only then commit or push, and only if requested. + +For a missing, duplicate, or stalled chat response, trace the full chain: +platform receipt -> session creation -> durable message -> execution -> event +stream -> render obligation -> final platform message. A healthy pod or one log +line is not proof of successful delivery. If investigation and remediation are +both requested, preserve a bounded evidence snapshot before destructive action +when it is safe to do so. + +Useful repository-wide checks include: + +```bash +pnpm install --frozen-lockfile +helm lint contrib/chart +git diff --check +``` + +Python code targets Python 3.11+ and uses `uv` for environments and commands. +Follow the local package's import style: service modules generally use +top-level absolute imports, while independently packaged tool CLIs and optional +dependencies may deliberately import lazily. Do not mechanically rewrite those +boundaries. Rust, Ruby, TypeScript, shell, and image-only services have their +own commands in local guides; there is no single repository-wide lint command +that accurately validates every service. + +## Tools and workflows + +Tool plugins under `tools/` are independently packaged CLIs. Keep secret access +in the client through the SDK placeholder mechanism; do not load dotenv files +in reusable clients. A tool visible to agents needs a `[project.scripts]` entry, +and its CLI wrapper should remain thin. Validate catalog discovery, ` +--help`, and one real command from a local sandbox. + +Workflow definitions under `workflows/` declare a unique `WORKFLOW_NAME` and an +async handler. Use durable context primitives for side effects, sleeps, events, +child workflows, agents, and tools; do not add process-local durability. Keep +step names stable and test replay behavior after failures or restarts. + +For a credentialed tool change, trace the complete path: tool declaration -> +principal/role grant -> proxy configuration -> controlled request from a real +sandbox. Configuration presence alone does not prove usable or appropriately +scoped access. + +## Reviews and incident reports + +- For reviews, report concrete findings in severity order with file and line + references. Passing tests do not prove protocol, authorization, or recovery + correctness. Do not edit unless asked to resolve findings. +- For incidents, distinguish durable state, observed logs/metrics, live runtime + state, deployed version/configuration, and user-visible outcome. State what is + verified versus inferred. +- Check authorization, credential exposure, idempotency, retry behavior, + cancellation, and crash recovery early when those boundaries are involved. +- For a broad review, split independent protocol, authorization/lifecycle, and + deployment-wiring passes, then deduplicate and prioritize the findings. + +## Canonical references + +- `README.md` and `docs/pages/architecture.mdx`: system overview. +- `docs/pages/quickstart.mdx`: local stack and end-to-end smoke path. +- `contrib/chart/values.yaml`: supported deployment configuration. +- Service `README.md` files, where present: behavior and environment variables. +- `services/api-rs/rfcs/`: control-plane and sandbox design contracts. + +"Chat SDK" means the Vercel Chat SDK. When adapter behavior matters, inspect +the source checkout at `~/github/vercel/chat` rather than generated files under +`node_modules`. diff --git a/Justfile b/Justfile index e1349f291..b3be354c8 100644 --- a/Justfile +++ b/Justfile @@ -30,7 +30,7 @@ build: just _build-all-sequential else pids=() - for recipe in _build-api-rs _build-iron-proxy _build-slackbotv2 _build-linearbot _build-discordbot _build-teamsbot _build-agent _build-console; do + for recipe in _build-api-rs _build-iron-proxy _build-slackbotv2 _build-linearbot _build-discordbot _build-githubbot _build-teamsbot _build-agent _build-console; do just "$recipe" & pids+=("$!") done @@ -47,6 +47,7 @@ _build-all-sequential: just _build-slackbotv2 just _build-linearbot just _build-discordbot + just _build-githubbot just _build-teamsbot just _build-agent just _build-console @@ -60,8 +61,10 @@ build-one service: slackbotv2) just _build-slackbotv2 ;; linearbot) just _build-linearbot ;; discordbot) just _build-discordbot ;; + githubbot) just _build-githubbot ;; teamsbot) just _build-teamsbot ;; agent|sandbox) just _build-agent ;; + workflow-python) just _build-workflow-python ;; console) just _build-console ;; *) echo "unknown service: {{service}}" >&2; exit 2 ;; esac @@ -81,12 +84,20 @@ _build-linearbot: _build-discordbot: docker build -t centaur-discordbot:latest -f services/discordbot/Dockerfile . +_build-githubbot: + docker build -t centaur-githubbot:latest -f services/githubbot/Dockerfile . + _build-teamsbot: docker build -t centaur-teamsbot:latest -f services/teamsbot/Dockerfile . _build-agent: docker build --target "{{agent_build_target}}" -t "{{agent_image}}" -f "{{agent_dockerfile}}" . +# The Python workflow host is embedded in both consumer images. +_build-workflow-python: + just _build-api-rs + just _build-agent + # The console builds from its own subdirectory context (services/console), unlike # the other services which build from the repo root. _build-console: @@ -98,7 +109,7 @@ _build-console: _push-registry: #!/usr/bin/env bash set -euo pipefail - for img in centaur-api-rs centaur-iron-proxy centaur-slackbotv2 centaur-linearbot centaur-discordbot centaur-teamsbot centaur-agent centaur-console; do + for img in centaur-api-rs centaur-iron-proxy centaur-slackbotv2 centaur-linearbot centaur-discordbot centaur-githubbot centaur-teamsbot centaur-agent centaur-console; do target="{{registry}}/library/${img}:latest" echo "pushing ${img}:latest -> ${target}..." docker tag "${img}:latest" "${target}" @@ -111,7 +122,7 @@ _push-registry: _import-k3s: #!/usr/bin/env bash set -euo pipefail - for img in centaur-api-rs centaur-iron-proxy centaur-slackbotv2 centaur-linearbot centaur-discordbot centaur-teamsbot centaur-agent centaur-console; do + for img in centaur-api-rs centaur-iron-proxy centaur-slackbotv2 centaur-linearbot centaur-discordbot centaur-githubbot centaur-teamsbot centaur-agent centaur-console; do echo "importing ${img}:latest into k3s containerd..." docker save "${img}:latest" | {{k3s_ctr}} images import - done @@ -128,14 +139,15 @@ deploy: local) ;; ghcr) extra_args+=( - --set apiRs.image.repository=ghcr.io/tiplink/centaur/centaur-api-rs - --set ironProxy.image.repository=ghcr.io/tiplink/centaur/centaur-iron-proxy - --set slackbotv2.image.repository=ghcr.io/tiplink/centaur/centaur-slackbotv2 - --set linearbot.image.repository=ghcr.io/tiplink/centaur/centaur-linearbot - --set discordbot.image.repository=ghcr.io/tiplink/centaur/centaur-discordbot - --set teamsbot.image.repository=ghcr.io/tiplink/centaur/centaur-teamsbot - --set sandbox.image.repository=ghcr.io/tiplink/centaur/centaur-agent - --set console.image.repository=ghcr.io/tiplink/centaur/centaur-console + --set apiRs.image.repository=ghcr.io/paradigmxyz/centaur/centaur-api-rs + --set ironProxy.image.repository=ghcr.io/paradigmxyz/centaur/centaur-iron-proxy + --set slackbotv2.image.repository=ghcr.io/paradigmxyz/centaur/centaur-slackbotv2 + --set linearbot.image.repository=ghcr.io/paradigmxyz/centaur/centaur-linearbot + --set discordbot.image.repository=ghcr.io/paradigmxyz/centaur/centaur-discordbot + --set githubbot.image.repository=ghcr.io/paradigmxyz/centaur/centaur-githubbot + --set teamsbot.image.repository=ghcr.io/paradigmxyz/centaur/centaur-teamsbot + --set sandbox.image.repository=ghcr.io/paradigmxyz/centaur/centaur-agent + --set console.image.repository=ghcr.io/paradigmxyz/centaur/centaur-console ) ;; *) echo "unknown source: {{source}} (expected local or ghcr)" >&2; exit 2 ;; diff --git a/centaur_sdk/README.md b/centaur_sdk/README.md index ba3ccb664..d50da1f74 100644 --- a/centaur_sdk/README.md +++ b/centaur_sdk/README.md @@ -30,17 +30,3 @@ from centaur_sdk.backends import configure, DotEnvBackend configure(DotEnvBackend(".env")) ``` - -### CLI tables - -```python -from centaur_sdk import Table, render_text_table - -# Rich table (interactive) -table = Table(title="Results") -table.add_column("Name") -table.add_row("example") - -# Plain-text table (for piping) -print(render_text_table(["Name", "Value"], [["a", "1"], ["b", "2"]])) -``` diff --git a/centaur_sdk/__init__.py b/centaur_sdk/__init__.py index 29bbce785..e27b6421c 100644 --- a/centaur_sdk/__init__.py +++ b/centaur_sdk/__init__.py @@ -2,13 +2,10 @@ Public API: secret(key) — resolve a secret via the pluggable backend - Table — Rich table (re-export for CLI tools) - render_text_table — plain-text table renderer """ from __future__ import annotations -from centaur_sdk.cli_tables import Table, render_text_table from centaur_sdk.tool_sdk import ( ToolContext, current_session_context, @@ -23,13 +20,11 @@ ) __all__ = [ - "Table", "ToolContext", "current_session_context", "current_slack_thread", "current_thread_key", "get_tool_context", - "render_text_table", "reset_tool_context", "save_attachment", "save_attachment_from_path", diff --git a/centaur_sdk/cli_tables.py b/centaur_sdk/cli_tables.py deleted file mode 100644 index 4db2a04c7..000000000 --- a/centaur_sdk/cli_tables.py +++ /dev/null @@ -1,30 +0,0 @@ -"""Table rendering helpers for tool CLIs.""" - -from __future__ import annotations - -from rich.table import Table as RichTable - -Table = RichTable - - -def render_text_table(headers: list[str], rows: list[list[str]]) -> str: - """Render a plain-text table with padded columns. - - Useful for CLIs that should avoid hardcoding one-off spacing logic. - """ - if not headers: - return "" - if not rows: - return "No rows." - - widths = [len(header) for header in headers] - for row in rows: - for idx, cell in enumerate(row): - widths[idx] = max(widths[idx], len(cell)) - - def _format(row: list[str]) -> str: - return " ".join(cell.ljust(widths[idx]) for idx, cell in enumerate(row)) - - lines = [_format(headers), " ".join("-" * width for width in widths)] - lines.extend(_format(row) for row in rows) - return "\n".join(lines) diff --git a/centaur_sdk/pyproject.toml b/centaur_sdk/pyproject.toml index 33342adcc..7f2753b3f 100644 --- a/centaur_sdk/pyproject.toml +++ b/centaur_sdk/pyproject.toml @@ -5,9 +5,7 @@ description = "Lightweight SDK for building Centaur-compatible tools" requires-python = ">=3.11" readme = "README.md" license = "Apache-2.0 OR MIT" -dependencies = [ - "rich>=13.0", -] +dependencies = [] [project.optional-dependencies] http = ["httpx>=0.28.0"] diff --git a/centaur_sdk/tests/test_tool_sdk.py b/centaur_sdk/tests/test_tool_sdk.py index 8f042b347..754a2f3d9 100644 --- a/centaur_sdk/tests/test_tool_sdk.py +++ b/centaur_sdk/tests/test_tool_sdk.py @@ -1,15 +1,16 @@ from __future__ import annotations import threading +from pathlib import Path import pytest -import centaur_sdk.tool_sdk as tool_sdk from centaur_sdk import ( ToolContext, current_session_context, current_slack_thread, reset_tool_context, + save_attachment, secret, set_tool_context, ) @@ -49,46 +50,6 @@ def test_secret_uses_backend_when_context_is_missing(monkeypatch: pytest.MonkeyP assert secret("TOKEN") == "from-backend" -def test_centaur_api_key_prefers_refreshed_file( - monkeypatch: pytest.MonkeyPatch, - tmp_path, -): - key_file = tmp_path / ".api_key" - key_file.write_text("fresh-token\n", encoding="utf-8") - monkeypatch.setattr(tool_sdk, "CENTAUR_API_KEY_FILE", key_file) - monkeypatch.setattr( - registry, - "_backend", - MappingBackend({"CENTAUR_API_KEY": "from-backend"}), - ) - token = set_tool_context( - ToolContext(name="fake-tool", secrets={"CENTAUR_API_KEY": "from-context"}) - ) - try: - assert secret("CENTAUR_API_KEY") == "fresh-token" - finally: - reset_tool_context(token) - - -def test_centaur_api_key_uses_tool_context_when_refreshed_file_missing( - monkeypatch: pytest.MonkeyPatch, - tmp_path, -): - monkeypatch.setattr(tool_sdk, "CENTAUR_API_KEY_FILE", tmp_path / "missing") - monkeypatch.setattr( - registry, - "_backend", - MappingBackend({"CENTAUR_API_KEY": "from-backend"}), - ) - token = set_tool_context( - ToolContext(name="fake-tool", secrets={"CENTAUR_API_KEY": "from-context"}) - ) - try: - assert secret("CENTAUR_API_KEY") == "from-context" - finally: - reset_tool_context(token) - - def test_secret_uses_default_after_context_and_backend_miss( monkeypatch: pytest.MonkeyPatch, ): @@ -150,6 +111,22 @@ def fake_urlopen(request, timeout): reset_tool_context(token) +def test_current_session_context_requires_api_server_capability( + monkeypatch: pytest.MonkeyPatch, +): + monkeypatch.setattr( + registry, + "_backend", + MappingBackend({"CENTAUR_SANDBOX_API_SERVER_ENABLED": "false"}), + ) + token = set_tool_context(ToolContext(name="fake-tool", thread_key="slack:C123:123.456")) + try: + with pytest.raises(RuntimeError, match="API server sandbox capability"): + current_session_context() + finally: + reset_tool_context(token) + + def test_current_slack_thread_returns_api_slack_destination( monkeypatch: pytest.MonkeyPatch, ): @@ -180,6 +157,70 @@ def read(self) -> bytes: reset_tool_context(token) +def test_save_attachment_writes_to_sandbox_uploads_dir( + monkeypatch: pytest.MonkeyPatch, tmp_path +): + def fail_urlopen(*_args, **_kwargs): + raise AssertionError("save_attachment should not call the API in sandbox mode") + + monkeypatch.setenv("CENTAUR_UPLOADS_DIR", str(tmp_path)) + monkeypatch.setattr("urllib.request.urlopen", fail_urlopen) + + result = save_attachment( + name="../report.txt", + data=b"hello", + mime_type="text/plain", + source_url="https://example.test/report", + ) + + saved_path = tmp_path / "report.txt" + assert saved_path.read_bytes() == b"hello" + assert result == { + "attachment_id": None, + "filename": "report.txt", + "mime_type": "text/plain", + "download_url": None, + "path": str(saved_path), + "local_path": str(saved_path), + "source_url": "https://example.test/report", + "size_bytes": 5, + } + + +def test_save_attachment_requires_api_server_capability_without_uploads_dir( + monkeypatch: pytest.MonkeyPatch, +): + monkeypatch.delenv("CENTAUR_UPLOADS_DIR", raising=False) + monkeypatch.setattr( + registry, + "_backend", + MappingBackend({"CENTAUR_SANDBOX_API_SERVER_ENABLED": "false"}), + ) + token = set_tool_context(ToolContext(name="fake-tool", thread_key="slack:C123:123.456")) + try: + with pytest.raises(RuntimeError, match="API server sandbox capability"): + save_attachment(name="report.txt", data=b"hello") + finally: + reset_tool_context(token) + + +def test_save_attachment_uses_unique_local_name_on_collision( + monkeypatch: pytest.MonkeyPatch, tmp_path +): + monkeypatch.setenv("CENTAUR_UPLOADS_DIR", str(tmp_path)) + + first = save_attachment(name="same.txt", data=b"first") + second = save_attachment(name="same.txt", data=b"second") + + assert first["path"] != second["path"] + assert (tmp_path / "same.txt").read_bytes() == b"first" + second_path = Path(str(second["path"])) + assert second_path.exists() + assert second_path.read_bytes() == b"second" + assert second_path.name.startswith("same-") + assert second_path.suffix == ".txt" + + @pytest.mark.asyncio async def test_stub_backend_returns_key_placeholders(): backend = StubBackend() diff --git a/centaur_sdk/tool_sdk.py b/centaur_sdk/tool_sdk.py index 0716f6349..65f51fa93 100644 --- a/centaur_sdk/tool_sdk.py +++ b/centaur_sdk/tool_sdk.py @@ -7,17 +7,17 @@ import json import logging import mimetypes -from urllib.parse import quote +import os import urllib.request +import uuid from contextvars import ContextVar from dataclasses import dataclass, field from pathlib import Path from typing import Any +from urllib.parse import quote log = logging.getLogger(__name__) -CENTAUR_API_KEY_FILE = Path("/home/agent/.api_key") - @dataclass class ToolContext: @@ -47,29 +47,13 @@ def get_tool_context() -> ToolContext: # --------------------------------------------------------------------------- -def _refreshed_centaur_api_key() -> str | None: - try: - value = CENTAUR_API_KEY_FILE.read_text(encoding="utf-8").strip() - except OSError: - return None - return value or None - - def secret(key: str, default: str | None = None) -> str: - """Get a secret. Resolution order: refreshed API key → tool context → backend → default. + """Get a secret. Resolution order: tool context → pluggable backend → default. - - **Refreshed API key**: ``CENTAUR_API_KEY`` prefers - ``/home/agent/.api_key`` when present so long-lived sandboxes can use a - rotated control-plane token. - **ToolContext**: Set by ToolManager, populated from .env files (if any). - **Pluggable backend**: Configured via ``centaur_sdk.backends.registry`` (env vars, HTTP sidecar, etc.). """ - if key == "CENTAUR_API_KEY": - val = _refreshed_centaur_api_key() - if val is not None: - return val - # 1. Check tool context if available (server mode) try: ctx = _tool_ctx.get() @@ -95,6 +79,14 @@ def secret(key: str, default: str | None = None) -> str: raise KeyError(f"Missing secret '{key}'{ctx_name}") +def _require_api_server_enabled(operation: str) -> None: + if secret("CENTAUR_SANDBOX_API_SERVER_ENABLED", "true").strip().lower() == "false": + raise RuntimeError( + f"{operation} requires the API server sandbox capability, but it is disabled " + "for this principal." + ) + + def current_thread_key() -> str: """Return the active thread key for a tool call.""" try: @@ -116,6 +108,7 @@ def current_session_context() -> dict[str, Any]: ``slack.thread_ts``. The API remains the source of truth so warm pooled sandboxes do not need per-thread environment mutation. """ + _require_api_server_enabled("current_session_context") thread_key = current_thread_key() base_url = secret("CENTAUR_API_URL", "http://api:8000").rstrip("/") headers: dict[str, str] = {} @@ -143,6 +136,47 @@ def current_slack_thread() -> dict[str, str]: } +def _sandbox_uploads_dir() -> Path | None: + configured = os.environ.get("CENTAUR_UPLOADS_DIR", "").strip() + if configured: + return Path(configured) + if os.environ.get("CENTAUR_THREAD_KEY", "").strip(): + return Path.home() / "uploads" + return None + + +def _unique_upload_path(uploads_dir: Path, name: str) -> Path: + candidate = uploads_dir / name + if not candidate.exists(): + return candidate + suffix = candidate.suffix + stem = candidate.stem or "attachment" + return uploads_dir / f"{stem}-{uuid.uuid4().hex}{suffix}" + + +def _save_local_attachment( + *, + name: str, + data: bytes, + mime_type: str, + source_url: str | None, + uploads_dir: Path, +) -> dict[str, Any]: + uploads_dir.mkdir(parents=True, exist_ok=True) + path = _unique_upload_path(uploads_dir, name) + path.write_bytes(data) + return { + "attachment_id": None, + "filename": name, + "mime_type": mime_type, + "download_url": None, + "path": str(path), + "local_path": str(path), + "source_url": source_url, + "size_bytes": len(data), + } + + def save_attachment( *, name: str, @@ -151,9 +185,20 @@ def save_attachment( source_url: str | None = None, ) -> dict[str, Any]: """Persist bytes as a Centaur attachment scoped to the current tool thread.""" - thread_key = current_thread_key() safe_name = Path(name).name or "attachment" resolved_mime = mime_type or mimetypes.guess_type(safe_name)[0] or "application/octet-stream" + uploads_dir = _sandbox_uploads_dir() + if uploads_dir is not None: + return _save_local_attachment( + name=safe_name, + data=data, + mime_type=resolved_mime, + source_url=source_url, + uploads_dir=uploads_dir, + ) + + _require_api_server_enabled("save_attachment") + thread_key = current_thread_key() base_url = secret("CENTAUR_API_URL", "http://api:8000").rstrip("/") payload = json.dumps( { diff --git a/contrib/chart/Chart.yaml b/contrib/chart/Chart.yaml index 9e2da353e..efa9031d8 100644 --- a/contrib/chart/Chart.yaml +++ b/contrib/chart/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v2 name: centaur description: Helm chart for the trusted Centaur control plane type: application -version: 0.1.79 +version: 0.1.98 appVersion: "0.1.0" dependencies: - name: connect diff --git a/contrib/chart/templates/_helpers.tpl b/contrib/chart/templates/_helpers.tpl index 93de8cc2b..d77b86f39 100644 --- a/contrib/chart/templates/_helpers.tpl +++ b/contrib/chart/templates/_helpers.tpl @@ -75,6 +75,11 @@ app.kubernetes.io/component: {{ .component }} {{- $storageType -}} {{- end -}} +{{- define "centaur.repositoryVisibility" -}} +{{- $visibility := lower (default "private" .) -}} +{{- if eq $visibility "public" -}}public{{- else -}}private{{- end -}} +{{- end -}} + {{- define "centaur.overlaySources" -}} {{- $sources := list -}} {{- with .Values.overlays.sources -}} @@ -82,6 +87,7 @@ app.kubernetes.io/component: {{ .component }} {{- if .repo -}} {{- $source := dict "repo" .repo -}} {{- with .ref }}{{- $_ := set $source "ref" . -}}{{- end -}} +{{- $_ := set $source "visibility" (include "centaur.repositoryVisibility" .visibility) -}} {{- /* Subdir defaults: an omitted key falls back to the conventional layout (tools, workflows, .agents/skills); a key explicitly set to "" disables @@ -112,11 +118,13 @@ so the defaults are safe for repos that only carry some surfaces. {{- if and .Values.toolServer.enabled .Values.toolServer.repo -}} {{- $source := dict "repo" .Values.toolServer.repo "toolsSubdir" (default "tools" .Values.toolServer.subdir) "workflowsSubdir" "workflows" "skillsSubdir" ".agents/skills" -}} {{- with .Values.toolServer.ref }}{{- $_ := set $source "ref" . -}}{{- end -}} +{{- $_ := set $source "visibility" (include "centaur.repositoryVisibility" .Values.toolServer.visibility) -}} {{- $sources = append $sources $source -}} {{- range .Values.toolServer.extraSources -}} {{- if .repo -}} {{- $source := dict "repo" .repo "toolsSubdir" (default "tools" .subdir) "workflowsSubdir" (default "workflows" .workflowsSubdir) "skillsSubdir" (default ".agents/skills" .skillsSubdir) -}} {{- with .ref }}{{- $_ := set $source "ref" . -}}{{- end -}} +{{- $_ := set $source "visibility" (include "centaur.repositoryVisibility" .visibility) -}} {{- $sources = append $sources $source -}} {{- end -}} {{- end -}} @@ -125,6 +133,27 @@ so the defaults are safe for repos that only carry some surfaces. {{- toJson $sources -}} {{- end -}} +{{- /* +Hash every configured input whose contents are copied into a sandbox at boot. +The API folds this value into SandboxSpec, making warm-pool identity sensitive +to skills/workflow/prompt-only sources as well as tools and transitional images. +Refs and image tags must still be immutable in production; this is an identity +bridge, not a resolver for mutable branches or tags. +*/ -}} +{{- define "centaur.sandboxContentRevision" -}} +{{- $payload := dict + "schemaVersion" 1 + "overlaySources" (include "centaur.overlaySources" . | fromJsonArray) + "repositoryRefs" (.Values.repoCache.repositoryRefs | default dict) + "sandboxImage" (.Values.sandbox.image | default dict) + "ironProxyImage" (.Values.ironProxy.image | default dict) + "overlayImage" (.Values.overlay.image | default dict) + "overlaySystemPrompt" (.Values.overlay.systemPrompt | default "") + "sandboxHarness" (.Values.sandbox.harnessEngine | default "") + "operatorRevision" (.Values.apiRs.sandboxContentRevision | default "") -}} +{{- toJson $payload | sha256sum -}} +{{- end -}} + {{- define "centaur.httpRouteName" -}} {{- $suffix := default (printf "route-%v" .index) .route.name -}} {{- printf "%s-%s" (include "centaur.fullname" .root) $suffix | trunc 63 | trimSuffix "-" -}} diff --git a/contrib/chart/templates/apirs.yaml b/contrib/chart/templates/apirs.yaml index 9cee84749..39d4458fa 100644 --- a/contrib/chart/templates/apirs.yaml +++ b/contrib/chart/templates/apirs.yaml @@ -1,5 +1,6 @@ {{- if .Values.apiRs.enabled }} {{- $console := include "centaur.consoleValues" . | fromYaml -}} +{{- $mcpPublicUrl := default .Values.slackbotv2.mcpPublicUrl .Values.apiRs.mcpPublicUrl -}} {{- $apiRsName := include "centaur.componentName" (dict "root" . "component" "api-rs") -}} {{- $sandboxOverlayMountPath := .Values.overlay.sandboxMountPath | default "/home/agent/overlay/org" -}} {{- $githubAppBrokerBootstrapEnabled := and (or .Values.tokenBroker.enabled .Values.tokenBroker.githubApp.enabled) .Values.tokenBroker.githubApp.enabled .Values.tokenBroker.githubApp.bootstrap.enabled -}} @@ -30,60 +31,116 @@ {{- end -}} {{- $overlaySources := include "centaur.overlaySources" . | fromJsonArray -}} {{- $toolSources := list -}} +{{- $publicToolSources := list -}} {{- $apiWorkflowDirList := list -}} {{- $sandboxWorkflowDirList := list -}} {{- $skillDirs := list -}} +{{- $publicSkillDirs := list -}} {{- range $source := $overlaySources -}} {{- $repo := get $source "repo" | default "" -}} +{{- $visibility := get $source "visibility" | default "private" -}} {{- if $repo -}} {{- with (get $source "toolsSubdir") -}} -{{- $toolSource := dict "repo" $repo "subdir" . -}} +{{- $toolSource := dict "repo" $repo "subdir" . "visibility" $visibility -}} {{- with (get $source "ref") }}{{- $_ := set $toolSource "ref" . -}}{{- end -}} {{- $toolSources = append $toolSources $toolSource -}} +{{- if eq $visibility "public" -}} +{{- $publicToolSources = append $publicToolSources $toolSource -}} +{{- end -}} {{- end -}} {{- with (get $source "workflowsSubdir") -}} {{- $apiWorkflowDirList = append $apiWorkflowDirList (printf "%s/%s/%s" $.Values.repoCache.hostPath $repo .) -}} {{- $sandboxWorkflowDirList = append $sandboxWorkflowDirList (printf "/home/agent/github/%s/%s" $repo .) -}} {{- end -}} {{- with (get $source "skillsSubdir") -}} -{{- $skillDirs = append $skillDirs (printf "/home/agent/github/%s/%s" $repo .) -}} +{{- $skillDir := printf "/home/agent/github/%s/%s" $repo . -}} +{{- $skillDirs = append $skillDirs $skillDir -}} +{{- if eq $visibility "public" -}} +{{- $publicSkillDirs = append $publicSkillDirs $skillDir -}} +{{- end -}} {{- end -}} {{- end -}} {{- end -}} {{- $toolsUseRepoCache := and .Values.repoCache.enabled (gt (len $toolSources) 0) -}} {{- $useOverlayRepoCache := and .Values.repoCache.enabled (gt (len $overlaySources) 0) -}} {{- $toolDirs := list "/app/tools" -}} -{{- $workflowDirList := list "/app/workflows" -}} +{{- $workflowDirs := "/app/workflows" -}} {{- if $toolsUseRepoCache -}} {{- $toolDirs = list -}} +{{- if .Values.overlay.image.repository -}} +{{- $toolDirs = append $toolDirs (printf "%s/tools" .Values.overlay.mountPath) -}} +{{- end -}} {{- range $source := $toolSources }} {{- $toolDirs = append $toolDirs (printf "%s/%s/%s" $.Values.repoCache.hostPath (get $source "repo") (get $source "subdir")) -}} {{- end -}} +{{- else if .Values.overlay.image.repository -}} +{{- $toolDirs = append $toolDirs (printf "%s/tools" .Values.overlay.mountPath) -}} {{- end -}} -{{- if and .Values.repoCache.enabled (gt (len $apiWorkflowDirList) 0) -}} -{{- $workflowDirList = $apiWorkflowDirList -}} +{{- $publicToolDirs := list -}} +{{- if and .Values.repoCache.enabled (gt (len $publicToolSources) 0) -}} +{{- range $source := $publicToolSources }} +{{- $publicToolDirs = append $publicToolDirs (printf "%s/%s/%s" $.Values.repoCache.hostPath (get $source "repo") (get $source "subdir")) -}} {{- end -}} -{{- if .Values.overlay.image.repository -}} -{{- $toolDirs = append $toolDirs (printf "%s/tools" .Values.overlay.mountPath) -}} -{{- $workflowDirList = append $workflowDirList (printf "%s/workflows" .Values.overlay.mountPath) -}} {{- end -}} -{{- $workflowDirs := join ":" $workflowDirList -}} +{{- if and .Values.repoCache.enabled (gt (len $apiWorkflowDirList) 0) -}} +{{- $workflowDirs = join ":" $apiWorkflowDirList -}} +{{- end -}} {{- $sandboxWorkflowDirs := "" -}} {{- if and .Values.repoCache.enabled (gt (len $sandboxWorkflowDirList) 0) -}} {{- $sandboxWorkflowDirs = join ":" $sandboxWorkflowDirList -}} {{- end -}} -{{- if .Values.overlay.image.repository -}} -{{- if $sandboxWorkflowDirs -}} -{{- $sandboxWorkflowDirs = printf "%s:%s" $sandboxWorkflowDirs (printf "%s/workflows" $sandboxOverlayMountPath) -}} -{{- else -}} -{{- $sandboxWorkflowDirs = printf "%s/workflows" $sandboxOverlayMountPath -}} +{{- if and .Values.overlay.image.repository (not (and .Values.repoCache.enabled (gt (len $apiWorkflowDirList) 0))) -}} +{{- $workflowDirs = printf "%s:%s" $workflowDirs (printf "%s/workflows" .Values.overlay.mountPath) -}} {{- end -}} +{{- if and .Values.overlay.image.repository (not (and .Values.repoCache.enabled (gt (len $sandboxWorkflowDirList) 0))) -}} +{{- $sandboxWorkflowDirs = printf "%s/workflows" $sandboxOverlayMountPath -}} {{- end -}} {{- $apiRsMetricsAnnotations := dict -}} {{- if .Values.apiRs.metrics.scrapeAnnotations -}} {{- $apiRsMetricsAnnotations = dict "prometheus.io/scrape" "true" "prometheus.io/path" .Values.apiRs.metrics.path "prometheus.io/port" (printf "%v" .Values.apiRs.port) -}} {{- $apiRsMetricsAnnotations = mergeOverwrite $apiRsMetricsAnnotations (.Values.apiRs.metrics.annotations | default dict) -}} {{- end -}} +{{- $apiRsEtl := .Values.apiRs.etl | default dict -}} +{{- $apiRsEtlEnv := list + (dict "name" "SLACK_ETL_ENABLED" "value" (dig "slack" "enabled" false $apiRsEtl)) + (dict "name" "SLACK_SYNC_INTERVAL_SECONDS" "value" (dig "slack" "syncIntervalSeconds" 3600 $apiRsEtl)) + (dict "name" "SLACK_SYNC_BACKFILL_LOOKBACK_DAYS" "value" (dig "slack" "syncBackfillLookbackDays" 30 $apiRsEtl)) + (dict "name" "SLACK_SYNC_THREAD_LOOKBACK_DAYS" "value" (dig "slack" "syncThreadLookbackDays" 3 $apiRsEtl)) + (dict "name" "SLACK_SYNC_INDEX_PRIVATE_CHANNELS" "value" (dig "slack" "indexPrivateChannels" false $apiRsEtl)) + (dict "name" "SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS" "value" (dig "slack" "excludedChannelPatterns" "" $apiRsEtl)) + (dict "name" "SLACK_ETL_ATTACHMENTS_ENABLED" "value" (dig "slack" "attachments" "enabled" true $apiRsEtl)) + (dict "name" "SLACK_ETL_ATTACHMENT_MAX_BYTES" "value" (dig "slack" "attachments" "maxBytes" 10485760 $apiRsEtl)) + (dict "name" "SLACK_BACKFILL_ENABLED" "value" (dig "slack" "backfill" "enabled" true $apiRsEtl)) + (dict "name" "SLACK_BACKFILL_INTERVAL_SECONDS" "value" (dig "slack" "backfill" "intervalSeconds" 600 $apiRsEtl)) + (dict "name" "SLACK_BACKFILL_CHANNEL_BATCH_LIMIT" "value" (dig "slack" "backfill" "channelBatchLimit" 50 $apiRsEtl)) + (dict "name" "SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB" "value" (dig "slack" "backfill" "channelPagesPerJob" 5 $apiRsEtl)) + (dict "name" "SLACK_RETENTION_ENABLED" "value" (dig "slack" "retention" "enabled" true $apiRsEtl)) + (dict "name" "SLACK_RETENTION_INTERVAL_MINUTES" "value" (dig "slack" "retention" "intervalMinutes" 60 $apiRsEtl)) + (dict "name" "SLACK_ETL_RETENTION_DAYS" "value" (dig "slack" "retention" "etlDays" 0 $apiRsEtl)) + (dict "name" "SLACK_DM_RETENTION_DAYS" "value" (dig "slack" "retention" "dmDays" 0 $apiRsEtl)) + (dict "name" "LINEAR_ETL_ENABLED" "value" (dig "linear" "enabled" false $apiRsEtl)) + (dict "name" "LINEAR_SYNC_INTERVAL_SECONDS" "value" (dig "linear" "syncIntervalSeconds" 14400 $apiRsEtl)) + (dict "name" "GOOGLE_DRIVE_ETL_ENABLED" "value" (dig "googleDrive" "enabled" false $apiRsEtl)) + (dict "name" "GOOGLE_DRIVE_SYNC_INTERVAL_SECONDS" "value" (dig "googleDrive" "syncIntervalSeconds" 14400 $apiRsEtl)) + (dict "name" "GOOGLE_CALENDAR_ETL_ENABLED" "value" (dig "googleCalendar" "enabled" false $apiRsEtl)) + (dict "name" "GOOGLE_CALENDAR_SYNC_INTERVAL_SECONDS" "value" (dig "googleCalendar" "syncIntervalSeconds" 14400 $apiRsEtl)) + (dict "name" "COMPANY_CONTEXT_DOCUMENTS_ENABLED" "value" (dig "companyContextDocuments" "enabled" true $apiRsEtl)) + (dict "name" "COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS" "value" (dig "companyContextDocuments" "intervalSeconds" 14400 $apiRsEtl)) + (dict "name" "COMPANY_CONTEXT_DOCUMENTS_MAX_WINDOW_SECONDS" "value" (dig "companyContextDocuments" "maxWindowSeconds" 21600 $apiRsEtl)) +-}} +{{- $apiRsEtlPassthroughNames := list -}} +{{- range $env := $apiRsEtlEnv -}} +{{- $apiRsEtlPassthroughNames = append $apiRsEtlPassthroughNames $env.name -}} +{{- end -}} +{{- with (get .Values.apiRs.extraEnv "SESSION_SANDBOX_PASSTHROUGH_ENV") -}} +{{- range $name := splitList "," (toString .) -}} +{{- $trimmedName := trim $name -}} +{{- if $trimmedName -}} +{{- $apiRsEtlPassthroughNames = append $apiRsEtlPassthroughNames $trimmedName -}} +{{- end -}} +{{- end -}} +{{- end -}} +{{- $apiRsEtlPassthroughNames = uniq $apiRsEtlPassthroughNames -}} apiVersion: v1 kind: ServiceAccount metadata: @@ -159,14 +216,12 @@ spec: annotations: checksum/infra-secrets: {{ include "centaur.infraSecretsChecksum" . }} checksum/overlay: {{ dict "overlay" .Values.overlay "overlays" .Values.overlays | toJson | sha256sum }} -{{- if $githubAppBrokerBootstrapEnabled }} - checksum/github-app-broker-bootstrap: {{ .Values.tokenBroker.githubApp | toJson | sha256sum }} -{{- end }} {{- with $apiRsMetricsAnnotations }} {{ toYaml . | nindent 8 }} {{- end }} labels: {{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 8 }} + centaur.ai/observability-enabled: "true" spec: terminationGracePeriodSeconds: {{ .Values.apiRs.terminationGracePeriodSeconds }} automountServiceAccountToken: true @@ -325,8 +380,76 @@ spec: secretKeyRef: name: {{ include "centaur.secretEnvName" . }} key: {{ printf "%sDATABASE_URL" .Values.secretManager.envPrefix }} + - name: CENTAUR_JWT_SIGNING_SECRET + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sCENTAUR_JWT_SIGNING_SECRET" .Values.secretManager.envPrefix }} + - name: CENTAUR_CONTROL_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%s%s" .Values.secretManager.envPrefix .Values.apiRs.controlApiKeySecretKey }} + # Normalize ingress service keys even when Secret data uses an + # operator prefix. Missing disabled-bot keys remain optional. + - name: SLACKBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sSLACKBOT_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: GITHUBBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sGITHUBBOT_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: LINEARBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sLINEARBOT_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: DISCORDBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sDISCORDBOT_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: TEAMSBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sTEAMSBOT_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: SLACK_FEEDBACK_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sSLACK_FEEDBACK_API_KEY" .Values.secretManager.envPrefix }} + optional: true + - name: WORKFLOW_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sWORKFLOW_API_KEY" .Values.secretManager.envPrefix }} + optional: true - name: BIND_ADDR value: {{ printf "0.0.0.0:%v" .Values.apiRs.port | quote }} + - name: SLACK_BOT_TOKEN + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sSLACK_BOT_TOKEN" .Values.secretManager.envPrefix }} + optional: true +{{- if $mcpPublicUrl }} + - name: CENTAUR_MCP_PUBLIC_URL + value: {{ $mcpPublicUrl | quote }} +{{- end }} +{{- if $console.publicUrl }} + - name: CENTAUR_CONSOLE_PUBLIC_URL + value: {{ $console.publicUrl | quote }} +{{- end }} - name: RUN_MIGRATIONS value: {{ .Values.apiRs.runMigrations | quote }} - name: IRON_CONTROL_SYNC_INFRA_SECRETS @@ -335,12 +458,18 @@ spec: value: info - name: TOOL_DIRS value: {{ join ":" $toolDirs | quote }} +{{- if gt (len $publicToolDirs) 0 }} + - name: KUBERNETES_PUBLIC_TOOL_DIRS + value: {{ join ":" $publicToolDirs | quote }} +{{- end }} - name: WORKFLOW_DIRS value: {{ $workflowDirs | quote }} {{- if $sandboxWorkflowDirs }} - name: KUBERNETES_WORKFLOW_DIRS value: {{ $sandboxWorkflowDirs | quote }} {{- end }} + - name: WORKFLOW_API_ALLOWED_NAMES + value: {{ .Values.apiRs.workflowApiAllowedNames | quote }} {{- if not (hasKey .Values.apiRs.extraEnv "WORKFLOW_ENABLE_MODE") }} - name: WORKFLOW_ENABLE_MODE value: {{ .Values.apiRs.workflowEnableMode | quote }} @@ -349,6 +478,17 @@ spec: - name: WORKFLOW_ALLOWED_NAMES value: {{ .Values.apiRs.workflowAllowedNames | quote }} {{- end }} +{{- range $env := $apiRsEtlEnv }} +{{- if not (hasKey $.Values.apiRs.extraEnv $env.name) }} + - name: {{ $env.name }} + value: {{ $env.value | toString | quote }} +{{- end }} +{{- end }} + # Forward the chart-rendered ETL workflow config into workflow-host + # sandboxes. Operators should set apiRs.etl.* values, not maintain + # SESSION_SANDBOX_PASSTHROUGH_ENV by hand. + - name: SESSION_SANDBOX_PASSTHROUGH_ENV + value: {{ join "," $apiRsEtlPassthroughNames | quote }} {{- if or .Values.overlay.systemPrompt .Values.overlay.image.repository }} - name: CENTAUR_OVERLAY_DIR value: {{ .Values.overlay.mountPath | quote }} @@ -369,6 +509,11 @@ spec: value: {{ .Values.apiRs.sandboxBackend | quote }} - name: SESSION_SANDBOX_WORKLOAD value: {{ .Values.apiRs.sandboxWorkload | quote }} + # Full boot-content identity. Unlike the tools compatibility view, + # this includes skills/workflow/prompt-only sources and effective + # repo-cache ref overrides. + - name: CENTAUR_SANDBOX_CONTENT_REVISION + value: {{ include "centaur.sandboxContentRevision" . | quote }} # Default harness for warm sandboxes. Per-session sandboxes run # their session's harness regardless (pinned via container args). - name: SESSION_SANDBOX_HARNESS @@ -379,8 +524,10 @@ spec: value: {{ .Values.apiRs.sandboxWarmPoolSize | quote }} - name: SESSION_SANDBOX_WARM_POOL_REPLENISH_INTERVAL_SECS value: {{ .Values.apiRs.sandboxWarmPoolReplenishIntervalSecs | quote }} - - name: SESSION_SANDBOX_IDLE_STOP_TTL_SECS - value: {{ .Values.apiRs.sandboxIdleStopTtlSecs | quote }} + - name: SESSION_SANDBOX_RUNNING_LIMIT + value: {{ .Values.apiRs.sandboxRunningLimit | quote }} + - name: SESSION_SANDBOX_HOT_IDLE_GRACE_SECS + value: {{ .Values.apiRs.sandboxHotIdleGraceSecs | quote }} - name: SESSION_SANDBOX_MAX_LIFETIME_SECS value: {{ .Values.apiRs.sandboxMaxLifetimeSecs | quote }} - name: SESSION_SANDBOX_REAP_INTERVAL_SECS @@ -389,6 +536,22 @@ spec: value: {{ .Values.apiRs.sandboxCleanupIntervalSecs | quote }} - name: SESSION_SANDBOX_IDLE_CLEANUP_BACKSTOP_SECS value: {{ .Values.apiRs.sandboxIdleCleanupBackstopSecs | quote }} + - name: SESSION_ACTIVITY_SUMMARY_ENABLED + value: {{ .Values.apiRs.activitySummary.enabled | quote }} +{{- if .Values.apiRs.activitySummary.enabled }} + - name: SESSION_ACTIVITY_SUMMARY_MODEL + value: {{ .Values.apiRs.activitySummary.model | quote }} + - name: SESSION_ACTIVITY_SUMMARY_OPENAI_BASE_URL + value: {{ .Values.apiRs.activitySummary.openaiBaseUrl | quote }} + - name: SESSION_ACTIVITY_SUMMARY_MIN_INTERVAL_SECS + value: {{ .Values.apiRs.activitySummary.minIntervalSecs | quote }} + - name: SESSION_ACTIVITY_SUMMARY_TIMEOUT_SECS + value: {{ .Values.apiRs.activitySummary.timeoutSecs | quote }} + - name: SESSION_ACTIVITY_SUMMARY_MAX_FACTS + value: {{ .Values.apiRs.activitySummary.maxFacts | quote }} + - name: SESSION_ACTIVITY_SUMMARY_MAX_OUTPUT_TOKENS + value: {{ .Values.apiRs.activitySummary.maxOutputTokens | quote }} +{{- end }} - name: SESSION_SANDBOX_K8S_NAMESPACE value: {{ .Release.Namespace | quote }} - name: SESSION_SANDBOX_IMAGE @@ -416,6 +579,8 @@ spec: value: {{ printf "%s:%s" .Values.ironProxy.image.repository .Values.ironProxy.image.tag | quote }} - name: KUBERNETES_IRON_PROXY_IMAGE_PULL_POLICY value: {{ .Values.ironProxy.image.pullPolicy | quote }} + - name: KUBERNETES_IRON_PROXY_UPSTREAM_DENY_CIDRS + value: {{ join "," .Values.ironProxy.upstreamDenyCidrs | quote }} - name: KUBERNETES_FIREWALL_CA_SECRET_NAME value: {{ include "centaur.trustedCaSecretName" . | quote }} - name: KUBERNETES_FIREWALL_CA_KEY_SECRET_NAME @@ -434,6 +599,8 @@ spec: value: {{ .Values.ironProxy.secretSource | quote }} - name: FIREWALL_MANAGER_SECRET_TTL value: {{ .Values.ironProxy.secretTtl | quote }} + - name: FIREWALL_MANAGER_SECRET_ENV_PREFIX + value: {{ .Values.secretManager.envPrefix | quote }} {{- if .Values.apiRs.opVault }} - name: OP_VAULT value: {{ .Values.apiRs.opVault | quote }} @@ -459,6 +626,9 @@ spec: {{- if and .Values.repoCache.enabled (gt (len $skillDirs) 0) }} {{- $sandboxEnvList = append $sandboxEnvList (dict "name" "CENTAUR_SKILL_DIRS" "value" (join ":" $skillDirs)) }} {{- end }} +{{- if and .Values.repoCache.enabled (gt (len $publicSkillDirs) 0) }} +{{- $sandboxEnvList = append $sandboxEnvList (dict "name" "CENTAUR_PUBLIC_SKILL_DIRS" "value" (join ":" $publicSkillDirs)) }} +{{- end }} {{- range $k, $v := .Values.sandbox.extraEnv }} {{- $sandboxEnvList = append $sandboxEnvList (dict "name" $k "value" ($v | toString)) }} {{- end }} @@ -508,6 +678,8 @@ spec: {{- end }} - name: KUBERNETES_TOOLS_SUBDIR value: {{ get $firstToolSource "subdir" | quote }} + - name: KUBERNETES_TOOLS_VISIBILITY + value: {{ get $firstToolSource "visibility" | default "private" | quote }} {{- if $extraToolSources }} - name: KUBERNETES_TOOLS_EXTRA_SOURCES value: {{ toJson $extraToolSources | quote }} @@ -515,6 +687,8 @@ spec: {{- if $toolsUseRepoCache }} - name: KUBERNETES_TOOLS_REPO_CACHE_PATH value: {{ .Values.repoCache.hostPath | quote }} + - name: KUBERNETES_TOOLS_AUTO_RELOAD + value: {{ .Values.repoCache.autoReload | quote }} {{- if $repoCacheUsePvc }} - name: KUBERNETES_TOOLS_REPO_CACHE_PVC value: {{ $repoCachePvcName | quote }} @@ -532,8 +706,10 @@ spec: {{- end }} {{- end }} {{- range $name, $value := .Values.apiRs.extraEnv }} +{{- if ne $name "SESSION_SANDBOX_PASSTHROUGH_ENV" }} - name: {{ $name }} value: {{ $value | quote }} +{{- end }} {{- end }} envFrom: - secretRef: @@ -601,6 +777,14 @@ spec: type: DirectoryOrCreate {{- end }} {{- end }} +{{- if $githubAppBrokerBootstrapEnabled }} + - name: github-app-broker-secret + secret: + secretName: {{ .Values.tokenBroker.githubApp.existingSecretName | quote }} + items: + - key: {{ .Values.tokenBroker.githubApp.existingSecretKeys.privateKey | quote }} + path: private-key +{{- end }} {{- if .Values.overlay.image.repository }} - name: overlay-root emptyDir: {} @@ -612,14 +796,6 @@ spec: - key: SYSTEM_PROMPT.md path: services/sandbox/SYSTEM_PROMPT.md {{- end }} -{{- if $githubAppBrokerBootstrapEnabled }} - - name: github-app-broker-secret - secret: - secretName: {{ .Values.tokenBroker.githubApp.existingSecretName | quote }} - items: - - key: {{ .Values.tokenBroker.githubApp.existingSecretKeys.privateKey | quote }} - path: private-key -{{- end }} {{- end }} --- apiVersion: v1 diff --git a/contrib/chart/templates/console-worker.yaml b/contrib/chart/templates/console-worker.yaml index 5164ce0a8..739e8d05c 100644 --- a/contrib/chart/templates/console-worker.yaml +++ b/contrib/chart/templates/console-worker.yaml @@ -3,6 +3,7 @@ {{- $secretEnv := include "centaur.secretEnvName" . }} {{- $prefix := .Values.secretManager.envPrefix }} {{- $apiRsName := include "centaur.componentName" (dict "root" . "component" "api-rs") }} +{{- $mcpPublicUrl := default .Values.slackbotv2.mcpPublicUrl .Values.apiRs.mcpPublicUrl }} # console background job worker — runs Solid Queue (`bin/jobs`) so # console's enqueued jobs actually execute. The most important of these is # the broker-credential OAuth refresh loop, which mints and refreshes the access @@ -85,6 +86,11 @@ spec: secretKeyRef: name: {{ $secretEnv }} key: {{ printf "%sIRON_CONTROL_DATABASE_URL" $prefix }} + - name: CENTAUR_CONSOLE_CENTAUR_DATABASE_URL + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sDATABASE_URL" $prefix }} - name: IRON_CONTROL_INITIAL_USER_EMAIL valueFrom: secretKeyRef: @@ -120,6 +126,19 @@ spec: secretKeyRef: name: {{ $secretEnv }} key: {{ printf "%sIRON_CONTROL_SECRET_KEY_BASE" $prefix }} + - name: CENTAUR_JWT_SIGNING_SECRET + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sCENTAUR_JWT_SIGNING_SECRET" $prefix }} +{{- if $console.publicUrl }} + - name: CENTAUR_CONSOLE_PUBLIC_URL + value: {{ $console.publicUrl | quote }} +{{- end }} +{{- if $mcpPublicUrl }} + - name: CENTAUR_MCP_PUBLIC_URL + value: {{ $mcpPublicUrl | quote }} +{{- end }} {{- if $console.googleOauth.enabled }} # Google OAuth app credentials, needed by the broker-credential # OAuth refresh loop. Gated on console.googleOauth.enabled. @@ -143,8 +162,15 @@ spec: - name: RAILS_LOG_TO_STDOUT value: "1" {{- if .Values.apiRs.enabled }} + - name: CENTAUR_API_URL + value: {{ printf "http://%s:%v" $apiRsName .Values.apiRs.port | quote }} - name: CENTAUR_CONSOLE_CENTAUR_API_URL value: {{ printf "http://%s:%v" $apiRsName .Values.apiRs.port | quote }} + - name: CENTAUR_CONSOLE_CENTAUR_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%s%s" .Values.secretManager.envPrefix .Values.apiRs.controlApiKeySecretKey }} {{- end }} securityContext: {{ toYaml .Values.containerSecurityContext | nindent 12 }} diff --git a/contrib/chart/templates/console.yaml b/contrib/chart/templates/console.yaml index 4268c6d56..3fc8a6841 100644 --- a/contrib/chart/templates/console.yaml +++ b/contrib/chart/templates/console.yaml @@ -4,6 +4,7 @@ {{- $prefix := .Values.secretManager.envPrefix }} {{- $dbName := $console.database.name }} {{- $apiRsName := include "centaur.componentName" (dict "root" . "component" "api-rs") }} +{{- $mcpPublicUrl := default .Values.slackbotv2.mcpPublicUrl .Values.apiRs.mcpPublicUrl }} # console — Rails control plane for authenticated API access and encrypted # secret storage. The chart owns the Deployment shape (image, port, env, # security context) so operators tune it via `helm upgrade`. It runs against a @@ -111,6 +112,31 @@ spec: secretKeyRef: name: {{ $secretEnv }} key: {{ printf "%sIRON_CONTROL_DATABASE_URL" $prefix }} + # Thread browsing reads api-rs session rows from its logical DB. + # Keep this under a Console-specific name so Rails' primary + # DATABASE_URL remains pointed at the console database. + - name: CENTAUR_CONSOLE_CENTAUR_DATABASE_URL + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sDATABASE_URL" $prefix }} + - name: CENTAUR_CONSOLE_SLACKBOTV2_USER_NAME + value: {{ .Values.slackbotv2.userName | quote }} + - name: CENTAUR_CONSOLE_SLACK_BOT_TOKEN + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sSLACK_BOT_TOKEN" $prefix }} + optional: true +{{- range $name := tuple "CLAUDE_MODEL" "CODEX_MODEL" }} +{{- if hasKey $.Values.sandbox.extraEnv $name }} + # Mirror the deployer's harness default-model override + # (sandbox.extraEnv) so the Threads view names the model threads + # without a recorded override actually ran on. + - name: {{ $name }} + value: {{ index $.Values.sandbox.extraEnv $name | toString | quote }} +{{- end }} +{{- end }} - name: IRON_CONTROL_INITIAL_USER_EMAIL valueFrom: secretKeyRef: @@ -146,6 +172,19 @@ spec: secretKeyRef: name: {{ $secretEnv }} key: {{ printf "%sIRON_CONTROL_SECRET_KEY_BASE" $prefix }} + - name: CENTAUR_JWT_SIGNING_SECRET + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sCENTAUR_JWT_SIGNING_SECRET" $prefix }} +{{- if $console.publicUrl }} + - name: CENTAUR_CONSOLE_PUBLIC_URL + value: {{ $console.publicUrl | quote }} +{{- end }} +{{- if $mcpPublicUrl }} + - name: CENTAUR_MCP_PUBLIC_URL + value: {{ $mcpPublicUrl | quote }} +{{- end }} {{- if $console.googleOauth.enabled }} # Google OAuth app credentials (sign-in + brokered token refresh). # Gated on console.googleOauth.enabled; the keys must exist in the @@ -160,6 +199,24 @@ spec: secretKeyRef: name: {{ $secretEnv }} key: {{ printf "%sIRON_CONTROL_GOOGLE_CLIENT_SECRET" $prefix }} +{{- end }} +{{- if $console.slackOauth.enabled }} + # Slack OIDC app credentials (console sign-in only — distinct from + # the DB-managed Slack OAuth app the bot/DM sync uses). Gated on + # console.slackOauth.enabled; the keys must exist in the shared + # infra Secret when enabled. Uses the modern CENTAUR_CONSOLE_* + # names (unlike the legacy IRON_CONTROL_* keys above) since this + # block postdates the rename. + - name: CENTAUR_CONSOLE_SLACK_CLIENT_ID + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sCENTAUR_CONSOLE_SLACK_CLIENT_ID" $prefix }} + - name: CENTAUR_CONSOLE_SLACK_CLIENT_SECRET + valueFrom: + secretKeyRef: + name: {{ $secretEnv }} + key: {{ printf "%sCENTAUR_CONSOLE_SLACK_CLIENT_SECRET" $prefix }} {{- end }} - name: RAILS_ENV value: {{ $console.railsEnv | quote }} @@ -174,8 +231,15 @@ spec: - name: RAILS_SERVE_STATIC_FILES value: "1" {{- if .Values.apiRs.enabled }} + - name: CENTAUR_API_URL + value: {{ printf "http://%s:%v" $apiRsName .Values.apiRs.port | quote }} - name: CENTAUR_CONSOLE_CENTAUR_API_URL value: {{ printf "http://%s:%v" $apiRsName .Values.apiRs.port | quote }} + - name: CENTAUR_CONSOLE_CENTAUR_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%s%s" .Values.secretManager.envPrefix .Values.apiRs.controlApiKeySecretKey }} {{- end }} ports: - containerPort: {{ $console.service.httpPort }} diff --git a/contrib/chart/templates/githubbot.yaml b/contrib/chart/templates/githubbot.yaml new file mode 100644 index 000000000..fa1b9b32a --- /dev/null +++ b/contrib/chart/templates/githubbot.yaml @@ -0,0 +1,172 @@ +{{- if .Values.githubbot.enabled }} +{{- $apiRsName := include "centaur.componentName" (dict "root" . "component" "api-rs") -}} +{{- if or .Values.githubbot.reviewPrompt .Values.githubbot.issuePrompt .Values.githubbot.managementPrompt }} +# Optional review / issue-work methodology overrides, delivered as mounted files +# the bot reads at boot via GITHUBBOT_REVIEW_PROMPT_FILE / _ISSUE_PROMPT_FILE. +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }}-prompts + labels: +{{ include "centaur.componentLabels" (dict "root" . "component" "githubbot") | nindent 4 }} +data: +{{- if .Values.githubbot.reviewPrompt }} + review-prompt.md: |- +{{- .Values.githubbot.reviewPrompt | nindent 4 }} +{{- end }} +{{- if .Values.githubbot.issuePrompt }} + issue-prompt.md: |- +{{- .Values.githubbot.issuePrompt | nindent 4 }} +{{- end }} +{{- if .Values.githubbot.managementPrompt }} + management-prompt.md: |- +{{- .Values.githubbot.managementPrompt | nindent 4 }} +{{- end }} +--- +{{- end }} +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }} + labels: +{{ include "centaur.componentLabels" (dict "root" . "component" "githubbot") | nindent 4 }} +spec: + replicas: {{ .Values.githubbot.replicaCount }} + selector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 6 }} + template: + metadata: + annotations: + checksum/infra-secrets: {{ include "centaur.infraSecretsChecksum" . }} +{{- if or .Values.githubbot.reviewPrompt .Values.githubbot.issuePrompt .Values.githubbot.managementPrompt }} + checksum/prompts: {{ dict "review" .Values.githubbot.reviewPrompt "issue" .Values.githubbot.issuePrompt "management" .Values.githubbot.managementPrompt | toJson | sha256sum }} +{{- end }} + labels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 8 }} + spec: + # Give in-flight agent turns time to finish on a rollout; the bot drains + # background work for GITHUBBOT_SHUTDOWN_DRAIN_MS (set below from this) on + # SIGTERM so a deploy doesn't drop a running CI fix / review / issue turn. + terminationGracePeriodSeconds: {{ .Values.githubbot.terminationGracePeriodSeconds }} + automountServiceAccountToken: false + {{- with .Values.global.imagePullSecrets }} + imagePullSecrets: +{{ toYaml . | nindent 8 }} + {{- end }} + containers: + - name: githubbot + image: {{ printf "%s:%s" .Values.githubbot.image.repository .Values.githubbot.image.tag | quote }} + imagePullPolicy: {{ .Values.githubbot.image.pullPolicy }} + env: + - name: PORT + value: "3001" + - name: CENTAUR_API_URL + value: {{ printf "http://%s:%v" $apiRsName .Values.apiRs.port | quote }} + # New threads without an explicit --claude/--amp/--codex flag run + # the deployment's default harness. + - name: GITHUBBOT_DEFAULT_HARNESS + value: {{ .Values.sandbox.harnessEngine | quote }} + # Personal access token for the bot's GitHub teammate account — kept + # distinct from the sandbox tool token so the bot acts as its own + # GitHub user (requestable as a reviewer, @-mentionable). + - name: GITHUB_TOKEN + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sGITHUBBOT_TOKEN" .Values.secretManager.envPrefix }} + # githubbot's own webhook signing secret (the GitHub repo/org webhook). + - name: GITHUB_WEBHOOK_SECRET + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sGITHUBBOT_WEBHOOK_SECRET" .Values.secretManager.envPrefix }} + - name: GITHUBBOT_API_KEY + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sGITHUBBOT_API_KEY" .Values.secretManager.envPrefix }} + - name: GITHUBBOT_DATABASE_URL + valueFrom: + secretKeyRef: + name: {{ include "centaur.secretEnvName" . }} + key: {{ printf "%sDATABASE_URL" .Values.secretManager.envPrefix }} + - name: GITHUB_BOT_USERNAME + value: {{ required "githubbot.userName is required (the bot account's GitHub login)" .Values.githubbot.userName | quote }} + # v2 PR self-management knobs. + - name: GITHUBBOT_AUTO_MERGE + value: {{ .Values.githubbot.autoMerge | quote }} + - name: GITHUBBOT_MERGE_METHOD + value: {{ .Values.githubbot.mergeMethod | quote }} +{{- if .Values.githubbot.escalationHandle }} + - name: GITHUBBOT_ESCALATION_HANDLE + value: {{ .Values.githubbot.escalationHandle | quote }} +{{- end }} +{{- if .Values.githubbot.reviewPrompt }} + - name: GITHUBBOT_REVIEW_PROMPT_FILE + value: /etc/githubbot/prompts/review-prompt.md +{{- end }} +{{- if .Values.githubbot.issuePrompt }} + - name: GITHUBBOT_ISSUE_PROMPT_FILE + value: /etc/githubbot/prompts/issue-prompt.md +{{- end }} +{{- if .Values.githubbot.managementPrompt }} + - name: GITHUBBOT_MANAGEMENT_PROMPT_FILE + value: /etc/githubbot/prompts/management-prompt.md +{{- end }} +{{- if .Values.githubbot.allowedAuthorAssociations }} + # author_association allowlist for the comment-mention path; unset = + # the bot's safe default (OWNER, MEMBER, COLLABORATOR). + - name: GITHUBBOT_ALLOWED_AUTHOR_ASSOCIATIONS + value: {{ .Values.githubbot.allowedAuthorAssociations | quote }} +{{- end }} + # Drain budget on shutdown — kept ~10s under the grace period so the + # final flush + exit still fit before the pod is force-killed. + - name: GITHUBBOT_SHUTDOWN_DRAIN_MS + value: {{ mul (sub (int .Values.githubbot.terminationGracePeriodSeconds) 10) 1000 | quote }} +{{- range $name, $value := .Values.githubbot.extraEnv }} + - name: {{ $name }} + value: {{ $value | quote }} +{{- end }} + ports: + - containerPort: 3001 + name: http + readinessProbe: + httpGet: + path: /health + port: 3001 + timeoutSeconds: 5 + livenessProbe: + httpGet: + path: /health + port: 3001 + timeoutSeconds: 5 + securityContext: +{{ toYaml .Values.containerSecurityContext | nindent 12 }} + resources: +{{ toYaml .Values.githubbot.resources | nindent 12 }} +{{- if or .Values.githubbot.reviewPrompt .Values.githubbot.issuePrompt .Values.githubbot.managementPrompt }} + volumeMounts: + - name: prompts + mountPath: /etc/githubbot/prompts + readOnly: true + volumes: + - name: prompts + configMap: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }}-prompts +{{- end }} +--- +apiVersion: v1 +kind: Service +metadata: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }} + labels: +{{ include "centaur.componentLabels" (dict "root" . "component" "githubbot") | nindent 4 }} +spec: + selector: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 4 }} + ports: + - name: http + port: 3001 + targetPort: 3001 +{{- end }} diff --git a/contrib/chart/templates/ingress.yaml b/contrib/chart/templates/ingress.yaml index 99bc17809..93e3a5ec1 100644 --- a/contrib/chart/templates/ingress.yaml +++ b/contrib/chart/templates/ingress.yaml @@ -1,19 +1,10 @@ -{{- $hasSlackbotV2 := .Values.slackbotv2.enabled -}} -{{- $hasLinearbot := .Values.linearbot.enabled -}} -{{- if and .Values.ingress.enabled (or $hasSlackbotV2 $hasLinearbot) }} -{{- if and .Values.ingress.defaultBackend $hasSlackbotV2 $hasLinearbot }} -{{- fail "ingress.defaultBackend cannot be used when both slackbotv2.enabled and linearbot.enabled are true; set ingress.defaultBackend=false to render path rules" }} -{{- end }} -{{- $ingressComponent := "slackbotv2" -}} -{{- if and (not $hasSlackbotV2) $hasLinearbot -}} -{{- $ingressComponent = "linearbot" -}} -{{- end -}} +{{- if and .Values.ingress.enabled (or .Values.slackbotv2.enabled .Values.linearbot.enabled .Values.githubbot.enabled) }} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: - name: {{ include "centaur.componentName" (dict "root" . "component" $ingressComponent) }} + name: {{ include "centaur.componentName" (dict "root" . "component" "slackbotv2") }} labels: -{{ include "centaur.componentLabels" (dict "root" . "component" $ingressComponent) | nindent 4 }} +{{ include "centaur.componentLabels" (dict "root" . "component" "slackbotv2") | nindent 4 }} {{- with .Values.ingress.annotations }} annotations: {{ toYaml . | nindent 4 }} @@ -25,7 +16,7 @@ spec: {{- if .Values.ingress.defaultBackend }} defaultBackend: service: - name: {{ include "centaur.componentName" (dict "root" . "component" $ingressComponent) }} + name: {{ include "centaur.componentName" (dict "root" . "component" "slackbotv2") }} port: number: 3001 {{- else }} @@ -35,7 +26,7 @@ spec: {{- end }} http: paths: -{{- if $hasLinearbot }} +{{- if .Values.linearbot.enabled }} # Linear webhook deliveries route to the linearbot; everything else # stays with slackbotv2 (the catch-all below). Funnel-style # defaultBackend ingresses cannot route both — use httpRoutes or a @@ -48,7 +39,18 @@ spec: port: number: 3001 {{- end }} -{{- if $hasSlackbotV2 }} +{{- if .Values.githubbot.enabled }} + # GitHub webhook deliveries (issue/PR comments + review requests) route + # to the githubbot. + - path: /api/webhooks/github + pathType: Prefix + backend: + service: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }} + port: + number: 3001 +{{- end }} +{{- if .Values.slackbotv2.enabled }} - path: / pathType: Prefix backend: diff --git a/contrib/chart/templates/networkpolicy.yaml b/contrib/chart/templates/networkpolicy.yaml index f171e8297..21934549a 100644 --- a/contrib/chart/templates/networkpolicy.yaml +++ b/contrib/chart/templates/networkpolicy.yaml @@ -71,6 +71,11 @@ spec: matchLabels: {{ include "centaur.componentSelectorLabels" (dict "root" . "component" "linearbot") | nindent 14 }} {{- end }} +{{- if .Values.githubbot.enabled }} + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 14 }} +{{- end }} {{- if .Values.discordbot.enabled }} - podSelector: matchLabels: @@ -150,7 +155,6 @@ spec: {{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 6 }} policyTypes: - Ingress - - Egress ingress: - from: {{- if .Values.slackbotv2.enabled }} @@ -163,6 +167,11 @@ spec: matchLabels: {{ include "centaur.componentSelectorLabels" (dict "root" . "component" "linearbot") | nindent 14 }} {{- end }} +{{- if .Values.githubbot.enabled }} + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 14 }} +{{- end }} {{- if .Values.discordbot.enabled }} - podSelector: matchLabels: @@ -185,12 +194,21 @@ spec: {{ include "centaur.componentSelectorLabels" (dict "root" . "component" "console-worker") | nindent 14 }} {{- end }} {{- end }} - # Sandboxes call back into the control plane. agent-k8s labels its pods - # centaur.ai/managed-by=api-rs (MANAGED_BY_VALUE in centaur-sandbox-agent-k8s), - # which also covers the per-sandbox iron-proxy pods. + # New sandboxes call back into the control plane only when their + # principal has the API server sandbox capability. + - podSelector: + matchLabels: + centaur.ai/api-server-enabled: "true" +{{- if .Values.networkPolicy.legacyManagedByApiServerAccess }} + # Transitional selector for pre-capability-label sandbox/proxy pods. + # Remove after the old warm pool and all assigned sessions are drained. - podSelector: matchLabels: centaur.ai/managed-by: api-rs + matchExpressions: + - key: centaur.ai/api-server-enabled + operator: DoesNotExist +{{- end }} {{- range .Values.networkPolicy.apiIngressSourceNamespaces }} - namespaceSelector: matchLabels: @@ -199,6 +217,141 @@ spec: ports: - protocol: TCP port: {{ .Values.apiRs.port }} +--- +# Sandboxes with the API server capability may call api-rs. New sandbox and +# proxy pods receive centaur.ai/api-server-enabled=true from api-rs only when +# the principal has the capability, so default-deny blocks new pods without it. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.fullname" . }}-sandbox-api-server + labels: +{{ include "centaur.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + centaur.ai/api-server-enabled: "true" + policyTypes: + - Egress + egress: + - to: + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 14 }} + ports: + - protocol: TCP + port: {{ .Values.apiRs.port }} +{{- if .Values.networkPolicy.legacyManagedByApiServerAccess }} +--- +# Transitional egress companion for sandbox/proxy pods created before +# centaur.ai/api-server-enabled was introduced. Default-on for staged upgrades; +# disable only after all legacy sessions and warm sandboxes have drained. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.fullname" . }}-legacy-managed-by-api-server + labels: +{{ include "centaur.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + centaur.ai/managed-by: api-rs + matchExpressions: + - key: centaur.ai/api-server-enabled + operator: DoesNotExist + policyTypes: + - Egress + egress: + - to: + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 14 }} + ports: + - protocol: TCP + port: {{ .Values.apiRs.port }} +{{- end }} +--- +{{- if .Values.networkPolicy.observabilityEgress.enabled }} +{{- if not .Values.networkPolicy.observabilityEgress.destinations }} +{{- fail "networkPolicy.observabilityEgress.destinations must contain at least one destination when networkPolicy.observabilityEgress.enabled is true" }} +{{- end }} +# Sandboxes with the observability capability may call the configured +# in-cluster observability backends. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.fullname" . }}-observability-egress + labels: +{{ include "centaur.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + centaur.ai/observability-enabled: "true" + policyTypes: + - Egress + egress: +{{- range $index, $destination := .Values.networkPolicy.observabilityEgress.destinations }} +{{- if not $destination.ports }} +{{- fail (printf "networkPolicy.observabilityEgress.destinations[%d].ports must contain at least one port" $index) }} +{{- end }} + - to: + - namespaceSelector: +{{- if $destination.namespaceSelector }} +{{ toYaml $destination.namespaceSelector | nindent 12 }} +{{- else }} + matchLabels: + kubernetes.io/metadata.name: {{ required (printf "networkPolicy.observabilityEgress.destinations[%d].namespace is required when namespaceSelector is unset" $index) $destination.namespace | quote }} +{{- end }} +{{- with $destination.podSelector }} + podSelector: +{{ toYaml . | nindent 12 }} +{{- end }} + ports: +{{- range $portIndex, $port := $destination.ports }} + - protocol: {{ default "TCP" $port.protocol }} + port: {{ required (printf "networkPolicy.observabilityEgress.destinations[%d].ports[%d].port is required" $index $portIndex) $port.port }} +{{- end }} +{{- end }} +--- +{{- end }} +{{- if .Values.networkPolicy.otlpEgress.enabled }} +# Pods with observability enabled may call the configured in-cluster OTLP +# collector. Without the centaur.ai/observability-enabled=true label, default +# deny blocks access to Laminar or other OTLP endpoints. +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.fullname" . }}-otlp-egress + labels: +{{ include "centaur.labels" . | nindent 4 }} +spec: + podSelector: + matchLabels: + centaur.ai/observability-enabled: "true" + policyTypes: + - Egress + egress: + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ required "networkPolicy.otlpEgress.namespace is required when networkPolicy.otlpEgress.enabled" .Values.networkPolicy.otlpEgress.namespace | quote }} + ports: + - protocol: TCP + port: {{ .Values.networkPolicy.otlpEgress.port }} +--- +{{- end }} +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.componentName" (dict "root" . "component" "api-rs") }}-egress + labels: +{{ include "centaur.componentLabels" (dict "root" . "component" "api-rs") | nindent 4 }} +spec: + podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 6 }} + policyTypes: + - Egress egress: {{- if .Values.postgres.enabled }} - to: @@ -230,18 +383,6 @@ spec: ports: - protocol: TCP port: {{ $console.service.httpPort }} -{{- end }} -{{- if .Values.networkPolicy.otlpEgress.enabled }} - # OTLP trace export (e.g. Laminar). The collector lives in another - # namespace; without this rule api-rs's own spans die with - # BatchSpanProcessor "network error" export failures. - - to: - - namespaceSelector: - matchLabels: - kubernetes.io/metadata.name: {{ required "networkPolicy.otlpEgress.namespace is required when networkPolicy.otlpEgress.enabled" .Values.networkPolicy.otlpEgress.namespace | quote }} - ports: - - protocol: TCP - port: {{ .Values.networkPolicy.otlpEgress.port }} {{- end }} - ports: - protocol: TCP @@ -352,6 +493,55 @@ spec: port: 443 --- {{- end }} +{{- if .Values.githubbot.enabled }} +apiVersion: networking.k8s.io/v1 +kind: NetworkPolicy +metadata: + name: {{ include "centaur.componentName" (dict "root" . "component" "githubbot") }} + labels: +{{ include "centaur.componentLabels" (dict "root" . "component" "githubbot") | nindent 4 }} +spec: + podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "githubbot") | nindent 6 }} + policyTypes: + - Ingress + - Egress + ingress: + - from: +{{- range $ingressSourceNamespaces }} + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: {{ . | quote }} +{{- end }} + ports: + - protocol: TCP + port: 3001 + egress: +{{- if .Values.apiRs.enabled }} + - to: + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "api-rs") | nindent 14 }} + ports: + - protocol: TCP + port: {{ .Values.apiRs.port }} +{{- end }} +{{- if .Values.postgres.enabled }} + - to: + - podSelector: + matchLabels: +{{ include "centaur.componentSelectorLabels" (dict "root" . "component" "postgres") | nindent 14 }} + ports: + - protocol: TCP + port: 5432 +{{- end }} + # GitHub REST API egress (direct HTTPS; githubbot does not use the firewall proxy). + - ports: + - protocol: TCP + port: 443 +--- +{{- end }} {{- if .Values.discordbot.enabled }} apiVersion: networking.k8s.io/v1 kind: NetworkPolicy diff --git a/contrib/chart/templates/repo-cache.yaml b/contrib/chart/templates/repo-cache.yaml index 8d0f919eb..474e2448d 100644 --- a/contrib/chart/templates/repo-cache.yaml +++ b/contrib/chart/templates/repo-cache.yaml @@ -1,12 +1,37 @@ {{- if .Values.repoCache.enabled }} {{- $overlaySources := include "centaur.overlaySources" . | fromJsonArray -}} {{- $repoCacheRepositories := list -}} +{{- $repoCacheRepositoryVisibilities := dict -}} +{{- $repoCacheRepositoryRefs := list -}} +{{- $repoCacheRepositoryRefRepos := list -}} +{{- range $repo, $ref := .Values.repoCache.repositoryRefs }} +{{- $repoCacheRepositoryRefs = append $repoCacheRepositoryRefs (printf "%s=%s" $repo $ref) -}} +{{- $repoCacheRepositoryRefRepos = append $repoCacheRepositoryRefRepos $repo -}} +{{- end }} {{- range .Values.repoCache.repositories }} -{{- $repoCacheRepositories = append $repoCacheRepositories . -}} +{{- if kindIs "string" . -}} +{{- $repo := . -}} +{{- if $repo -}} +{{- $repoCacheRepositories = append $repoCacheRepositories $repo -}} +{{- $_ := set $repoCacheRepositoryVisibilities $repo "private" -}} +{{- end -}} +{{- else -}} +{{- $repo := get . "repo" | default "" -}} +{{- if $repo -}} +{{- $repoCacheRepositories = append $repoCacheRepositories $repo -}} +{{- $_ := set $repoCacheRepositoryVisibilities $repo (include "centaur.repositoryVisibility" (get . "visibility")) -}} +{{- $ref := get . "ref" | default "" -}} +{{- if and $ref (not (has $repo $repoCacheRepositoryRefRepos)) -}} +{{- $repoCacheRepositoryRefs = append $repoCacheRepositoryRefs (printf "%s=%s" $repo $ref) -}} +{{- $repoCacheRepositoryRefRepos = append $repoCacheRepositoryRefRepos $repo -}} +{{- end -}} +{{- end -}} +{{- end -}} {{- end }} {{- range $source := $overlaySources }} {{- with (get $source "repo") }} {{- $repoCacheRepositories = append $repoCacheRepositories . -}} +{{- $_ := set $repoCacheRepositoryVisibilities . (get $source "visibility" | default "private") -}} {{- end }} {{- end }} {{- $repoCacheRepositories = uniq $repoCacheRepositories -}} @@ -19,12 +44,6 @@ {{- $repoCacheUsePvc := eq $repoCacheStorageType "persistentVolumeClaim" -}} {{- $repoCachePvcName := include "centaur.repoCachePvcName" . -}} {{- $repoCacheCreatePvc := and $repoCacheUsePvc .Values.repoCache.storage.persistentVolumeClaim.create (not .Values.repoCache.storage.persistentVolumeClaim.existingClaim) -}} -{{- $repoCacheRepositoryRefs := list -}} -{{- $repoCacheRepositoryRefRepos := list -}} -{{- range $repo, $ref := .Values.repoCache.repositoryRefs }} -{{- $repoCacheRepositoryRefs = append $repoCacheRepositoryRefs (printf "%s=%s" $repo $ref) -}} -{{- $repoCacheRepositoryRefRepos = append $repoCacheRepositoryRefRepos $repo -}} -{{- end }} {{- range $source := $overlaySources }} {{- $repo := get $source "repo" | default "" -}} {{- $ref := get $source "ref" | default "" -}} @@ -33,6 +52,10 @@ {{- $repoCacheRepositoryRefRepos = append $repoCacheRepositoryRefRepos $repo -}} {{- end }} {{- end }} +{{- $repoCacheRepositoryVisibilityEntries := list -}} +{{- range $repo := $repoCacheRepositories }} +{{- $repoCacheRepositoryVisibilityEntries = append $repoCacheRepositoryVisibilityEntries (printf "%s=%s" $repo (get $repoCacheRepositoryVisibilities $repo | default "private")) -}} +{{- end }} {{- if $repoCacheCreatePvc }} apiVersion: v1 kind: PersistentVolumeClaim @@ -91,128 +114,14 @@ spec: image: {{ printf "%s:%s" $repoCacheImageRepository $repoCacheImageTag | quote }} imagePullPolicy: {{ $repoCacheImagePullPolicy }} command: - - /bin/bash - - -ec - - | - set -o pipefail - token_file=/github-token/token - if [ -s "$token_file" ]; then - cat > /tmp/git-askpass <<'EOF' - #!/bin/sh - case "$1" in - *Username*) printf '%s\n' x-access-token ;; - *Password*) cat /github-token/token ;; - *) printf '\n' ;; - esac - EOF - chmod 0700 /tmp/git-askpass - export GIT_ASKPASS=/tmp/git-askpass - fi - git config --global --add safe.directory '*' - git config --global init.defaultBranch main - umask 022 - ready_file=/cache/.repo-cache-ready - ready_tmp="${ready_file}.tmp" - - repository_fingerprint() { - printf 'repositories=%s\nrepository_refs=%s\n' "$REPOSITORIES" "$REPOSITORY_REFS" - } - - repo_ref() { - local repo="$1" - for entry in $REPOSITORY_REFS; do - case "$entry" in - "$repo="*) printf '%s\n' "${entry#*=}"; return 0 ;; - esac - done - } - - checkout_repo() { - local repo="$1" - local target="$2" - local requested_ref - local default_branch - requested_ref="$(repo_ref "$repo")" - if [ -n "$requested_ref" ]; then - if git -C "$target" rev-parse --verify --quiet "origin/${requested_ref}^{commit}" >/dev/null; then - git -C "$target" checkout -q --detach "origin/${requested_ref}" - elif git -C "$target" rev-parse --verify --quiet "${requested_ref}^{commit}" >/dev/null; then - git -C "$target" checkout -q --detach "$requested_ref" - else - git -C "$target" -c gc.auto=0 fetch --prune --tags origin "$requested_ref" - git -C "$target" checkout -q --detach FETCH_HEAD - fi - return - fi - - default_branch="$(git -C "$target" symbolic-ref --short refs/remotes/origin/HEAD 2>/dev/null | sed 's#^origin/##' || true)" - if [ -z "$default_branch" ] || [ "$default_branch" = "(unknown)" ]; then - default_branch=main - fi - git -C "$target" checkout -q -B "$default_branch" "origin/$default_branch" - } - - sync_repo() { - local repo="$1" - local repo_url="https://github.com/${repo}.git" - local target="/cache/${repo}" - local tmp - # Deterministic temp name. This script runs as a Kubernetes - # container command, and Kubernetes collapses a doubled dollar - # sign to a single one during its own $(VAR) expansion before - # bash sees it, so a PID-based suffix is neither unique nor - # valid. Sync is sequential per pod, so a fixed name plus the - # rm -rf below is enough. - tmp="${target}.tmp" - - mkdir -p "$(dirname "$target")" - if git -C "$target" rev-parse --git-dir >/dev/null 2>&1; then - echo "Updating $repo" - git -C "$target" config gc.auto 0 || true - git -C "$target" remote set-url origin "$repo_url" || git -C "$target" remote add origin "$repo_url" - git -C "$target" -c gc.auto=0 fetch --prune --tags origin - git -C "$target" remote set-head origin -a || true - checkout_repo "$repo" "$target" - git -C "$target" clean -fd - else - echo "Cloning $repo" - # Also sweep any stale "${target}.tmp.*" dirs left by the - # previous PID-suffix scheme so they don't accumulate on disk. - rm -rf "${target}".tmp* "$target" - git clone --quiet "$repo_url" "$tmp" - git -C "$tmp" config gc.auto 0 - git -C "$tmp" -c gc.auto=0 fetch --prune --tags origin - git -C "$tmp" remote set-head origin -a || true - checkout_repo "$repo" "$tmp" - git -C "$tmp" clean -fd - mv "$tmp" "$target" - fi - } - - while true; do - sync_ok=1 - for repo in $REPOSITORIES; do - if ! sync_repo "$repo"; then - echo "Failed to sync $repo" >&2 - sync_ok=0 - fi - done - if [ "$sync_ok" = "1" ]; then - { - repository_fingerprint - printf 'synced_at=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" - } > "$ready_tmp" - mv "$ready_tmp" "$ready_file" - else - rm -f "$ready_tmp" "$ready_file" - fi - sleep "$SYNC_INTERVAL_SECONDS" - done + - /usr/local/bin/repo-cache-sync env: - name: REPOSITORIES value: {{ join " " $repoCacheRepositories | quote }} - name: REPOSITORY_REFS value: {{ join " " $repoCacheRepositoryRefs | quote }} + - name: REPOSITORY_VISIBILITIES + value: {{ join " " $repoCacheRepositoryVisibilityEntries | quote }} - name: SYNC_INTERVAL_SECONDS value: {{ .Values.repoCache.syncIntervalSeconds | quote }} - name: GIT_TERMINAL_PROMPT @@ -220,16 +129,8 @@ spec: readinessProbe: exec: command: - - /bin/bash - - -ec - - | - ready_file=/cache/.repo-cache-ready - expected="$(printf 'repositories=%s\nrepository_refs=%s\n' "$REPOSITORIES" "$REPOSITORY_REFS")" - actual="$(sed -n '1,2p' "$ready_file" 2>/dev/null || true)" - [ "$actual" = "$expected" ] || exit 1 - for repo in $REPOSITORIES; do - [ -d "/cache/$repo/.git" ] || exit 1 - done + - /usr/local/bin/repo-cache-sync + - --check-ready initialDelaySeconds: {{ .Values.repoCache.readinessProbe.initialDelaySeconds }} periodSeconds: {{ .Values.repoCache.readinessProbe.periodSeconds }} timeoutSeconds: {{ .Values.repoCache.readinessProbe.timeoutSeconds }} diff --git a/contrib/chart/templates/slackbotv2.yaml b/contrib/chart/templates/slackbotv2.yaml index 30c6ac6cb..3f73158a4 100644 --- a/contrib/chart/templates/slackbotv2.yaml +++ b/contrib/chart/templates/slackbotv2.yaml @@ -1,5 +1,6 @@ {{- if .Values.slackbotv2.enabled }} {{- $apiRsName := include "centaur.componentName" (dict "root" . "component" "api-rs") -}} +{{- $console := include "centaur.consoleValues" . | fromYaml -}} {{- $slackbotV2MetricsAnnotations := dict -}} {{- if .Values.slackbotv2.metrics.scrapeAnnotations -}} {{- $slackbotV2MetricsAnnotations = dict "prometheus.io/scrape" "true" "prometheus.io/path" .Values.slackbotv2.metrics.path "prometheus.io/port" "3001" -}} @@ -67,6 +68,15 @@ spec: key: {{ printf "%sDATABASE_URL" .Values.secretManager.envPrefix }} - name: SLACKBOTV2_USER_NAME value: {{ .Values.slackbotv2.userName | quote }} + - name: SLACKBOTV2_ACTIVITY_SUMMARY_STATUS_ENABLED + value: {{ .Values.apiRs.activitySummary.enabled | quote }} +{{- if $console.publicUrl }} + # Public origin of the Console UI (matches the Console's own + # CENTAUR_CONSOLE_PUBLIC_URL). When set, the first assistant message + # in a Slack thread gets an "Open session in Console" link. + - name: CENTAUR_CONSOLE_PUBLIC_URL + value: {{ $console.publicUrl | quote }} +{{- end }} {{- if .Values.slackbotv2.assistantStatus }} - name: SLACKBOTV2_ASSISTANT_STATUS value: {{ .Values.slackbotv2.assistantStatus | quote }} @@ -79,6 +89,16 @@ spec: - name: SLACKBOT_TRIGGER_BOT_ALLOWLIST value: {{ .Values.slackbotv2.triggerBotAllowlist | quote }} {{- end }} +{{- range $name := tuple "CLAUDE_MODEL" "CODEX_MODEL" }} +{{- if and (hasKey $.Values.sandbox.extraEnv $name) (not (hasKey $.Values.slackbotv2.extraEnv $name)) }} + # Mirror the deployer's harness default-model override + # (sandbox.extraEnv) so the Slack Console-link line names the model + # sandboxes actually run. slackbotv2.extraEnv wins if it sets the + # same variable. + - name: {{ $name }} + value: {{ index $.Values.sandbox.extraEnv $name | toString | quote }} +{{- end }} +{{- end }} {{- range $name, $value := .Values.slackbotv2.extraEnv }} - name: {{ $name }} value: {{ $value | quote }} diff --git a/contrib/chart/tests/test_overlay_image_compat.sh b/contrib/chart/tests/test_overlay_image_compat.sh new file mode 100644 index 000000000..1c87fccf4 --- /dev/null +++ b/contrib/chart/tests/test_overlay_image_compat.sh @@ -0,0 +1,214 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +chart_dir=${1:-contrib/chart} +scratch=$(mktemp -d -t centaur-overlay-chart.XXXXXXXXXX) +trap 'rm -rf "$scratch"' EXIT + +common=( + --set apiRs.enabled=true + --set console.enabled=false + --set slackbotv2.enabled=false + --set ironProxy.enabled=false +) + +helm template test "$chart_dir" "${common[@]}" >"$scratch/default.yaml" +if grep -q 'name: overlay-bootstrap' "$scratch/default.yaml"; then + echo "overlay image bootstrap must stay disabled by default" >&2 + exit 1 +fi +if ! grep -qF 'name: test-centaur-legacy-managed-by-api-server' "$scratch/default.yaml"; then + echo "legacy sandbox API access must remain enabled during staged rollout" >&2 + exit 1 +fi +if [[ "$(grep -cF 'operator: DoesNotExist' "$scratch/default.yaml")" -lt 2 ]]; then + echo "legacy API access must exclude new capability-labeled restricted pods" >&2 + exit 1 +fi +if grep -qF 'name: test-centaur-otlp-egress' "$scratch/default.yaml"; then + echo "OTLP egress must remain disabled by default" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set networkPolicy.otlpEgress.enabled=true \ + --set networkPolicy.otlpEgress.namespace=laminar \ + --set networkPolicy.otlpEgress.port=8000 >"$scratch/otlp.yaml" +for expected in \ + 'name: test-centaur-otlp-egress' \ + 'centaur.ai/observability-enabled: "true"' \ + 'kubernetes.io/metadata.name: "laminar"' \ + 'port: 8000'; do + if ! grep -qF "$expected" "$scratch/otlp.yaml"; then + echo "enabled OTLP policy is missing: $expected" >&2 + exit 1 + fi +done + +helm template test "$chart_dir" "${common[@]}" \ + --set networkPolicy.legacyManagedByApiServerAccess=false >"$scratch/no-legacy.yaml" +if grep -qF 'legacy-managed-by-api-server' "$scratch/no-legacy.yaml"; then + echo "legacy sandbox API access flag did not disable the transition policy" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set repoCache.enabled=false \ + --set overlay.image.repository=ghcr.io/tiplink/overlay \ + --set overlay.image.tag=sha-test >"$scratch/overlay.yaml" + +for expected in \ + 'name: overlay-bootstrap' \ + 'image: "ghcr.io/tiplink/overlay:sha-test"' \ + 'name: CENTAUR_OVERLAY_IMAGE' \ + 'name: CENTAUR_SANDBOX_OVERLAY_DIR' \ + 'name: TOOLS_OVERLAY_PATH' \ + 'name: overlay-root'; do + if ! grep -qF "$expected" "$scratch/overlay.yaml"; then + echo "rendered chart is missing transitional overlay contract: $expected" >&2 + exit 1 + fi +done + +if ! grep -qF 'value: "/app/tools:/app/overlay/org/tools"' "$scratch/overlay.yaml"; then + echo "legacy image tools must remain available when no repo-cache tool source is configured" >&2 + exit 1 +fi +if ! grep -qF 'value: "/app/workflows:/app/overlay/org/workflows"' "$scratch/overlay.yaml"; then + echo "legacy image workflows must remain available when no repo-cache workflow source is configured" >&2 + exit 1 +fi +if ! grep -qF 'value: "/home/agent/overlay/org/workflows"' "$scratch/overlay.yaml"; then + echo "legacy sandbox image workflows must remain available when no repo-cache workflow source is configured" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set repoCache.enabled=true \ + --set-string 'apiRs.workflowApiAllowedNames=reminder\,compliance_cdd_research' \ + --set overlay.image.repository=ghcr.io/tiplink/overlay \ + --set overlay.image.tag=sha-test \ + --set-string 'overlays.sources[0].repo=paradigmxyz/centaur' \ + --set-string 'overlays.sources[1].repo=TipLink/fineas-centaur-overlay' \ + >"$scratch/repo-and-image-overlay.yaml" + +if ! grep -qF 'value: "reminder,compliance_cdd_research"' "$scratch/repo-and-image-overlay.yaml"; then + echo "sandbox workflow API allowlist must render into api-rs" >&2 + exit 1 +fi + +if ! grep -qF 'value: "/app/overlay/org/tools:/var/lib/centaur/repos/paradigmxyz/centaur/tools:/var/lib/centaur/repos/TipLink/fineas-centaur-overlay/tools"' "$scratch/repo-and-image-overlay.yaml"; then + echo "repo-cache tools must follow and therefore override transitional image tools" >&2 + exit 1 +fi +if ! grep -qF 'value: "/var/lib/centaur/repos/paradigmxyz/centaur/workflows:/var/lib/centaur/repos/TipLink/fineas-centaur-overlay/workflows"' "$scratch/repo-and-image-overlay.yaml"; then + echo "API workflow discovery must use repo-cache sources exclusively when they are configured" >&2 + exit 1 +fi +if ! grep -qF 'value: "/home/agent/github/paradigmxyz/centaur/workflows:/home/agent/github/TipLink/fineas-centaur-overlay/workflows"' "$scratch/repo-and-image-overlay.yaml"; then + echo "sandbox workflow discovery must use repo-cache sources exclusively when they are configured" >&2 + exit 1 +fi +if grep -F 'value: "/var/lib/centaur/repos/' "$scratch/repo-and-image-overlay.yaml" | grep -qF '/app/overlay/org/workflows'; then + echo "API workflow discovery must not combine repo-cache and duplicate image workflow trees" >&2 + exit 1 +fi +if grep -F 'value: "/home/agent/github/' "$scratch/repo-and-image-overlay.yaml" | grep -qF '/home/agent/overlay/org/workflows'; then + echo "sandbox workflow discovery must not combine repo-cache and duplicate image workflow trees" >&2 + exit 1 +fi + +# A skills-only source never enters KUBERNETES_TOOLS_*, but its immutable ref +# must still rotate the sandbox content revision and therefore the warm key. +for revision in 1111111111111111111111111111111111111111 2222222222222222222222222222222222222222; do + helm template test "$chart_dir" "${common[@]}" \ + --set repoCache.enabled=true \ + --set-string 'overlays.sources[0].repo=TipLink/fin-skills' \ + --set-string "overlays.sources[0].ref=$revision" \ + --set-string 'overlays.sources[0].toolsSubdir=' \ + --set-string 'overlays.sources[0].workflowsSubdir=' \ + --set-string 'overlays.sources[0].skillsSubdir=centaur-skills' \ + >"$scratch/skills-$revision.yaml" +done +content_revision() { + awk ' + $1 == "-" && $2 == "name:" && $3 == "CENTAUR_SANDBOX_CONTENT_REVISION" { + getline + value = $2 + gsub(/^"|"$/, "", value) + print value + exit + } + ' "$1" +} +first_content_revision="$(content_revision "$scratch/skills-1111111111111111111111111111111111111111.yaml")" +second_content_revision="$(content_revision "$scratch/skills-2222222222222222222222222222222222222222.yaml")" +if [[ ! "$first_content_revision" =~ ^[0-9a-f]{64}$ || ! "$second_content_revision" =~ ^[0-9a-f]{64}$ ]]; then + echo "sandbox content revisions must be rendered SHA-256 values" >&2 + exit 1 +fi +if [[ "$first_content_revision" == "$second_content_revision" ]]; then + echo "skills-only source ref did not rotate the sandbox content revision" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set console.enabled=true >"$scratch/control-api-auth.yaml" +if ! grep -qF 'name: CENTAUR_CONTROL_API_KEY' "$scratch/control-api-auth.yaml"; then + echo "api-rs must receive the trusted Console control key" >&2 + exit 1 +fi +if [[ "$(grep -cF 'name: CENTAUR_CONSOLE_CENTAUR_API_KEY' "$scratch/control-api-auth.yaml")" -lt 2 ]]; then + echo "Console web and worker must receive the authenticated Centaur API client key" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set console.enabled=true \ + --set-string secretManager.envPrefix=PREFIX_ >"$scratch/prefixed-control-api-auth.yaml" +for expected in \ + 'name: CENTAUR_CONTROL_API_KEY' \ + 'key: PREFIX_CENTAUR_CONTROL_API_KEY' \ + 'name: SLACKBOT_API_KEY' \ + 'key: PREFIX_SLACKBOT_API_KEY' \ + 'name: SLACK_FEEDBACK_API_KEY' \ + 'key: PREFIX_SLACK_FEEDBACK_API_KEY' \ + 'name: WORKFLOW_API_KEY' \ + 'key: PREFIX_WORKFLOW_API_KEY' \ + 'name: FIREWALL_MANAGER_SECRET_ENV_PREFIX' \ + 'value: "PREFIX_"' \ + 'name: CENTAUR_CONSOLE_CENTAUR_API_KEY'; do + if ! grep -qF "$expected" "$scratch/prefixed-control-api-auth.yaml"; then + echo "prefixed control API auth render is missing: $expected" >&2 + exit 1 + fi +done + +helm template test "$chart_dir" "${common[@]}" \ + --set console.enabled=true \ + --set tokenBroker.githubApp.enabled=true \ + --set tokenBroker.githubApp.credentialId=fineas-github-app \ + --set tokenBroker.githubApp.existingSecretName=fineas-github-app \ + >"$scratch/github-app-alias.yaml" +if ! grep -qF 'value: "github-app fineas-github-app"' "$scratch/github-app-alias.yaml"; then + echo "GitHub App compatibility alias must not replace the canonical github-app credential" >&2 + exit 1 +fi + +helm template test "$chart_dir" "${common[@]}" \ + --set console.enabled=true \ + --set tokenBroker.githubApp.enabled=true \ + --set tokenBroker.githubApp.existingSecretName=fineas-github-app \ + >"$scratch/github-app.yaml" +for expected in \ + 'name: github-app-broker-bootstrap' \ + 'grant: "github_app_installation"' \ + 'name: GITHUB_APP_ID' \ + 'name: github-app-broker-secret' \ + 'secretName: "fineas-github-app"'; do + if ! grep -qF "$expected" "$scratch/github-app.yaml"; then + echo "rendered chart is missing GitHub App broker bootstrap contract: $expected" >&2 + exit 1 + fi +done diff --git a/contrib/chart/values.dev.yaml b/contrib/chart/values.dev.yaml index e5502b1c8..f6a06fe6a 100644 --- a/contrib/chart/values.dev.yaml +++ b/contrib/chart/values.dev.yaml @@ -32,6 +32,11 @@ apiRs: image: pullPolicy: IfNotPresent +console: + enabled: true + image: + pullPolicy: IfNotPresent + ironProxy: image: pullPolicy: IfNotPresent diff --git a/contrib/chart/values.schema.json b/contrib/chart/values.schema.json index c23707c5b..b01846220 100644 --- a/contrib/chart/values.schema.json +++ b/contrib/chart/values.schema.json @@ -6,7 +6,9 @@ "type": "object", "properties": { "enabled": { "type": "boolean" }, + "legacyManagedByApiServerAccess": { "type": "boolean" }, "replicaCount": { "type": "integer" }, + "publicUrl": { "type": "string" }, "railsEnv": { "type": "string" }, "image": { "type": "object", @@ -93,7 +95,11 @@ } }, "secretSource": { "type": "string" }, - "secretTtl": { "type": "string" } + "secretTtl": { "type": "string" }, + "upstreamDenyCidrs": { + "type": "array", + "items": { "type": "string" } + } } }, "toolServer": { @@ -103,6 +109,7 @@ "port": { "type": "integer" }, "repo": { "type": "string" }, "ref": { "type": "string" }, + "visibility": { "type": "string" }, "subdir": { "type": "string" }, "extraSources": { "type": "array", @@ -111,6 +118,7 @@ "properties": { "repo": { "type": "string" }, "ref": { "type": "string" }, + "visibility": { "type": "string" }, "subdir": { "type": "string" } }, "required": ["repo"] @@ -197,6 +205,7 @@ "properties": { "repo": { "type": "string" }, "ref": { "type": "string" }, + "visibility": { "type": "string" }, "toolsSubdir": { "type": "string" }, "workflowsSubdir": { "type": "string" }, "skillsSubdir": { "type": "string" }, @@ -220,10 +229,24 @@ "type": "object", "properties": { "enabled": { "type": "boolean" }, + "autoReload": { "type": "boolean" }, "hostPath": { "type": "string" }, "repositories": { "type": "array", - "items": { "type": "string" } + "items": { + "anyOf": [ + { "type": "string" }, + { + "type": "object", + "properties": { + "repo": { "type": "string" }, + "ref": { "type": "string" }, + "visibility": { "type": "string" } + }, + "required": ["repo"] + } + ] + } }, "repositoryRefs": { "type": "object", @@ -296,6 +319,79 @@ "type": "object", "properties": { "syncInfraSecrets": { "type": "boolean" }, + "mcpPublicUrl": { "type": "string" }, + "sandboxRunningLimit": { "type": "integer", "minimum": 0 }, + "sandboxHotIdleGraceSecs": { "type": "integer", "minimum": 0 }, + "etl": { + "type": "object", + "properties": { + "slack": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "syncIntervalSeconds": { "type": "integer" }, + "syncBackfillLookbackDays": { "type": "integer" }, + "syncThreadLookbackDays": { "type": "integer" }, + "indexPrivateChannels": { "type": "boolean" }, + "excludedChannelPatterns": { "type": "string" }, + "attachments": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "maxBytes": { "type": "integer" } + } + }, + "backfill": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "intervalSeconds": { "type": "integer" }, + "channelBatchLimit": { "type": "integer" }, + "channelPagesPerJob": { "type": "integer" } + } + }, + "retention": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "intervalMinutes": { "type": "integer" }, + "etlDays": { "type": "integer" }, + "dmDays": { "type": "integer" } + } + } + } + }, + "linear": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "syncIntervalSeconds": { "type": "integer" } + } + }, + "googleDrive": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "syncIntervalSeconds": { "type": "integer" } + } + }, + "googleCalendar": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "syncIntervalSeconds": { "type": "integer" } + } + }, + "companyContextDocuments": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "intervalSeconds": { "type": "integer" }, + "maxWindowSeconds": { "type": "integer" } + } + } + } + }, "metrics": { "type": "object", "properties": { @@ -316,6 +412,7 @@ "slackbotv2": { "type": "object", "properties": { + "mcpPublicUrl": { "type": "string" }, "metrics": { "type": "object", "properties": { @@ -366,6 +463,54 @@ "ingressControllerNamespaces": { "type": "array", "items": { "type": "string" } + }, + "apiServerPort": { "type": "integer" }, + "otlpEgress": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "namespace": { "type": "string" }, + "port": { "type": "integer" } + } + }, + "observabilityEgress": { + "type": "object", + "properties": { + "enabled": { "type": "boolean" }, + "destinations": { + "type": "array", + "minItems": 0, + "items": { + "type": "object", + "properties": { + "namespace": { "type": "string" }, + "namespaceSelector": { "type": "object" }, + "podSelector": { "type": "object" }, + "ports": { + "type": "array", + "minItems": 1, + "items": { + "type": "object", + "properties": { + "port": { + "oneOf": [ + { "type": "integer" }, + { "type": "string" } + ] + }, + "protocol": { + "type": "string", + "enum": ["TCP", "UDP", "SCTP"] + } + }, + "required": ["port"] + } + } + }, + "required": ["ports"] + } + } + } } } } diff --git a/contrib/chart/values.yaml b/contrib/chart/values.yaml index f81f8a0d0..bf13ee363 100644 --- a/contrib/chart/values.yaml +++ b/contrib/chart/values.yaml @@ -28,6 +28,17 @@ ironProxy: # a single listener (routing by database name); NetworkPolicies allow this # one port so callers can reach it. pgPort: 5432 + # CIDRs iron-proxy must never dial upstream, even when the host allowlist + # matches. Keep the local/IMDS defaults and include the default k3s pod/service + # CIDRs so sandboxes cannot proxy back into in-cluster services like + # onepassword-connect. + upstreamDenyCidrs: + - "169.254.169.254/32" + - "fd00:ec2::254/128" + - "127.0.0.0/8" + - "::1/128" + - "10.42.0.0/16" + - "10.43.0.0/16" secretSource: onepassword secretTtl: 10m @@ -43,11 +54,14 @@ toolServer: port: 8001 # owner/name of the GitHub repo carrying the tools tree. REQUIRED when enabled; # override for forks. Private repos additionally need githubToken below. - repo: TipLink/centaur + repo: paradigmxyz/centaur # Branch, tag, or commit to check out; empty = the repo's default branch. Pin # this to the tool set the api-rs image was built with so the creds api-rs # grants match the tools the sandbox installs. ref: "" + # Repo visibility for sandbox repo-cache access. Defaults to private; set to + # public only for repos safe to expose to principals with sandbox_repo_cache=public. + visibility: private # Subdirectory in the repo holding the tools tree (mounted at /app/tools). subdir: tools # Additional repo/subdir sources copied after the base tools source. Duplicate @@ -56,6 +70,7 @@ toolServer: extraSources: [] # - repo: tempoxyz/centaur-tempo # ref: "" + # visibility: private # subdir: tools # Image the tools-bootstrap init container runs in; empty fields fall back to # sandbox.image, which carries git and install-tool-shims. @@ -85,6 +100,12 @@ toolServer: console: enabled: false replicaCount: 1 + # Public URL users reach in a browser (e.g. https://console.example.com), used + # as CENTAUR_CONSOLE_PUBLIC_URL. Set this when console is exposed behind + # Tailscale/Ingress so MCP OAuth issuer metadata and JWT validation agree. + # When set, the slackbotv2 deployment also links the first assistant message + # in a Slack thread to the Console session view; leave empty to omit the link. + publicUrl: "" image: repository: centaur-console tag: latest @@ -106,6 +127,17 @@ console: # _SECRET), which must exist or the console pods CreateContainerConfigError. googleOauth: enabled: false + # Slack as an OIDC identity provider for console sign-in. Off by default — + # not every deployment uses Slack to log in. When enabled, the + # CENTAUR_CONSOLE_SLACK_CLIENT_ID and CENTAUR_CONSOLE_SLACK_CLIENT_SECRET + # env vars are sourced from the shared infra Secret (keys + # CENTAUR_CONSOLE_SLACK_CLIENT_ID / _SECRET), which + # must exist or the console pods CreateContainerConfigError. This is the + # Sign-in-with-Slack OIDC app (scopes openid/email/profile, redirect URL + # /auth/slack/callback) — not the DB-managed Slack OAuth app the + # bot and DM sync use. + slackOauth: + enabled: false # OAuth app slug whose user-level Slack credentials are used by the DM sync # worker. Keep this aligned with the Console OAuth app users authorize. slackDmSync: @@ -146,22 +178,18 @@ console: replicaCount: 1 resources: {} -# Managed short-lived token credentials for sandbox proxy rules. The built-in -# proxy fragment uses a `github-app` broker credential for github.com and -# api.github.com so long-lived sandboxes do not keep using an expired -# GITHUB_TOKEN value from pod startup. +# Managed short-lived token credentials for sandbox proxy rules. This +# transitional bootstrap upserts the GitHub App installation credential into +# the upstream Console credential-grant registry before api-rs starts. tokenBroker: enabled: false githubApp: enabled: false - # The built-in sandbox proxy fragment references `github-app`. Keep this - # value unless you also change the proxy fragment. If an environment already - # used another ID, put it here; the chart always creates `github-app` too. + # Compatibility alias provisioned alongside the canonical `github-app` + # credential. The built-in infra role always consumes `github-app`; do not + # add a second same-scope proxy source for this alias. credentialId: github-app extraCredentialIds: [] - # The unqualified built-in reference resolves in the default namespace. - # `infra` is included as a compatibility alias for deployments that already - # provisioned grants/secrets there. credentialNamespaces: - default - infra @@ -197,9 +225,11 @@ overlay: mountPath: /app/overlay/org sandboxMountPath: /home/agent/overlay/org systemPrompt: "" + # Deprecated transitional path retained for staged migration and rollback. + # Prefer overlays.sources for new deployments. image: repository: "" - tag: "" + tag: latest pullPolicy: IfNotPresent sourcePath: /overlay @@ -214,14 +244,19 @@ overlay: # only need `repo`; set `ref` to a branch, tag, or commit when you do not want # the repo's default branch. Directories missing from a repo are skipped at # runtime. Set a subdir to "" to explicitly disable that surface for a source. +# Repo visibility defaults to private; set visibility: public only for repos +# safe to expose to principals with sandbox_repo_cache=public. overlays: sources: [] # - repo: paradigmxyz/centaur # ref: "" + # visibility: public # - repo: your-org/centaur-overlay # ref: main + # visibility: private # - repo: your-org/workflows-only # ref: main + # visibility: private # toolsSubdir: "" # skillsSubdir: "" @@ -250,7 +285,10 @@ sandbox: # overrides keep working credentials). Per-session sandboxes always run # their session's harness via container args; the sandbox image CMD only # matters for containers started outside api-rs. - harnessEngine: claudecode + harnessEngine: codex + # Copied into every sandbox pod. CLAUDE_MODEL / CODEX_MODEL set here (the + # harness default-model overrides) are also mirrored into slackbotv2 and the + # Console so their model displays match what sandboxes actually run. extraEnv: {} stateVolume: enabled: false @@ -264,6 +302,7 @@ sandbox: repoCache: enabled: true + autoReload: true hostPath: /var/lib/centaur/repos storage: # hostPath preserves the existing DaemonSet/node-local behavior. Set to @@ -279,9 +318,14 @@ repoCache: - ReadWriteMany storageClassName: "" size: 20Gi - # Additional repos to cache. toolServer.repo is cached automatically when the - # tool server is enabled. + # Additional repos to cache. Entries can be strings (visibility defaults to + # private) or objects with repo/ref/visibility. toolServer.repo is cached + # automatically when the tool server is enabled. repositories: [] + # - paradigmxyz/public-docs + # - repo: your-org/public-docs + # ref: main + # visibility: public repositoryRefs: {} syncIntervalSeconds: 30 image: @@ -330,12 +374,28 @@ apiRs: tag: latest pullPolicy: Always port: 8080 + # Shared trusted control-plane key used by Console for authenticated session + # and workflow administration. Sandbox principals never receive this value. + controlApiKeySecretKey: CENTAUR_CONTROL_API_KEY + # Public/local MCP endpoint advertised through OAuth protected-resource + # metadata. Empty falls back to slackbotv2.mcpPublicUrl. + mcpPublicUrl: "" runMigrations: true sandboxBackend: agent-k8s sandboxWorkload: codex-app-server sandboxReadyTimeoutSecs: 90 sandboxWarmPoolSize: 3 sandboxWarmPoolReplenishIntervalSecs: 5 + # Optional operator-controlled input to the automatically computed sandbox + # boot-content revision. Normally the chart's source/image manifest is + # sufficient; bump this only for an external boot-copied input not represented + # by chart values. + sandboxContentRevision: "" + # Capacity manager: when nonzero, api-rs keeps observed running-like + # sandboxes at or below this limit by discarding ready warm sandboxes first, + # then pausing least-recently-active idle session sandboxes. 0 disables. + sandboxRunningLimit: 0 + sandboxHotIdleGraceSecs: 300 # When true, api-rs upserts the shared iron-control infra role and its # backing secrets at startup and on tool-secret reconciliation intervals. # Set false when multiple Centaur instances share one console/1Password @@ -346,16 +406,66 @@ apiRs: # in workflowAllowedNames as a comma/whitespace-separated string. workflowEnableMode: all workflowAllowedNames: "" - # Reaper: stop sandboxes idle-paused longer than the idle TTL or older than - # the max lifetime. 0 disables that sweep. Interval must be >= 1. - sandboxIdleStopTtlSecs: 10800 # 3 hours + # Sandbox-facing workflow control API policy. Run creation/list/get/cancel + # additionally requires a Console JWT scoped to input.thread_key's Slack + # channel. Empty is deny-all; list only workflows intentionally callable by + # agent tools. Scheduled and authenticated webhook execution are unaffected. + workflowApiAllowedNames: "" + # Scheduled ETL workflow configuration. The chart renders these into api-rs + # env so workflow discovery sees the right schedules, then derives the + # workflow-host passthrough list from the same non-secret config. + etl: + slack: + enabled: false + syncIntervalSeconds: 3600 + syncBackfillLookbackDays: 30 + syncThreadLookbackDays: 3 + indexPrivateChannels: false + excludedChannelPatterns: "" + attachments: + enabled: true + maxBytes: 10485760 + backfill: + enabled: true + intervalSeconds: 600 + channelBatchLimit: 50 + channelPagesPerJob: 5 + retention: + enabled: true + intervalMinutes: 60 + etlDays: 0 + dmDays: 0 + linear: + enabled: false + syncIntervalSeconds: 14400 + googleDrive: + enabled: false + syncIntervalSeconds: 14400 + googleCalendar: + enabled: false + syncIntervalSeconds: 14400 + companyContextDocuments: + enabled: true + intervalSeconds: 14400 + maxWindowSeconds: 21600 + # Reaper: stop sandboxes older than the max lifetime, regardless of whether + # they are running or suspended. 0 disables the sweep. Interval must be >= 1. sandboxMaxLifetimeSecs: 259200 # 3 days sandboxReapIntervalSecs: 300 # Cleanup worker: stop unreferenced session/warm-pool sandboxes after two # consecutive sweeps and restore idle-pauses lost across api-rs restarts. - # 0 disables the corresponding arm. + # The idle backstop is only used when older execution rows have no persisted + # idle_timeout_ms. 0 disables the corresponding arm. sandboxCleanupIntervalSecs: 300 sandboxIdleCleanupBackstopSecs: 21600 # 6 hours + activitySummary: + enabled: false + model: gpt-5.4-nano + openaiBaseUrl: https://api.openai.com/v1 + minIntervalSecs: 20 + timeoutSecs: 5 + maxFacts: 12 + maxOutputTokens: 128 ironProxy: mode: enabled metrics: @@ -396,6 +506,9 @@ slackbotv2: pullPolicy: Always userName: centaur assistantStatus: "" + # Public/local MCP endpoint shown in MCP SSO setup messages. The default + # assumes users connect through `kubectl port-forward ... 3000:8080`. + mcpPublicUrl: "http://localhost:3000" externalOrgAllowlist: "" triggerBotAllowlist: "" metrics: @@ -423,6 +536,48 @@ linearbot: extraEnv: {} resources: {} +# Chat SDK GitHub bot — GitHub teammate (PAT) ingress. Receives issue/PR comment +# webhooks on /api/webhooks/github (the @chat-adapter/github adapter), answers in +# the comment thread, and runs a review when the bot account is requested as a +# reviewer. Forwards sessions to the api-rs control plane (:8080). Disabled by +# default: requires the GITHUBBOT_TOKEN (PAT) and GITHUBBOT_WEBHOOK_SECRET secrets +# (see contrib/scripts/bootstrap-k8s-secrets.sh). userName must be the bot +# account's GitHub login so @-mention and review-request matching work. +githubbot: + enabled: false + replicaCount: 1 + image: + repository: centaur-githubbot + tag: latest + pullPolicy: Always + userName: "" + # v2 PR self-management (only acts on PRs assigned to the bot account). + # Auto-merge respects branch protection and is paused per-PR by the hold label + # / draft status. + autoMerge: true + mergeMethod: squash + # Fallback @handle (no leading @) the bot tags when it gives up; empty = none. + escalationHandle: "" + # Optional review / issue-work methodology overrides. When set, the chart writes + # each into a ConfigMap, mounts it as a file, and points the bot at it via + # GITHUBBOT_REVIEW_PROMPT_FILE / GITHUBBOT_ISSUE_PROMPT_FILE. Empty = the bot's + # bundled default is used. Supply the whole methodology; it's used verbatim. + reviewPrompt: "" + issuePrompt: "" + # Extra guidance prepended to owned-PR management turns (CI-fix / conflict / + # address-review) — the riskiest autonomous surface. Same delivery as the + # review/issue prompts (ConfigMap + mounted file). Empty = built-in preamble only. + managementPrompt: "" + # GitHub author_association values allowed to drive the comment-mention path. + # Empty = the bot's safe default (OWNER, MEMBER, COLLABORATOR). Use "*" to allow + # everyone (e.g. a fully-private repo). Comma-separated. + allowedAuthorAssociations: "" + # How long (seconds) to let in-flight turns finish on shutdown before the pod is + # killed, so a deploy doesn't drop running CI fixes / reviews / issue work. + terminationGracePeriodSeconds: 120 + extraEnv: {} + resources: {} + # Discord chat ingress — mirrors slackbotv2, forwards to the api-rs control plane (:8080) over a # persistent Discord Gateway connection. Off by default; needs a Discord app + Message Content # Intent + a guild allowlist. Always exactly one replica (singleton Gateway session). @@ -509,6 +664,11 @@ httpRoutes: [] networkPolicy: enabled: true + # Transitional compatibility for sandboxes created before capability labels. + # New agent/proxy pods always carry api-server-enabled=true|false, and this + # policy selects only managed pods where that label is absent. Disable only + # after every old ready/claimed/running sandbox has drained. + legacyManagedByApiServerAccess: true ingressSourceNamespaces: [] apiIngressSourceNamespaces: [] ingressControllerNamespaces: @@ -520,15 +680,42 @@ networkPolicy: # on clusters whose API server endpoint already listens on 443. apiServerPort: 6443 # Egress to an in-cluster OTLP trace collector (e.g. Laminar) in another - # namespace. api-rs needs this to export its own spans; sandboxes get their - # equivalent per-sandbox rule from api-rs, derived from the OTLP endpoint in - # sandbox.extraEnv. The collector's own namespace must also admit ingress - # from this namespace's pods. + # namespace. The policy selects only pods labeled + # centaur.ai/observability-enabled=true, matching observabilityEgress. api-rs + # always receives that label; sandbox pods receive it only when their + # principal has observability enabled. Iron-proxy pods use their explicit + # per-sandbox OTLP egress rule instead of this selector. The collector's own + # namespace must also admit ingress from this namespace's pods. otlpEgress: enabled: false # kubernetes.io/metadata.name of the collector's namespace, e.g. "laminar". namespace: "" port: 8000 + # Egress to observability backends such as VictoriaLogs or VictoriaMetrics. + # This policy selects only pods labeled centaur.ai/observability-enabled=true. + # Sandboxes receive that label only when their principal has the observability + # sandbox capability. + observabilityEgress: + enabled: false + destinations: [] + # Example: + # destinations: + # - namespace: observability + # podSelector: + # matchLabels: + # app.kubernetes.io/instance: vls + # app.kubernetes.io/name: victoria-logs-single + # ports: + # - port: 9428 + # protocol: TCP + # - namespace: observability + # podSelector: + # matchLabels: + # app.kubernetes.io/instance: vms + # app.kubernetes.io/name: victoria-metrics-single + # ports: + # - port: 8428 + # protocol: TCP podSecurityContext: fsGroupChangePolicy: OnRootMismatch diff --git a/contrib/scripts/bootstrap-k8s-secrets.sh b/contrib/scripts/bootstrap-k8s-secrets.sh index 76dcdf8ea..5a958f55a 100755 --- a/contrib/scripts/bootstrap-k8s-secrets.sh +++ b/contrib/scripts/bootstrap-k8s-secrets.sh @@ -9,8 +9,14 @@ usage() { Usage: scripts/bootstrap-k8s-secrets.sh [--namespace NAMESPACE] [--force] Creates the required local-dev Kubernetes infra Secrets consumed by the Helm chart. -Requires OP_SERVICE_ACCOUNT_TOKEN, OP_VAULT, SLACK_BOT_TOKEN, -SLACK_SIGNING_SECRET, and SLACKBOT_API_KEY in the shell environment. +When creating centaur-infra-env from scratch or with --force, requires +OP_SERVICE_ACCOUNT_TOKEN, OP_VAULT, SLACK_BOT_TOKEN, SLACK_SIGNING_SECRET, +and SLACKBOT_API_KEY in the shell environment. A stable +CENTAUR_CONTROL_API_KEY is generated for Console/API administration, and a +distinct SLACK_FEEDBACK_API_KEY is generated for the sandbox feedback tool. +Existing Secrets are only topped up with newly generated keys when absent. +All configured control, bot, workflow, and feedback API credentials must be +pairwise distinct; api-rs refuses to start if trust lanes share a value. Optional 1Password Connect bootstrap (when ironProxy.manager.secretSource is set to onepassword-connect in the Helm values): @@ -40,6 +46,16 @@ Optional Linear bot bootstrap (consumed when linearbot.enabled=true): LINEARBOT_API_KEY bearer the bot sends to api-rs; auto-generated when absent +Optional GitHub ingress bootstrap (consumed when githubbot.enabled=true): + GITHUBBOT_TOKEN personal access token for the bot's GitHub + teammate account; required together with the + webhook secret (partial config fails fast). Kept + distinct from GITHUB_TOKEN (the repo-cache / + sandbox tool token) so the bot acts as its own user. + GITHUBBOT_WEBHOOK_SECRET signing secret from the GitHub repo/org webhook + GITHUBBOT_API_KEY bearer the bot sends to api-rs; auto-generated + when absent + Optional Discord ingress bootstrap (consumed when discordbot.enabled=true): DISCORD_BOT_TOKEN when set, seeds the discordbot keys; requires DISCORD_PUBLIC_KEY and DISCORD_APPLICATION_ID @@ -127,11 +143,6 @@ rand_hex() { require_cmd kubectl require_cmd openssl -require_env OP_SERVICE_ACCOUNT_TOKEN -require_env OP_VAULT -require_env SLACK_BOT_TOKEN -require_env SLACK_SIGNING_SECRET -require_env SLACKBOT_API_KEY # Linear config is optional but must be complete: a token without the webhook # secret (or vice versa) deploys a linearbot that boots and then rejects every @@ -141,6 +152,14 @@ if [[ -n "${LINEAR_ACCESS_TOKEN:-}" || -n "${LINEARBOT_WEBHOOK_SECRET:-}" ]]; th require_env LINEARBOT_WEBHOOK_SECRET fi +# GitHub bot config is optional but must be complete: a PAT without the webhook +# secret (or vice versa) deploys a githubbot that boots and then rejects every +# delivery, which reads as silence. +if [[ -n "${GITHUBBOT_TOKEN:-}" || -n "${GITHUBBOT_WEBHOOK_SECRET:-}" ]]; then + require_env GITHUBBOT_TOKEN + require_env GITHUBBOT_WEBHOOK_SECRET +fi + # Discord keys are optional as a group, but partial configuration would silently # seed empty values and crashloop the bot at deploy time instead of failing here. if [[ -n "${DISCORD_BOT_TOKEN:-}" ]]; then @@ -163,6 +182,14 @@ delete_if_forced centaur-firewall-ca delete_if_forced centaur-firewall-ca-key delete_if_forced centaur-onepassword-connect-credentials +if ! secret_exists centaur-infra-env; then + require_env OP_SERVICE_ACCOUNT_TOKEN + require_env OP_VAULT + require_env SLACK_BOT_TOKEN + require_env SLACK_SIGNING_SECRET + require_env SLACKBOT_API_KEY +fi + secret_key_present() { local key="$1" local value @@ -171,6 +198,35 @@ secret_key_present() { [[ -n "$value" ]] } +assert_service_api_keys_distinct() { + local keys=( + CENTAUR_CONTROL_API_KEY + SLACKBOT_API_KEY + GITHUBBOT_API_KEY + LINEARBOT_API_KEY + DISCORDBOT_API_KEY + TEAMSBOT_API_KEY + WORKFLOW_API_KEY + SLACK_FEEDBACK_API_KEY + ) + local names=() + local values=() + local key value index + for key in "${keys[@]}"; do + value="$(kubectl -n "$NAMESPACE" get secret centaur-infra-env \ + -o "jsonpath={.data.${key}}" 2>/dev/null || true)" + [[ -n "$value" ]] || continue + for index in "${!values[@]}"; do + if [[ "$value" == "${values[$index]}" ]]; then + echo "${names[$index]} and $key must contain distinct service credentials" >&2 + return 1 + fi + done + names+=("$key") + values+=("$value") + done +} + if secret_exists centaur-infra-env; then patch_data=() if [[ -n "${OP_CONNECT_TOKEN:-}" ]]; then @@ -182,6 +238,12 @@ if secret_exists centaur-infra-env; then if ! secret_key_present IRON_BROKER_TOKEN; then patch_data+=("\"IRON_BROKER_TOKEN\":\"$(rand_hex | base64 | tr -d '\n')\"") fi + if ! secret_key_present CENTAUR_CONTROL_API_KEY; then + patch_data+=("\"CENTAUR_CONTROL_API_KEY\":\"$(rand_hex | base64 | tr -d '\n')\"") + fi + if ! secret_key_present SLACK_FEEDBACK_API_KEY; then + patch_data+=("\"SLACK_FEEDBACK_API_KEY\":\"$(rand_hex | base64 | tr -d '\n')\"") + fi if [[ -n "${LOCAL_DEV_API_KEY:-}" ]]; then patch_data+=("\"LOCAL_DEV_API_KEY\":\"$(printf '%s' "$LOCAL_DEV_API_KEY" | base64 | tr -d '\n')\"") fi @@ -249,6 +311,9 @@ if secret_exists centaur-infra-env; then if ! secret_key_present IRON_CONTROL_SECRET_KEY_BASE; then patch_data+=("\"IRON_CONTROL_SECRET_KEY_BASE\":\"$(printf '%s%s' "$(rand_hex)" "$(rand_hex)" | base64 | tr -d '\n')\"") fi + if ! secret_key_present CENTAUR_JWT_SIGNING_SECRET; then + patch_data+=("\"CENTAUR_JWT_SIGNING_SECRET\":\"$(printf '%s%s' "$(rand_hex)" "$(rand_hex)" | base64 | tr -d '\n')\"") + fi # Linear bot credentials. Set whenever present so the OAuth token can be # rotated; the api-rs bearer is generated once and kept stable. if [[ -n "${LINEAR_ACCESS_TOKEN:-}" ]]; then @@ -260,11 +325,23 @@ if secret_exists centaur-infra-env; then patch_data+=("\"LINEARBOT_API_KEY\":\"$(rand_hex | base64 | tr -d '\n')\"") fi fi + # GitHub bot credentials. The PAT + webhook secret are set whenever present so + # they can be rotated; the api-rs bearer is generated once and kept stable. + if [[ -n "${GITHUBBOT_TOKEN:-}" ]]; then + patch_data+=("\"GITHUBBOT_TOKEN\":\"$(printf '%s' "$GITHUBBOT_TOKEN" | base64 | tr -d '\n')\"") + patch_data+=("\"GITHUBBOT_WEBHOOK_SECRET\":\"$(printf '%s' "$GITHUBBOT_WEBHOOK_SECRET" | base64 | tr -d '\n')\"") + if [[ -n "${GITHUBBOT_API_KEY:-}" ]]; then + patch_data+=("\"GITHUBBOT_API_KEY\":\"$(printf '%s' "$GITHUBBOT_API_KEY" | base64 | tr -d '\n')\"") + elif ! secret_key_present GITHUBBOT_API_KEY; then + patch_data+=("\"GITHUBBOT_API_KEY\":\"$(rand_hex | base64 | tr -d '\n')\"") + fi + fi if [[ "${#patch_data[@]}" -gt 0 ]]; then patch_json="{\"data\":{$(IFS=,; echo "${patch_data[*]}")}}" kubectl -n "$NAMESPACE" patch secret centaur-infra-env --type merge -p "$patch_json" >/dev/null echo "Updated optional keys in Secret centaur-infra-env in namespace $NAMESPACE" fi + assert_service_api_keys_distinct echo "Secret centaur-infra-env already exists in namespace $NAMESPACE; leaving unchanged" else POSTGRES_PASSWORD="$(rand_hex)" @@ -285,6 +362,8 @@ else --from-literal=SLACK_BOT_TOKEN="$SLACK_BOT_TOKEN" --from-literal=SLACK_SIGNING_SECRET="$SLACK_SIGNING_SECRET" --from-literal=SLACKBOT_API_KEY="$SLACKBOT_API_KEY" + --from-literal=CENTAUR_CONTROL_API_KEY="$(rand_hex)" + --from-literal=SLACK_FEEDBACK_API_KEY="$(rand_hex)" --from-literal=POSTGRES_PASSWORD="$POSTGRES_PASSWORD" --from-literal=DATABASE_URL="$DATABASE_URL" --from-literal=IRON_CONTROL_DATABASE_URL="$IRON_CONTROL_DATABASE_URL" @@ -295,6 +374,7 @@ else --from-literal=IRON_CONTROL_AR_ENCRYPTION_DETERMINISTIC_KEY="$(rand_hex)" --from-literal=IRON_CONTROL_AR_ENCRYPTION_KEY_DERIVATION_SALT="$(rand_hex)" --from-literal=IRON_CONTROL_SECRET_KEY_BASE="$(rand_hex)$(rand_hex)" + --from-literal=CENTAUR_JWT_SIGNING_SECRET="$(rand_hex)$(rand_hex)" ) if [[ -n "${DISCORD_BOT_TOKEN:-}" ]]; then secret_args+=( @@ -326,7 +406,13 @@ else secret_args+=(--from-literal=LINEARBOT_WEBHOOK_SECRET="$LINEARBOT_WEBHOOK_SECRET") secret_args+=(--from-literal=LINEARBOT_API_KEY="${LINEARBOT_API_KEY:-$(rand_hex)}") fi + if [[ -n "${GITHUBBOT_TOKEN:-}" ]]; then + secret_args+=(--from-literal=GITHUBBOT_TOKEN="$GITHUBBOT_TOKEN") + secret_args+=(--from-literal=GITHUBBOT_WEBHOOK_SECRET="$GITHUBBOT_WEBHOOK_SECRET") + secret_args+=(--from-literal=GITHUBBOT_API_KEY="${GITHUBBOT_API_KEY:-$(rand_hex)}") + fi kubectl "${secret_args[@]}" >/dev/null + assert_service_api_keys_distinct echo "Created Secret centaur-infra-env in namespace $NAMESPACE" fi diff --git a/crates/harness-server/src/amp.rs b/crates/harness-server/src/amp.rs index 63ecd93cc..7b7ea8f06 100644 --- a/crates/harness-server/src/amp.rs +++ b/crates/harness-server/src/amp.rs @@ -1,6 +1,7 @@ use std::collections::HashSet; use std::env; use std::process::Command as ProcessCommand; +use std::time::Duration; use codex_app_server_protocol::UserInput; use serde_json::json; @@ -165,8 +166,11 @@ impl HarnessServer for AmpHarness { Ok(normalizer.normalize(event)) } - fn finish_turn_on_assistant_end_turn(&self) -> bool { - true + /// Amp's stream has no native `result` event: the terminal assistant stop + /// IS the end of the turn, so complete immediately (a settle window would + /// add its full length to every turn). + fn terminal_assistant_stop_settle(&self) -> Option { + Some(Duration::ZERO) } } diff --git a/crates/harness-server/src/anthropic.rs b/crates/harness-server/src/anthropic.rs index ea2dd476e..a43f3b483 100644 --- a/crates/harness-server/src/anthropic.rs +++ b/crates/harness-server/src/anthropic.rs @@ -19,13 +19,19 @@ pub enum AnthropicStreamEvent { #[serde(default)] is_partial: bool, message: AnthropicMessage, + #[serde(default)] + parent_tool_use_id: Option, }, User { message: AnthropicMessage, tool_use_result: Option, + #[serde(default)] + parent_tool_use_id: Option, }, StreamEvent { event: AnthropicRawStreamEvent, + #[serde(default)] + parent_tool_use_id: Option, }, Result { subtype: Option, @@ -61,11 +67,34 @@ impl AnthropicStreamEvent { AnthropicRawStreamEvent::MessageDelta { delta: Some(delta), .. }, + .. } => delta.stop_reason.as_deref(), _ => None, } } + /// The Task tool-use id owning this event when it belongs to a subagent + /// sidechain. Sidechain messages stop with their own `end_turn` while the + /// parent turn keeps running, so they must never settle the turn. + pub fn parent_tool_use_id(&self) -> Option<&str> { + match self { + Self::Assistant { + parent_tool_use_id, .. + } + | Self::User { + parent_tool_use_id, .. + } + | Self::StreamEvent { + parent_tool_use_id, .. + } => parent_tool_use_id.as_deref(), + _ => None, + } + } + + pub fn is_sidechain(&self) -> bool { + self.parent_tool_use_id().is_some() + } + pub fn token_usage(&self) -> Option { match self { Self::Assistant { message, .. } => { @@ -140,7 +169,7 @@ pub struct AnthropicEventNormalizer { impl AnthropicEventNormalizer { pub fn normalize(&mut self, event: AnthropicStreamEvent) -> NormalizedEvent { match event { - AnthropicStreamEvent::StreamEvent { event } => self.normalize_stream_event(event), + AnthropicStreamEvent::StreamEvent { event, .. } => self.normalize_stream_event(event), event => self.normalize_message_event(event), } } @@ -218,6 +247,7 @@ impl AnthropicEventNormalizer { AnthropicStreamEvent::Assistant { is_partial, message, + .. } => NormalizedEvent::AssistantMessage { partial: is_partial, stop_reason: message.stop_reason.clone(), @@ -226,6 +256,7 @@ impl AnthropicEventNormalizer { AnthropicStreamEvent::User { message, tool_use_result, + .. } => { let tool_use_result = tool_use_result.as_ref(); let results = message diff --git a/crates/harness-server/src/claude.rs b/crates/harness-server/src/claude.rs index e4a897cc0..d93c8c5a3 100644 --- a/crates/harness-server/src/claude.rs +++ b/crates/harness-server/src/claude.rs @@ -1,6 +1,7 @@ use std::env; use std::path::PathBuf; use std::process::Command as ProcessCommand; +use std::time::Duration; use codex_app_server_protocol::UserInput; use serde_json::json; @@ -275,8 +276,28 @@ impl HarnessServer for ClaudeCodeHarness { normalizer: &mut Self::EventNormalizer, event: Self::Event, ) -> Result> { + // Subagent sidechains (Task tool) interleave their own messages into + // the stream, ending with their own `end_turn` while the parent turn + // keeps running. Letting them through corrupts the pending-text state + // (their message ids clobber the main chain's) and their stop reasons + // would settle — and with the stop fallback, terminate — the parent + // turn. The subagent's report reaches the turn through the main + // chain's Task tool result. + if event.is_sidechain() { + return Ok(Vec::new()); + } Ok(normalizer.normalize(event)) } + + /// Claude Code normally ends a turn with a native `result` line, but + /// streams have been observed to stop at `message_delta.stop_reason` + /// without one (leaving the execution hung as "thinking" forever). Wait a + /// short window for the native result before completing on the stop, so + /// the trailing `result` is consumed by this turn instead of instantly + /// terminating the next one. + fn terminal_assistant_stop_settle(&self) -> Option { + Some(Duration::from_secs(2)) + } } #[cfg(test)] diff --git a/crates/harness-server/src/codex.rs b/crates/harness-server/src/codex.rs index f55a877d5..28d274b0d 100644 --- a/crates/harness-server/src/codex.rs +++ b/crates/harness-server/src/codex.rs @@ -1,7 +1,12 @@ +use std::collections::HashSet; use std::env; use std::io::{self, BufRead, Write}; use std::process::{Child, ChildStdin, Command as ProcessCommand, Stdio}; -use std::sync::mpsc::{self, Receiver, RecvTimeoutError}; +use std::sync::{ + Arc, + atomic::{AtomicBool, Ordering}, + mpsc::{self, Receiver, RecvTimeoutError}, +}; use std::thread; use std::time::Duration; @@ -13,8 +18,6 @@ use crate::server::{BlocksCommand, BlocksState, parse_blocks_line_with_state, wr use crate::util::write_value; use crate::{AppServerRuntime, HarnessServerError, Result}; -type BlocksCommandResult = Result; - #[derive(Debug, Clone, Copy)] pub struct CodexHarnessServer { fallback_model_provider: &'static str, @@ -124,11 +127,76 @@ pub(crate) fn run_codex_blocks_server(config: CodexHarnessServer) -> Result<()> // thread start (the app-server protocol has no per-turn provider), so this // lets a later conflicting override be surfaced rather than silently dropped. let mut thread_provider: Option = None; - let blocks_rx = spawn_blocks_input_reader(); + let (command_tx, command_rx) = mpsc::channel(); + let (active_turn_tx, active_turn_rx) = mpsc::channel(); + let turn_active = Arc::new(AtomicBool::new(false)); + + { + let turn_active = Arc::clone(&turn_active); + thread::spawn(move || { + let stdin = io::stdin(); + let mut blocks_state = BlocksState::default(); + for raw in stdin.lock().lines() { + let Ok(line) = raw else { + break; + }; + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + + match parse_blocks_line_with_state(trimmed, &mut blocks_state) { + Ok(BlocksCommand::Interrupt) if turn_active.load(Ordering::SeqCst) => { + if active_turn_tx + .send(CodexActiveTurnRequest::Interrupt) + .is_err() + { + break; + } + } + Ok(command @ BlocksCommand::User { .. }) + if turn_active.load(Ordering::SeqCst) => + { + if active_turn_tx + .send(CodexActiveTurnRequest::Steer(Box::new(command))) + .is_err() + { + break; + } + } + Ok(command @ BlocksCommand::User { .. }) => { + turn_active.store(true, Ordering::SeqCst); + if command_tx + .send(CodexBlocksReaderInput::Command(command)) + .is_err() + { + break; + } + } + Ok(command) => { + if command_tx + .send(CodexBlocksReaderInput::Command(command)) + .is_err() + { + break; + } + } + Err(error) => { + if command_tx + .send(CodexBlocksReaderInput::Error(error.to_string())) + .is_err() + { + break; + } + } + } + } + }); + } - while let Ok(command) = blocks_rx.recv() { - match command { - Ok(BlocksCommand::User { + while let Ok(input) = command_rx.recv() { + match input { + CodexBlocksReaderInput::Command(BlocksCommand::User { input, client_user_message_id, model, @@ -137,52 +205,56 @@ pub(crate) fn run_codex_blocks_server(config: CodexHarnessServer) -> Result<()> trace_context, }) => { let traceparent = trace_context.effective_traceparent(); - if codex.is_none() { - otel::configure_codex_otel_for_startup(&trace_context)?; - let mut child = CodexJsonRpcChild::spawn(&trace_context)?; - initialize_codex( - &mut child, + turn_active.store(true, Ordering::SeqCst); + let result = (|| -> Result<()> { + if codex.is_none() { + otel::configure_codex_otel_for_startup(&trace_context)?; + let mut child = CodexJsonRpcChild::spawn(&trace_context)?; + initialize_codex( + &mut child, + &mut stdout, + &mut request_id, + traceparent.as_deref(), + )?; + codex = Some(child); + } + let model = model.or_else(|| config.default_model()); + let model_provider = + config.model_provider_for(provider.as_deref(), model.as_deref()); + run_codex_user_turn( + codex.as_mut().expect("codex initialized"), &mut stdout, &mut request_id, + &mut thread_id, + &mut thread_provider, + input, + client_user_message_id, + (model, model_provider), + provider, + reasoning, + &active_turn_rx, traceparent.as_deref(), - )?; - codex = Some(child); - } - let model = model.or_else(|| config.default_model()); - let model_provider = - config.model_provider_for(provider.as_deref(), model.as_deref()); - if let Err(error) = run_codex_user_turn( - codex.as_mut().expect("codex initialized"), - &mut stdout, - &mut request_id, - &mut thread_id, - &mut thread_provider, - input, - client_user_message_id, - (model, model_provider), - provider, - reasoning, - traceparent.as_deref(), - &blocks_rx, - ) { + ) + })(); + turn_active.store(false, Ordering::SeqCst); + drain_codex_active_turn_requests(&active_turn_rx); + if let Err(error) = result { let fallback_thread_id = thread_id.as_deref().unwrap_or("codex"); eprintln!("Codex blocks turn failed: {error:#}"); write_blocks_error(&mut stdout, fallback_thread_id, "turn", error.to_string())?; } } - Ok(BlocksCommand::Interrupt) => { - eprintln!( - "Codex blocks interrupt ignored: no active stdin reader while a turn runs" - ); + CodexBlocksReaderInput::Command(BlocksCommand::Interrupt) => { + eprintln!("Codex blocks interrupt ignored: no active turn runs"); } - Ok(BlocksCommand::AttachmentChunk) => {} - Err(error) => { - eprintln!("invalid Codex blocks input: {error:#}"); + CodexBlocksReaderInput::Command(BlocksCommand::AttachmentChunk) => {} + CodexBlocksReaderInput::Error(error) => { + eprintln!("invalid Codex blocks input: {error}"); write_blocks_error( &mut stdout, thread_id.as_deref().unwrap_or("codex"), "input", - error.to_string(), + error, )?; } } @@ -191,28 +263,24 @@ pub(crate) fn run_codex_blocks_server(config: CodexHarnessServer) -> Result<()> Ok(()) } -fn spawn_blocks_input_reader() -> Receiver { - let (tx, rx) = mpsc::channel(); - thread::spawn(move || { - let stdin = io::stdin(); - let mut blocks_state = BlocksState::default(); - for raw in stdin.lock().lines() { - let command = match raw { - Ok(line) => { - let trimmed = line.trim(); - if trimmed.is_empty() { - continue; - } - parse_blocks_line_with_state(trimmed, &mut blocks_state) - } - Err(error) => Err(error.into()), - }; - if tx.send(command).is_err() { - break; - } - } - }); - rx +enum CodexBlocksReaderInput { + Command(BlocksCommand), + Error(String), +} + +enum CodexActiveTurnRequest { + Interrupt, + Steer(Box), +} + +#[derive(Default)] +struct CodexActiveRequestState { + interrupt_request_id: Option, + steer_request_ids: HashSet, +} + +fn drain_codex_active_turn_requests(rx: &Receiver) { + while rx.try_recv().is_ok() {} } fn initialize_codex( @@ -251,8 +319,8 @@ fn run_codex_user_turn( model_and_provider: (Option, String), requested_provider: Option, reasoning: Option, + active_turn_rx: &Receiver, traceparent: Option<&str>, - blocks_rx: &Receiver, ) -> Result<()> { let (model, model_provider) = model_and_provider; if thread_id.is_none() { @@ -296,7 +364,7 @@ fn run_codex_user_turn( } // Per-turn reasoning effort (codex `turn/start.effort`), parsed from the // `-rsn` message flag. Values match codex's ReasoningEffort enum - // (none|minimal|low|medium|high|xhigh); validation happens upstream. + // (none|minimal|low|medium|high|xhigh|max); validation happens upstream. if let Some(reasoning) = reasoning { params["effort"] = Value::String(reasoning); } @@ -327,8 +395,9 @@ fn run_codex_user_turn( stdout, thread_id.as_deref().unwrap_or_default(), &turn_id, + active_turn_rx, request_id, - blocks_rx, + traceparent, )? { TurnTermination::Done => return Ok(()), TurnTermination::RetriableEngineError { withheld } => { @@ -352,93 +421,6 @@ fn run_codex_user_turn( } } -fn send_codex_turn_steer( - codex: &mut CodexJsonRpcChild, - request_id: &mut i64, - thread_id: &str, - turn_id: &str, - input: Vec, - client_user_message_id: Option, - traceparent: Option<&str>, -) -> Result { - let steer_request_id = next_request_id(request_id); - let mut params = json!({ - "threadId": thread_id, - "expectedTurnId": turn_id, - "input": input, - }); - if let Some(client_user_message_id) = client_user_message_id { - params["clientUserMessageId"] = Value::String(client_user_message_id); - } - codex.send_request(steer_request_id, "turn/steer", params, traceparent)?; - Ok(steer_request_id) -} - -fn send_codex_turn_interrupt( - codex: &mut CodexJsonRpcChild, - request_id: &mut i64, - thread_id: &str, - turn_id: &str, -) -> Result { - let interrupt_request_id = next_request_id(request_id); - codex.send_request( - interrupt_request_id, - "turn/interrupt", - json!({ - "threadId": thread_id, - "turnId": turn_id, - }), - None, - )?; - Ok(interrupt_request_id) -} - -fn handle_active_blocks_command( - command: BlocksCommandResult, - codex: &mut CodexJsonRpcChild, - stdout: &mut W, - request_id: &mut i64, - thread_id: &str, - turn_id: &str, -) -> Result<()> { - match command { - Ok(BlocksCommand::User { - input, - client_user_message_id, - model, - provider, - reasoning, - trace_context, - }) => { - if model.is_some() || provider.is_some() || reasoning.is_some() { - eprintln!( - "Codex blocks steering ignored turn-start-only overrides: \ - model={model:?} provider={provider:?} reasoning={reasoning:?}" - ); - } - let traceparent = trace_context.effective_traceparent(); - send_codex_turn_steer( - codex, - request_id, - thread_id, - turn_id, - input, - client_user_message_id, - traceparent.as_deref(), - )?; - } - Ok(BlocksCommand::Interrupt) => { - send_codex_turn_interrupt(codex, request_id, thread_id, turn_id)?; - } - Ok(BlocksCommand::AttachmentChunk) => {} - Err(error) => { - eprintln!("invalid Codex blocks input during active turn: {error:#}"); - write_blocks_error(stdout, thread_id, "input", error.to_string())?; - } - } - Ok(()) -} - fn start_or_resume_thread( codex: &mut CodexJsonRpcChild, stdout: &mut W, @@ -524,7 +506,10 @@ impl CodexJsonRpcChild { .take() .ok_or(HarnessServerError::CodexStderrUnavailable)?; thread::spawn(move || { - let mut parent_stderr = io::stderr().lock(); + // Unlocked handle on purpose: this child lives across turns, so + // holding the StderrLock for the copy's lifetime would block every + // eprintln! in the server until the child exits. + let mut parent_stderr = io::stderr(); let _ = io::copy(&mut stderr, &mut parent_stderr); }); @@ -619,27 +604,45 @@ impl CodexJsonRpcChild { stdout: &mut W, thread_id: &str, turn_id: &str, + active_turn_rx: &Receiver, request_id: &mut i64, - blocks_rx: &Receiver, + traceparent: Option<&str>, ) -> Result { let mut guard = TurnGuard::default(); + let mut active_requests = CodexActiveRequestState::default(); loop { - while let Ok(command) = blocks_rx.try_recv() { - handle_active_blocks_command( - command, self, stdout, request_id, thread_id, turn_id, - )?; - } - - let Some(value) = self.read_value_timeout(Duration::from_millis(50))? else { - continue; + let value = match self.read_value_timeout(Duration::from_millis(50))? { + Some(value) => value, + None => { + self.forward_pending_active_requests( + active_turn_rx, + &mut active_requests, + request_id, + thread_id, + turn_id, + traceparent, + )?; + continue; + } }; if is_server_request(&value) { self.send_error_response(&value)?; continue; } - if response_id(&value).is_some() { - if let Some(error) = value.get("error") { - eprintln!("Codex active-turn request failed: {error}"); + if let Some(id) = response_id(&value) { + if Some(id) == active_requests.interrupt_request_id { + if let Some(error) = value.get("error") { + return Err(HarnessServerError::Protocol(format!( + "Codex app-server turn/interrupt request {id} failed: {error}" + ))); + } + continue; + } + if active_requests.steer_request_ids.remove(&id) { + if let Some(error) = value.get("error") { + eprintln!("Codex app-server turn/steer request {id} failed: {error}"); + } + continue; } continue; } @@ -663,7 +666,79 @@ impl CodexJsonRpcChild { return Ok(TurnTermination::Done); } } + self.forward_pending_active_requests( + active_turn_rx, + &mut active_requests, + request_id, + thread_id, + turn_id, + traceparent, + )?; + } + } + + fn forward_pending_active_requests( + &mut self, + active_turn_rx: &Receiver, + active_requests: &mut CodexActiveRequestState, + request_id: &mut i64, + thread_id: &str, + turn_id: &str, + traceparent: Option<&str>, + ) -> Result<()> { + while let Ok(request) = active_turn_rx.try_recv() { + match request { + CodexActiveTurnRequest::Interrupt => { + if active_requests.interrupt_request_id.is_some() { + eprintln!("Codex blocks interrupt ignored: interrupt already requested"); + continue; + } + let id = next_request_id(request_id); + self.send_request( + id, + "turn/interrupt", + json!({ + "threadId": thread_id, + "turnId": turn_id, + }), + traceparent, + )?; + active_requests.interrupt_request_id = Some(id); + } + CodexActiveTurnRequest::Steer(command) => { + let BlocksCommand::User { + input, + client_user_message_id, + model, + provider, + reasoning, + trace_context, + } = *command + else { + continue; + }; + if model.is_some() || provider.is_some() || reasoning.is_some() { + eprintln!( + "Codex blocks steering ignored turn-start-only overrides: \ + model={model:?} provider={provider:?} reasoning={reasoning:?}" + ); + } + let id = next_request_id(request_id); + let mut params = json!({ + "threadId": thread_id, + "expectedTurnId": turn_id, + "input": input, + }); + if let Some(client_user_message_id) = client_user_message_id { + params["clientUserMessageId"] = Value::String(client_user_message_id); + } + let steer_traceparent = trace_context.effective_traceparent(); + self.send_request(id, "turn/steer", params, steer_traceparent.as_deref())?; + active_requests.steer_request_ids.insert(id); + } + } } + Ok(()) } fn read_value(&mut self) -> Result { @@ -697,7 +772,7 @@ impl CodexJsonRpcChild { if trimmed.is_empty() { continue; } - return serde_json::from_str(trimmed).map(Some).map_err(Into::into); + return Ok(Some(serde_json::from_str(trimmed)?)); } } } @@ -947,14 +1022,14 @@ mod tests { } #[test] - fn derives_child_thread_key_env_from_trace_context() { + fn derives_child_thread_key_env_from_first_turn_trace_context() { let trace_context = otel::TraceContext { - thread_key: Some(" slack:C123:1780000000.000000 ".to_string()), + thread_key: Some(" slack:T123:C123:1780000000.000000 ".to_string()), ..Default::default() }; assert_eq!( centaur_thread_key_env_value(&trace_context), - Some("slack:C123:1780000000.000000") + Some("slack:T123:C123:1780000000.000000") ); let blank_trace_context = otel::TraceContext { diff --git a/crates/harness-server/src/error.rs b/crates/harness-server/src/error.rs index bc1b43be9..b38a53209 100644 --- a/crates/harness-server/src/error.rs +++ b/crates/harness-server/src/error.rs @@ -49,6 +49,8 @@ pub enum HarnessServerError { status: ExitStatus, stderr: String, }, + #[error("{kind:?} turn interrupted")] + TurnInterrupted { kind: HarnessKind }, #[error("failed to spawn {bin} app-server: {source}")] SpawnCodex { bin: String, diff --git a/crates/harness-server/src/main.rs b/crates/harness-server/src/main.rs index bd614d112..9ac6f4218 100644 --- a/crates/harness-server/src/main.rs +++ b/crates/harness-server/src/main.rs @@ -47,7 +47,7 @@ fn main() { fn run() -> Result<()> { match Cli::parse() .command - .unwrap_or(CliCommand::ClaudeCode(HarnessCommand { + .unwrap_or(CliCommand::Codex(HarnessCommand { mode: ServerMode::Blocks, })) { CliCommand::Codex(command) => run_mode(HarnessKind::Codex, command.mode), diff --git a/crates/harness-server/src/server.rs b/crates/harness-server/src/server.rs index d615625fd..c7b4040d1 100644 --- a/crates/harness-server/src/server.rs +++ b/crates/harness-server/src/server.rs @@ -4,8 +4,12 @@ use std::fs::OpenOptions; use std::io::{self, BufRead, Write}; use std::path::{Path, PathBuf}; use std::process::Stdio; -use std::sync::mpsc::{self, Receiver, RecvTimeoutError}; -use std::time::Duration; +use std::sync::{ + Arc, + atomic::{AtomicBool, Ordering}, + mpsc::{self, Receiver, RecvTimeoutError}, +}; +use std::time::{Duration, Instant}; use base64::Engine; use base64::engine::general_purpose::STANDARD as BASE64_STANDARD; @@ -75,21 +79,65 @@ pub fn run_validate_jsonrpc() -> Result<()> { } pub(crate) fn run_blocks_app_server(harness: &H) -> Result<()> { - let stdin = io::stdin(); let mut stdout = io::stdout().lock(); let mut state = initial_blocks_thread_state(harness)?; - let mut blocks_state = BlocksState::default(); - let (_request_tx, request_rx) = mpsc::channel(); + let (command_tx, command_rx) = mpsc::channel(); + let (request_tx, request_rx) = mpsc::channel(); + let turn_active = Arc::new(AtomicBool::new(false)); - for raw in stdin.lock().lines() { - let line = raw?; - let trimmed = line.trim(); - if trimmed.is_empty() { - continue; - } + { + let turn_active = Arc::clone(&turn_active); + std::thread::spawn(move || { + let stdin = io::stdin(); + let mut blocks_state = BlocksState::default(); + for raw in stdin.lock().lines() { + let Ok(line) = raw else { + break; + }; + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } - match parse_blocks_line_with_state(trimmed, &mut blocks_state) { - Ok(BlocksCommand::User { + match parse_blocks_line_with_state(trimmed, &mut blocks_state) { + Ok(BlocksCommand::Interrupt) if turn_active.load(Ordering::SeqCst) => { + if request_tx.send(ActiveTurnRequest::BlocksInterrupt).is_err() { + break; + } + } + Ok(command @ BlocksCommand::User { .. }) => { + turn_active.store(true, Ordering::SeqCst); + if command_tx + .send(BlocksReaderInput::Command(command)) + .is_err() + { + break; + } + } + Ok(command) => { + if command_tx + .send(BlocksReaderInput::Command(command)) + .is_err() + { + break; + } + } + Err(error) => { + if command_tx + .send(BlocksReaderInput::Error(error.to_string())) + .is_err() + { + break; + } + } + } + } + }); + } + + while let Ok(input) = command_rx.recv() { + match input { + BlocksReaderInput::Command(BlocksCommand::User { input, client_user_message_id, model, @@ -103,7 +151,7 @@ pub(crate) fn run_blocks_app_server(harness: &H) -> Result<()> if let Some(model) = model { state.model = model; } - if let Err(error) = run_blocks_turn( + let result = run_blocks_turn( harness, &mut state, input, @@ -111,18 +159,21 @@ pub(crate) fn run_blocks_app_server(harness: &H) -> Result<()> &trace_context, &mut stdout, &request_rx, - ) { + &turn_active, + ); + drain_active_turn_requests(&request_rx); + if let Err(error) = result { eprintln!("blocks turn failed: {error:#}"); write_blocks_error(&mut stdout, &state.id, "turn", error.to_string())?; } } - Ok(BlocksCommand::Interrupt) => { - eprintln!("blocks interrupt ignored: no active stdin reader while a turn runs"); + BlocksReaderInput::Command(BlocksCommand::Interrupt) => { + eprintln!("blocks interrupt ignored: no active turn runs"); } - Ok(BlocksCommand::AttachmentChunk) => {} - Err(error) => { - eprintln!("invalid blocks input: {error:#}"); - write_blocks_error(&mut stdout, &state.id, "input", error.to_string())?; + BlocksReaderInput::Command(BlocksCommand::AttachmentChunk) => {} + BlocksReaderInput::Error(error) => { + eprintln!("invalid blocks input: {error}"); + write_blocks_error(&mut stdout, &state.id, "input", error)?; } } } @@ -152,7 +203,10 @@ pub(crate) fn run_app_server(harness: &H) -> Result<()> { let JSONRPCMessage::Request(request) = message else { continue; }; - if request_tx.send(request).is_err() { + if request_tx + .send(ActiveTurnRequest::JsonRpc(request)) + .is_err() + { break; } } @@ -162,9 +216,17 @@ pub(crate) fn run_app_server(harness: &H) -> Result<()> { let mut threads: HashMap = HashMap::new(); while let Ok(request) = request_rx.recv() { - if let Err(error) = handle_request(harness, request, &request_rx, &mut threads, &mut stdout) - { - eprintln!("request failed: {error:#}"); + match request { + ActiveTurnRequest::JsonRpc(request) => { + if let Err(error) = + handle_request(harness, request, &request_rx, &mut threads, &mut stdout) + { + eprintln!("request failed: {error:#}"); + } + } + ActiveTurnRequest::BlocksInterrupt => { + eprintln!("blocks interrupt ignored: no active turn runs"); + } } } @@ -184,11 +246,13 @@ fn run_blocks_turn( client_user_message_id: Option, trace_context: &TraceContext, stdout: &mut W, - request_rx: &Receiver, + request_rx: &Receiver, + turn_active: &AtomicBool, ) -> Result<()> { let turn_id = format!("turn-{}", Uuid::new_v4().simple()); let mut normalizer = normalizer_for(harness, state, &turn_id); - run_normalized_turn( + turn_active.store(true, Ordering::SeqCst); + let result = run_normalized_turn( harness, state, &input, @@ -197,7 +261,23 @@ fn run_blocks_turn( &mut normalizer, stdout, request_rx, - ) + ); + turn_active.store(false, Ordering::SeqCst); + result +} + +enum BlocksReaderInput { + Command(BlocksCommand), + Error(String), +} + +enum ActiveTurnRequest { + JsonRpc(JSONRPCRequest), + BlocksInterrupt, +} + +fn drain_active_turn_requests(rx: &Receiver) { + while rx.try_recv().is_ok() {} } #[derive(Debug)] @@ -314,9 +394,16 @@ enum BlocksInput { struct AttachmentBlock { #[serde(rename = "type")] kind: String, + #[serde( + rename = "attachment_id", + alias = "attachmentId", + alias = "id", + default + )] + attachment_id: Option, #[serde(default)] name: Option, - #[serde(rename = "mimeType", default)] + #[serde(rename = "mimeType", alias = "mime_type", default)] mime_type: Option, #[serde(rename = "attachment_type", default)] attachment_type: Option, @@ -357,7 +444,7 @@ pub(crate) fn parse_blocks_line_with_state( .and_then(|message| message.content.as_ref()) .or(parsed.content.as_ref()); let mut input = match content { - Some(content) => blocks_content_to_user_input(content, state)?, + Some(content) => blocks_content_to_user_input(content, state, &trace_context)?, None => parsed .text .map(|text| { @@ -408,11 +495,12 @@ pub(crate) fn parse_blocks_line_with_state( fn blocks_content_to_user_input( content: &BlocksContent, state: &mut BlocksState, + trace_context: &TraceContext, ) -> Result> { match content { BlocksContent::Inputs(input) => input .iter() - .map(|item| blocks_input_to_user_input(item, state)) + .map(|item| blocks_input_to_user_input(item, state, trace_context)) .collect::>>() .map(|items| items.into_iter().flatten().collect()), BlocksContent::Text(text) => Ok(vec![UserInput::Text { @@ -425,12 +513,16 @@ fn blocks_content_to_user_input( fn blocks_input_to_user_input( input: &BlocksInput, state: &mut BlocksState, + _trace_context: &TraceContext, ) -> Result> { match input { BlocksInput::UserInput(input) => Ok(vec![input.clone()]), BlocksInput::Attachment(block) if block.kind == "attachment" => { attachment_block_to_user_input(block, state) } + BlocksInput::Attachment(block) if block.kind == "attachment_ref" => { + Ok(attachment_ref_block_to_user_input(block)) + } BlocksInput::Attachment(block) => Ok(vec![UserInput::Text { text: format!("[Unsupported attachment block type: {}]", block.kind), text_elements: Vec::new(), @@ -438,6 +530,33 @@ fn blocks_input_to_user_input( } } +fn attachment_ref_block_to_user_input(block: &AttachmentBlock) -> Vec { + let attachment_id = non_empty(block.attachment_id.as_deref()); + let mime_type = non_empty(block.mime_type.as_deref()); + let name = non_empty(block.name.as_deref()).unwrap_or("attachment"); + + let mut fields = Vec::new(); + if let Some(attachment_id) = attachment_id { + fields.push(format!("id={attachment_id}")); + } + fields.push(format!("name={name}")); + if let Some(mime_type) = mime_type { + fields.push(format!("mime={mime_type}")); + } + + let summary = if fields.is_empty() { + "attachment_ref".to_string() + } else { + format!("attachment_ref {}", fields.join(" ")) + }; + vec![UserInput::Text { + text: format!( + "[Attachment reference: {summary}. The file was not provided to this sandbox. Ask the caller to resend the attachment as an upload, inline file data, or staged attachment chunk before inspecting it.]" + ), + text_elements: Vec::new(), + }] +} + fn attachment_block_to_user_input( block: &AttachmentBlock, state: &mut BlocksState, @@ -668,7 +787,7 @@ fn clean_string(value: Option<&str>) -> Option { fn handle_request( harness: &H, request: JSONRPCRequest, - request_rx: &Receiver, + request_rx: &Receiver, threads: &mut HashMap, stdout: &mut W, ) -> Result<()> { @@ -893,9 +1012,14 @@ fn handle_active_turn_request( harness: &H, process: &mut HarnessChild, normalizer: &mut CodexTurnNormalizer, - request: JSONRPCRequest, + request: ActiveTurnRequest, stdout: &mut W, -) -> Result<()> { +) -> Result { + let ActiveTurnRequest::JsonRpc(request) = request else { + process.kill_and_wait()?; + return Ok(true); + }; + match request.method.as_str() { "turn/steer" => { let params: TurnSteerParams = request_params(request.params)?; @@ -906,7 +1030,7 @@ fn handle_active_turn_request( -32600, format!("unknown threadId {}", params.thread_id), )?; - return Ok(()); + return Ok(false); } if params.expected_turn_id != normalizer.turn_id() { write_error( @@ -919,7 +1043,7 @@ fn handle_active_turn_request( normalizer.turn_id() ), )?; - return Ok(()); + return Ok(false); } process .stdin @@ -939,9 +1063,33 @@ fn handle_active_turn_request( { write_value(stdout, ¬ification_to_wire_value(¬ification)?)?; } - Ok(()) + Ok(false) } "turn/interrupt" => { + let params: TurnInterruptParams = request_params(request.params)?; + if params.thread_id != normalizer.thread_id() { + write_error( + stdout, + request.id, + -32600, + format!("unknown threadId {}", params.thread_id), + )?; + return Ok(false); + } + if params.turn_id != normalizer.turn_id() { + write_error( + stdout, + request.id, + -32600, + format!( + "expected active turn id `{}` but found `{}`", + params.turn_id, + normalizer.turn_id() + ), + )?; + return Ok(false); + } + process.kill_and_wait()?; write_client_response( stdout, ClientResponse::TurnInterrupt { @@ -949,7 +1097,7 @@ fn handle_active_turn_request( response: TurnInterruptResponse {}, }, )?; - Ok(()) + Ok(true) } _ => { write_error( @@ -958,7 +1106,7 @@ fn handle_active_turn_request( -32600, format!("cannot handle {} while a turn is active", request.method), )?; - Ok(()) + Ok(false) } } } @@ -971,7 +1119,7 @@ fn run_normalized_turn( trace_context: Option<&TraceContext>, normalizer: &mut CodexTurnNormalizer, stdout: &mut W, - request_rx: &Receiver, + request_rx: &Receiver, ) -> Result<()> { for notification in normalizer.start_notifications(!state.thread_started_sent)? { if matches!(notification, ServerNotification::ThreadStarted(_)) { @@ -994,7 +1142,28 @@ fn run_normalized_turn( ) { Ok(Some(turn)) => state.completed_turns.push(turn), Ok(None) => {} - Err(error) => finish_turn_with_error(state, normalizer, stdout, error)?, + Err(HarnessServerError::TurnInterrupted { .. }) => { + state.process = None; + finish_turn_interrupted(state, normalizer, stdout)?; + } + Err(error) => { + state.process = None; + finish_turn_with_error(state, normalizer, stdout, error)?; + } + } + Ok(()) +} + +fn finish_turn_interrupted( + state: &mut ThreadState, + normalizer: &mut CodexTurnNormalizer, + stdout: &mut W, +) -> Result<()> { + if let Some(notification) = normalizer.finish_turn_interrupted()? { + if let ServerNotification::TurnCompleted(completed) = ¬ification { + state.completed_turns.push(completed.turn.clone()); + } + write_value(stdout, ¬ification_to_wire_value(¬ification)?)?; } Ok(()) } @@ -1028,7 +1197,7 @@ fn run_harness_turn( trace_context: Option<&TraceContext>, normalizer: &mut CodexTurnNormalizer, stdout: &mut W, - request_rx: &Receiver, + request_rx: &Receiver, ) -> Result> { let usage_span_start = otel::unix_time_nanos(); let usage_span_model = state.model.clone(); @@ -1037,56 +1206,117 @@ fn run_harness_turn( let usage_span_input = usage_span_input_value(input); let mut usage_span_output = UsageSpanOutput::default(); ensure_harness_process(harness, state)?; - let process = state - .process - .as_mut() - .ok_or(HarnessServerError::HarnessStdinUnavailable)?; - process.stdin.write_all(&harness.stdin_for_turn(input)?)?; - process.stdin.flush()?; + { + let process = state + .process + .as_mut() + .ok_or(HarnessServerError::HarnessStdinUnavailable)?; + // Anything already buffered predates this turn's input: a previous + // turn completed via the terminal-stop fallback can leave the CLI's + // late `result` (and trailing rate-limit noise) behind, which would + // otherwise read as this turn's instant terminal. + while process.stdout.try_recv().is_ok() {} + process.stdin.write_all(&harness.stdin_for_turn(input)?)?; + process.stdin.flush()?; + } + let settle_window = harness.terminal_assistant_stop_settle(); + // Armed after a terminal assistant stop with no native terminal event yet: + // once the stream stays quiet past this deadline the turn completes via + // the fallback. Any further output (the native result on its way, trailing + // noise, or a continuation of the turn) pushes the deadline back. + let mut settle_deadline: Option = None; let mut last_session_id = state.harness_session_id.clone(); let mut event_normalizer = H::EventNormalizer::default(); let mut completed_turn = None; let mut latest_usage = None; loop { while let Ok(request) = request_rx.try_recv() { - handle_active_turn_request(harness, process, normalizer, request, stdout)?; - } - - let line = match process.stdout.recv_timeout(Duration::from_millis(50)) { - Ok(line) => line?, - Err(RecvTimeoutError::Timeout) => continue, - Err(RecvTimeoutError::Disconnected) => { - let status = process.child.wait()?; - return Err(HarnessServerError::HarnessExited { + let process = state + .process + .as_mut() + .ok_or(HarnessServerError::HarnessStdinUnavailable)?; + if handle_active_turn_request(harness, process, normalizer, request, stdout)? { + state.process = None; + return Err(HarnessServerError::TurnInterrupted { kind: harness.kind(), - status, - stderr: String::new(), }); } - }; - let trimmed = line.trim(); - if trimmed.is_empty() { - continue; + // A steer re-opens the turn: the harness now owes a response whose + // first token can take longer than the settle window, so the + // pending fallback completion no longer applies. The response's + // own terminal stop re-arms it. + settle_deadline = None; } - let event = harness.parse_stdout_line(trimmed)?; - let normalized_events = harness.normalize_events(&mut event_normalizer, event)?; + let mut terminal = false; - for normalized in normalized_events { - if let Some(usage) = normalized.token_usage() { - latest_usage = Some(usage.clone()); - } - append_usage_span_output(&normalized, &mut usage_span_output); - if let Some(session_id) = normalized.session_id() { - last_session_id = Some(session_id.to_string()); - state.harness_session_id = Some(session_id.to_string()); + match state + .process + .as_mut() + .ok_or(HarnessServerError::HarnessStdoutUnavailable)? + .stdout + .recv_timeout(Duration::from_millis(50)) + { + Ok(line) => { + let line = line?; + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + let event = harness.parse_stdout_line(trimmed)?; + let normalized_events = harness.normalize_events(&mut event_normalizer, event)?; + let mut terminal_stop = false; + for normalized in normalized_events { + if let Some(usage) = normalized.token_usage() { + latest_usage = Some(usage.clone()); + } + append_usage_span_output(&normalized, &mut usage_span_output); + if let Some(session_id) = normalized.session_id() { + last_session_id = Some(session_id.to_string()); + state.harness_session_id = Some(session_id.to_string()); + } + for notification in normalizer.process_event(&normalized)? { + write_value(stdout, ¬ification_to_wire_value(¬ification)?)?; + } + terminal |= normalized.is_terminal(); + terminal_stop |= + settle_window.is_some() && normalized.is_terminal_assistant_stop(); + } + if !terminal { + match settle_window { + Some(window) if terminal_stop && window.is_zero() => terminal = true, + Some(window) if terminal_stop || settle_deadline.is_some() => { + settle_deadline = Some(Instant::now() + window); + } + _ => {} + } + } } - for notification in normalizer.process_event(&normalized)? { - write_value(stdout, ¬ification_to_wire_value(¬ification)?)?; + Err(RecvTimeoutError::Timeout) => match settle_deadline { + Some(deadline) if Instant::now() >= deadline => terminal = true, + _ => continue, + }, + Err(RecvTimeoutError::Disconnected) => { + let status = state + .process + .as_mut() + .ok_or(HarnessServerError::HarnessStdoutUnavailable)? + .child + .wait()?; + // A clean exit while waiting out the settle window means the + // native result is never coming: the terminal stop already + // seen ends the turn. + if settle_deadline.is_some() && status.success() { + state.process = None; + terminal = true; + } else { + return Err(HarnessServerError::HarnessExited { + kind: harness.kind(), + status, + stderr: String::new(), + }); + } } - terminal |= normalized.is_terminal() - || (harness.finish_turn_on_assistant_end_turn() - && normalized.is_assistant_end_turn()); } if terminal { export_harness_usage_if_available( @@ -1242,8 +1472,11 @@ fn append_usage_span_output(event: &NormalizedEvent, output: &mut UsageSpanOutpu } fn ensure_harness_process(harness: &H, state: &mut ThreadState) -> Result<()> { - if state.process.is_some() { - return Ok(()); + if let Some(process) = state.process.as_mut() { + if process.child.try_wait()?.is_none() { + return Ok(()); + } + state.process = None; } let mut command = harness.command_for_turn(state); @@ -1271,7 +1504,12 @@ fn ensure_harness_process(harness: &H, state: &mut ThreadState .take() .ok_or(HarnessServerError::HarnessStderrUnavailable)?; std::thread::spawn(move || { - let mut parent_stderr = io::stderr().lock(); + // Copy through the unlocked handle (it locks per write): the harness + // process outlives each turn, so its stderr never EOFs, and holding + // the StderrLock here for the copy's lifetime deadlocks every other + // eprintln! in the server — including turn-completion paths, which + // then never emit turn/completed. + let mut parent_stderr = io::stderr(); let _ = io::copy(&mut stderr, &mut parent_stderr); }); let (stdout_tx, stdout_rx) = mpsc::channel(); @@ -1403,6 +1641,25 @@ mod tests { assert_eq!(model, None); } + #[test] + fn parses_attachment_ref_as_recoverable_reference() { + let line = r#"{"type":"user","message":{"role":"user","content":[{"type":"text","text":"inspect this"},{"type":"attachment_ref","attachment_id":"att_123","name":"report.pdf","mime_type":"application/pdf"}]}}"#; + let BlocksCommand::User { input, .. } = parse_blocks_line(line).expect("parses") else { + panic!("expected user command"); + }; + + assert_eq!(input.len(), 2); + let UserInput::Text { text, .. } = &input[1] else { + panic!("expected attachment_ref to become text guidance"); + }; + assert!(text.contains("Attachment reference")); + assert!(text.contains("id=att_123")); + assert!(text.contains("name=report.pdf")); + assert!(text.contains("mime=application/pdf")); + assert!(text.contains("not provided to this sandbox")); + assert!(!text.contains("Unsupported attachment block type")); + } + #[test] fn parses_blocks_user_line_with_provider_override() { let line = r#"{"type":"user","thread_key":"web:t1","provider":"amazon-bedrock","message":{"role":"user","content":[{"type":"text","text":"hi"}]}}"#; @@ -1546,4 +1803,32 @@ mod tests { assert!(text.starts_with("[Attached file saved to ")); assert!(text.ends_with("notes.txt]")); } + + #[test] + fn inline_image_attachment_block_becomes_local_image_input() { + let _upload_dir = temp_upload_dir(); + let mut state = BlocksState::default(); + let user = r#"{"type":"user","message":{"role":"user","content":[{"type":"attachment","attachment_type":"image","dataBase64":"aGVsbG8=","name":"image.png","mimeType":"image/png","size":5}]}}"#; + let BlocksCommand::User { input, .. } = + parse_blocks_line_with_state(user, &mut state).expect("user parses") + else { + panic!("expected user command"); + }; + + assert_eq!(input.len(), 2); + let UserInput::Text { text, .. } = &input[0] else { + panic!("expected image attachment notice"); + }; + assert!(text.starts_with("[Attached image saved to ")); + assert!(text.ends_with("image.png]")); + + let UserInput::LocalImage { path, .. } = &input[1] else { + panic!("expected inline image attachment to become a local image"); + }; + assert_eq!( + path.file_name().and_then(|name| name.to_str()), + Some("image.png") + ); + assert_eq!(std::fs::read(path).expect("read image bytes"), b"hello"); + } } diff --git a/crates/harness-server/src/traits.rs b/crates/harness-server/src/traits.rs index f7c736de9..9779df361 100644 --- a/crates/harness-server/src/traits.rs +++ b/crates/harness-server/src/traits.rs @@ -2,6 +2,7 @@ use std::io; use std::path::PathBuf; use std::process::{Child, ChildStdin, Command as ProcessCommand}; use std::sync::mpsc::Receiver; +use std::time::Duration; use codex_app_server_protocol::{ThreadStartParams, Turn, UserInput}; use serde_json::Value; @@ -41,6 +42,13 @@ impl Drop for HarnessChild { } } +impl HarnessChild { + pub fn kill_and_wait(&mut self) -> io::Result<()> { + let _ = self.child.kill(); + self.child.wait().map(|_| ()) + } +} + pub trait AppServerRuntime { fn run_stdio(&self) -> Result<()>; } @@ -64,8 +72,17 @@ pub trait HarnessServer { normalizer: &mut Self::EventNormalizer, event: Self::Event, ) -> Result>; - fn finish_turn_on_assistant_end_turn(&self) -> bool { - false + /// How to treat an assistant message that stops with a terminal stop + /// reason (`end_turn`, ...) when no native terminal event has arrived. + /// `None` keeps the turn open until a native result/error (the default). + /// `Some(window)` completes the turn once the stream stays quiet for + /// `window` after the stop: a zero window completes immediately (for + /// streams with no native result event), a nonzero window gives the + /// harness's own `result` a chance to settle the turn first — and keeps + /// that trailing `result` from being read as the *next* turn's terminal — + /// while still completing when the result never comes. + fn terminal_assistant_stop_settle(&self) -> Option { + None } fn thread_state(&self, params: &ThreadStartParams, cwd: PathBuf) -> ThreadState { @@ -146,18 +163,25 @@ impl NormalizedEvent { } } - pub(crate) fn is_assistant_end_turn(&self) -> bool { + pub(crate) fn is_terminal_assistant_stop(&self) -> bool { matches!( self, Self::AssistantMessage { partial: false, stop_reason: Some(stop_reason), .. - } if stop_reason == "end_turn" + } if is_terminal_assistant_stop_reason(stop_reason) ) } } +fn is_terminal_assistant_stop_reason(reason: &str) -> bool { + matches!( + reason, + "end_turn" | "stop_sequence" | "max_tokens" | "refusal" + ) +} + #[derive(Debug, Clone)] pub enum NormalizedContent { AgentText { diff --git a/crates/harness-server/src/turn.rs b/crates/harness-server/src/turn.rs index d5463934b..75f37b30f 100644 --- a/crates/harness-server/src/turn.rs +++ b/crates/harness-server/src/turn.rs @@ -237,13 +237,28 @@ impl CodexTurnNormalizer { if self.completed { return Ok(None); } - self.completed = true; let error = failed.or_else(|| self.last_error.clone()); let status = if error.is_some() { TurnStatus::Failed } else { TurnStatus::Completed }; + self.finish_turn_with_status(status, error) + } + + pub fn finish_turn_interrupted(&mut self) -> Result> { + self.finish_turn_with_status(TurnStatus::Interrupted, None) + } + + fn finish_turn_with_status( + &mut self, + status: TurnStatus, + error: Option, + ) -> Result> { + if self.completed { + return Ok(None); + } + self.completed = true; let completed_at = now_secs(); Ok(Some(ServerNotification::TurnCompleted( TurnCompletedNotification { diff --git a/crates/harness-server/tests/app_server_stdio.rs b/crates/harness-server/tests/app_server_stdio.rs index c0b049087..7ebc50a40 100644 --- a/crates/harness-server/tests/app_server_stdio.rs +++ b/crates/harness-server/tests/app_server_stdio.rs @@ -106,6 +106,148 @@ fn fake_claude_app_server_streams_codex_v2_notifications() { assert_codex_v2_turn(&run.turn); } +#[test] +fn fake_claude_app_server_completes_on_stop_sequence_without_result() { + let fake_claude = concat!( + "printf '%s\\n' ", + "'{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"claude-session\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[]}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"fable answer\"}}}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"fable answer\"}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"stop_sequence\"}}}'" + ); + + let run = run_bridge_turn(BridgeTurnConfig { + harness: Harness::ClaudeCode, + command_override: Some(fake_claude.to_string()), + prompt: "say hello".to_string(), + timeout: Duration::from_secs(10), + }); + + assert_completed_turn(&run.turn); + assert_eq!(run.turn.text_from_deltas, "fable answer"); + assert_codex_v2_turn(&run.turn); +} + +#[test] +fn fake_claude_completes_on_terminal_stop_while_process_outlives_the_turn() { + // The real CLI does not exit after a turn — harness-server keeps it (and + // its stdin) alive for the next one. When the stream stops at the + // message_delta stop reason with no trailing `result`, the settle window + // must complete the turn instead of waiting on the live process forever + // (the fable "thinking..." hang). + let fake_claude = concat!( + "printf '%s\\n' ", + "'{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"claude-session\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[]}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"fable answer\"}}}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"fable answer\"}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"}}}'", + "; sleep 60" + ); + + let run = run_bridge_turn(BridgeTurnConfig { + harness: Harness::ClaudeCode, + command_override: Some(fake_claude.to_string()), + prompt: "say hello".to_string(), + timeout: Duration::from_secs(10), + }); + + assert_completed_turn(&run.turn); + assert_eq!(run.turn.text_from_deltas, "fable answer"); + assert_codex_v2_turn(&run.turn); +} + +#[test] +fn fake_claude_trailing_result_settles_the_turn_and_does_not_poison_the_next() { + // The native `result` trails the message_delta stop reason in real CLI + // output. The stop must not complete the turn so eagerly that the result + // is left buffered, where the next turn would read it as its own instant + // terminal and complete with no content. + let fake_claude = concat!( + "printf '%s\\n' '{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"claude-session\"}'; ", + "IFS= read -r _; ", + "printf '%s\\n' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[]}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"first answer\"}}}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"first answer\"}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_stop\"}}' ", + "'{\"type\":\"result\",\"subtype\":\"success\",\"result\":\"first answer\"}'; ", + "IFS= read -r _; ", + "printf '%s\\n' ", + "'{\"type\":\"assistant\",\"is_partial\":false,\"message\":{\"id\":\"msg_2\",\"content\":[{\"type\":\"text\",\"text\":\"second answer\"}]}}' ", + "'{\"type\":\"result\",\"subtype\":\"success\",\"result\":\"second answer\"}'; ", + "sleep 60" + ); + + let run = run_bridge_two_turns(BridgeTwoTurnConfig { + harness: Harness::ClaudeCode, + command_override: Some(fake_claude.to_string()), + first_prompt: "first".to_string(), + second_prompt: "second".to_string(), + timeout: Duration::from_secs(10), + }); + + assert_completed_turn(&run.turns[0]); + assert_eq!(run.turns[0].text_from_deltas, "first answer"); + assert_completed_turn(&run.turns[1]); + assert_eq!(run.turns[1].text_from_deltas, "second answer"); +} + +#[test] +fn fake_claude_subagent_sidechain_stop_does_not_complete_the_turn() { + // A Task subagent's sidechain messages end with their own end_turn while + // the parent turn keeps running (here: 3s of quiet before the main chain + // resumes, longer than the settle window). The sidechain stop must not + // complete the turn or leak subagent text into it. + let fake_claude = concat!( + "printf '%s\\n' ", + "'{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"claude-session\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[]}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"Delegating.\"}}}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_1\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"Delegating.\"},{\"type\":\"tool_use\",\"id\":\"toolu_task\",\"name\":\"Task\",\"input\":{\"prompt\":\"look it up\"}}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"tool_use\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_sub\",\"stop_reason\":null,\"content\":[]}},\"parent_tool_use_id\":\"toolu_task\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}},\"parent_tool_use_id\":\"toolu_task\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"sub answer\"}},\"parent_tool_use_id\":\"toolu_task\"}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_sub\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"sub answer\"}]},\"parent_tool_use_id\":\"toolu_task\"}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"}},\"parent_tool_use_id\":\"toolu_task\"}'", + "; sleep 3; printf '%s\\n' ", + "'{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"tool_result\",\"tool_use_id\":\"toolu_task\",\"content\":\"sub answer\",\"is_error\":false}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_start\",\"message\":{\"id\":\"msg_2\",\"stop_reason\":null,\"content\":[]}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_start\",\"index\":0,\"content_block\":{\"type\":\"text\",\"text\":\"\"}}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"content_block_delta\",\"index\":0,\"delta\":{\"type\":\"text_delta\",\"text\":\"main answer\"}}}' ", + "'{\"type\":\"assistant\",\"message\":{\"id\":\"msg_2\",\"stop_reason\":null,\"content\":[{\"type\":\"text\",\"text\":\"main answer\"}]}}' ", + "'{\"type\":\"stream_event\",\"event\":{\"type\":\"message_delta\",\"delta\":{\"stop_reason\":\"end_turn\"}}}' ", + "'{\"type\":\"result\",\"subtype\":\"success\",\"result\":\"main answer\"}'" + ); + + let run = run_bridge_turn(BridgeTurnConfig { + harness: Harness::ClaudeCode, + command_override: Some(fake_claude.to_string()), + prompt: "delegate then answer".to_string(), + timeout: Duration::from_secs(15), + }); + + assert_completed_turn(&run.turn); + assert!( + run.turn.text_from_deltas.contains("main answer"), + "main-chain answer missing: {:?}", + run.turn.text_from_deltas + ); + assert!( + !run.turn.text_from_deltas.contains("sub answer"), + "sidechain text leaked into the turn: {:?}", + run.turn.text_from_deltas + ); + assert_codex_v2_turn(&run.turn); +} + #[test] fn fake_codex_blocks_mode_uses_openrouter_provider_when_model_is_configured() { let fake_codex = temp_path("fake-openrouter-codex.sh"); @@ -219,6 +361,73 @@ fn fake_codex_blocks_mode_uses_openrouter_provider_for_explicit_model_slug() { let _ = std::fs::remove_file(fake_codex_log); } +#[test] +fn fake_codex_blocks_mode_uses_meta_responses_provider_when_selected() { + let fake_codex = temp_path("fake-meta-codex.sh"); + let fake_codex_log = temp_path("fake-meta-codex-requests.jsonl"); + let script = fake_codex_app_server_script(&fake_codex_log); + std::fs::write(&fake_codex, script).expect("write fake codex script"); + let mut permissions = std::fs::metadata(&fake_codex) + .expect("fake codex metadata") + .permissions(); + permissions.set_mode(0o755); + std::fs::set_permissions(&fake_codex, permissions).expect("chmod fake codex script"); + + let mut bridge = BridgeProcess::spawn_harness_blocks( + Harness::Codex, + None, + Some(( + "CODEX_BIN", + fake_codex.to_str().expect("utf-8 fake codex path"), + )), + ); + // `--meta` reaches the harness as an explicit `responses` provider. It must + // win even when the selected model contains `/`, which would otherwise + // trigger the OpenRouter model-slug heuristic. + let user_line = json!({ + "type": "user", + "thread_key": "slack:C123:123.456", + "provider": "responses", + "model": "meta-llama/llama-4-maverick", + "message": { + "role": "user", + "content": [{"type": "text", "text": "say meta blocks"}], + }, + }); + let turn = bridge.run_blocks_user_line(user_line, Duration::from_secs(10)); + bridge.finish_successfully(); + + assert_completed_turn(&turn); + assert_codex_v2_turn(&turn); + + let requests = std::fs::read_to_string(&fake_codex_log).expect("read fake codex request log"); + let requests: Vec = requests + .lines() + .map(|line| serde_json::from_str(line).expect("fake codex request JSON")) + .collect(); + let thread_start = requests + .iter() + .find(|value| value.get("method").and_then(Value::as_str) == Some("thread/start")) + .unwrap_or_else(|| panic!("blocks mode did not send thread/start; requests={requests:?}")); + assert_eq!( + thread_start + .pointer("/params/modelProvider") + .and_then(Value::as_str), + Some("responses") + ); + let turn_start = requests + .iter() + .find(|value| value.get("method").and_then(Value::as_str) == Some("turn/start")) + .unwrap_or_else(|| panic!("blocks mode did not send turn/start; requests={requests:?}")); + assert_eq!( + turn_start.pointer("/params/model").and_then(Value::as_str), + Some("meta-llama/llama-4-maverick") + ); + + let _ = std::fs::remove_file(fake_codex); + let _ = std::fs::remove_file(fake_codex_log); +} + #[test] fn fake_codex_blocks_mode_uses_bedrock_provider_when_selected() { let fake_codex = temp_path("fake-bedrock-codex.sh"); @@ -361,6 +570,35 @@ fn fake_amp_blocks_mode_accepts_user_blocks_by_default() { assert_codex_v2_turn(&run.turn); } +#[test] +fn fake_claude_blocks_mode_interrupts_back_to_back_stop() { + let fake_claude = concat!( + "printf '%s\\n' ", + "'{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"claude-session\"}'; ", + "while IFS= read -r _; do sleep 60; done" + ); + + let mut bridge = BridgeProcess::spawn_harness_blocks( + Harness::ClaudeCode, + Some(fake_claude.to_string()), + None, + ); + let turn = bridge.run_blocks_interrupted_turn("hang until stopped", Duration::from_secs(10)); + bridge.finish_successfully(); + + assert_eq!(turn.terminal_status.as_deref(), Some("interrupted")); + assert!( + turn.methods.contains(&"turn/started".to_string()), + "missing turn/started; got {:?}", + turn.methods + ); + assert!( + turn.methods.contains(&"turn/completed".to_string()), + "missing turn/completed; got {:?}", + turn.methods + ); +} + #[test] fn fake_codex_blocks_mode_spawns_app_server_and_translates_user_blocks() { let fake_codex = temp_path("fake-codex.sh"); @@ -518,6 +756,73 @@ fn fake_codex_blocks_mode_steers_active_turn_from_second_user_line() { let _ = std::fs::remove_file(fake_codex_log); } +#[test] +fn fake_codex_blocks_mode_interrupts_active_turn() { + let fake_codex = temp_path("fake-interruptible-codex.sh"); + let fake_codex_log = temp_path("fake-interruptible-codex-requests.jsonl"); + let script = fake_codex_interruptible_app_server_script(&fake_codex_log); + std::fs::write(&fake_codex, script).expect("write fake codex script"); + let mut permissions = std::fs::metadata(&fake_codex) + .expect("fake codex metadata") + .permissions(); + permissions.set_mode(0o755); + std::fs::set_permissions(&fake_codex, permissions).expect("chmod fake codex script"); + + let mut bridge = BridgeProcess::spawn_harness_blocks( + Harness::Codex, + None, + Some(( + "CODEX_BIN", + fake_codex.to_str().expect("utf-8 fake codex path"), + )), + ); + let turn = bridge.run_blocks_interrupted_turn("hang until stopped", Duration::from_secs(10)); + let stdout_lines = bridge.finish_successfully(); + + assert_eq!(turn.terminal_status.as_deref(), Some("interrupted")); + assert!( + turn.methods.contains(&"turn/started".to_string()), + "missing turn/started; got {:?}", + turn.methods + ); + assert!( + turn.methods.contains(&"turn/completed".to_string()), + "missing turn/completed; got {:?}", + turn.methods + ); + assert!( + stdout_lines + .iter() + .all(|line| response_id(&serde_json::from_str(line).expect("JSON stdout")).is_none()), + "blocks mode should emit notifications only, not JSON-RPC responses" + ); + + let requests = std::fs::read_to_string(&fake_codex_log).expect("read fake codex request log"); + let requests: Vec = requests + .lines() + .map(|line| serde_json::from_str(line).expect("fake codex request JSON")) + .collect(); + let interrupt = requests + .iter() + .find(|value| value.get("method").and_then(Value::as_str) == Some("turn/interrupt")) + .unwrap_or_else(|| { + panic!("blocks mode did not send turn/interrupt; requests={requests:?}") + }); + assert_eq!( + interrupt + .pointer("/params/threadId") + .and_then(Value::as_str), + Some("thread-1") + ); + assert_eq!( + interrupt.pointer("/params/turnId").and_then(Value::as_str), + Some("turn-1") + ); + + let _ = std::fs::remove_file(fake_codex); + let _ = std::fs::remove_file(fake_codex_log); +} + #[test] fn fake_codex_blocks_mode_forwards_traceparent_to_app_server_requests() { let fake_codex = temp_path("fake-codex-trace.sh"); @@ -601,7 +906,7 @@ fn fake_codex_blocks_mode_forwards_reasoning_as_turn_start_effort() { json!({ "type": "user", "thread_key": "slack:C123:123.456", - "reasoning": "high", + "reasoning": "max", "message": { "role": "user", "content": [{"type": "text", "text": "say codex blocks"}], @@ -620,7 +925,7 @@ fn fake_codex_blocks_mode_forwards_reasoning_as_turn_start_effort() { .expect("blocks mode did not send turn/start"); assert_eq!( turn_start.pointer("/params/effort").and_then(Value::as_str), - Some("high"), + Some("max"), "reasoning should be forwarded as turn/start effort; turn_start={turn_start}" ); @@ -709,6 +1014,89 @@ fn fake_harness_process_is_started_once_across_two_turns() { let _ = std::fs::remove_file(start_log); } +#[test] +fn turn_interrupt_kills_harness_process_and_finishes_turn() { + let start_log = temp_path("harness-interrupt-starts.log"); + let marker = temp_path("harness-interrupt-marker"); + let command = format!( + "printf 'start\\n' >> {start_log}; \ + trap 'printf \"killed\\n\" >> {start_log}; exit 143' TERM INT; \ + printf '%s\\n' '{{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"fake-session\"}}'; \ + while IFS= read -r _; do \ + if [ -f {marker} ]; then \ + printf '%s\\n' '{{\"type\":\"assistant\",\"is_partial\":false,\"message\":{{\"id\":\"msg_1\",\"content\":[{{\"type\":\"text\",\"text\":\"fresh turn\"}}]}}}}'; \ + printf '%s\\n' '{{\"type\":\"result\",\"subtype\":\"success\",\"result\":\"fresh turn\"}}'; \ + else \ + sleep 60; \ + fi; \ + done", + start_log = shell_quote(start_log.as_path()), + marker = shell_quote(marker.as_path()) + ); + let mut bridge = BridgeProcess::spawn_harness(Harness::ClaudeCode, Some(command), None); + let thread_id = + bridge.initialize_and_start_thread(Harness::ClaudeCode, Duration::from_secs(10)); + + let interrupted = bridge.run_interrupted_turn( + &thread_id, + 3, + 4, + "hang until stopped", + Duration::from_secs(10), + ); + assert_eq!(interrupted.terminal_status.as_deref(), Some("interrupted")); + + std::fs::write(&marker, b"fresh turn ready").expect("write fresh-turn marker"); + let fresh = bridge.run_turn( + &thread_id, + 5, + "run after interrupt", + None, + Duration::from_secs(10), + ); + assert_completed_turn(&fresh); + assert_eq!(fresh.text_from_deltas, "fresh turn"); + let _ = bridge.child.kill(); + let _ = bridge.child.wait(); + let _ = std::fs::remove_file(start_log); + let _ = std::fs::remove_file(marker); +} + +#[test] +fn turn_interrupt_rejects_wrong_thread_and_turn_without_killing_process() { + let start_log = temp_path("harness-rejected-interrupt-starts.log"); + let command = format!( + "printf 'start\\n' >> {start_log}; \ + printf '%s\\n' '{{\"type\":\"system\",\"subtype\":\"init\",\"session_id\":\"fake-session\"}}'; \ + while IFS= read -r _; do sleep 60; done", + start_log = shell_quote(start_log.as_path()), + ); + let mut bridge = BridgeProcess::spawn_harness(Harness::ClaudeCode, Some(command), None); + let thread_id = + bridge.initialize_and_start_thread(Harness::ClaudeCode, Duration::from_secs(10)); + + let interrupted = bridge.run_turn_with_rejected_interrupts( + &thread_id, + 3, + 4, + 5, + 6, + "hang until stopped", + Duration::from_secs(10), + ); + assert_eq!(interrupted.terminal_status.as_deref(), Some("interrupted")); + + let starts = std::fs::read_to_string(&start_log).expect("read start log"); + assert_eq!( + starts.lines().count(), + 1, + "rejected interrupts must not kill and restart the harness before the valid interrupt" + ); + let _ = bridge.child.kill(); + let _ = bridge.child.wait(); + let _ = std::fs::remove_file(start_log); +} + #[test] #[ignore = "runs real Claude Code and Codex/Amp-style networked binaries"] fn real_claude_code_long_streaming_is_anchored_to_native_cli() { @@ -1226,6 +1614,198 @@ impl BridgeProcess { capture } + fn run_interrupted_turn( + &mut self, + thread_id: &str, + request_id: i64, + interrupt_request_id: i64, + prompt: &str, + timeout: Duration, + ) -> TurnCapture { + self.send(json!({ + "id": request_id, + "method": "turn/start", + "params": { + "threadId": thread_id, + "input": [{"type": "text", "text": prompt, "text_elements": []}], + }, + })); + + let deadline = Instant::now() + timeout; + let mut capture = TurnCapture::default(); + let mut interrupt_sent = false; + let mut interrupt_acknowledged = false; + + loop { + let value = self.read_json(deadline); + if let Some(id) = response_id(&value) { + if id == request_id { + capture.turn_id = value + .pointer("/result/turn/id") + .and_then(Value::as_str) + .unwrap_or_else(|| panic!("turn/start did not return turn id: {value}")) + .to_string(); + } else if id == interrupt_request_id { + interrupt_acknowledged = true; + } + continue; + } + + if let Some(method) = value.get("method").and_then(Value::as_str) { + assert_notification_thread_id(&value, thread_id); + capture.consume_notification(method, &value); + if method == "turn/started" + && capture.turn_id.is_empty() + && let Some(turn_id) = value.pointer("/params/turn/id").and_then(Value::as_str) + { + capture.turn_id = turn_id.to_string(); + } + if method == "turn/started" && !interrupt_sent { + self.send(json!({ + "id": interrupt_request_id, + "method": "turn/interrupt", + "params": { + "threadId": thread_id, + "turnId": capture.turn_id, + }, + })); + interrupt_sent = true; + } + if method == "turn/completed" { + assert!( + interrupt_acknowledged, + "turn completed before interrupt response" + ); + break; + } + } + } + + capture + } + + fn run_turn_with_rejected_interrupts( + &mut self, + thread_id: &str, + request_id: i64, + wrong_thread_interrupt_request_id: i64, + wrong_turn_interrupt_request_id: i64, + valid_interrupt_request_id: i64, + prompt: &str, + timeout: Duration, + ) -> TurnCapture { + self.send(json!({ + "id": request_id, + "method": "turn/start", + "params": { + "threadId": thread_id, + "input": [{"type": "text", "text": prompt, "text_elements": []}], + }, + })); + + let deadline = Instant::now() + timeout; + let mut capture = TurnCapture::default(); + let mut wrong_thread_interrupt_sent = false; + let mut wrong_thread_interrupt_rejected = false; + let mut wrong_turn_interrupt_sent = false; + let mut wrong_turn_interrupt_rejected = false; + let mut valid_interrupt_sent = false; + let mut valid_interrupt_acknowledged = false; + + loop { + let value = self.read_json_allowing_error(deadline); + if let Some(id) = response_id(&value) { + if id == request_id { + capture.turn_id = value + .pointer("/result/turn/id") + .and_then(Value::as_str) + .unwrap_or_else(|| panic!("turn/start did not return turn id: {value}")) + .to_string(); + } else if id == wrong_thread_interrupt_request_id { + assert!( + value.get("error").is_some(), + "wrong-thread interrupt should be rejected: {value}" + ); + wrong_thread_interrupt_rejected = true; + self.send(json!({ + "id": wrong_turn_interrupt_request_id, + "method": "turn/interrupt", + "params": { + "threadId": thread_id, + "turnId": "wrong-turn", + }, + })); + wrong_turn_interrupt_sent = true; + } else if id == wrong_turn_interrupt_request_id { + assert!( + value.get("error").is_some(), + "wrong-turn interrupt should be rejected: {value}" + ); + wrong_turn_interrupt_rejected = true; + self.send(json!({ + "id": valid_interrupt_request_id, + "method": "turn/interrupt", + "params": { + "threadId": thread_id, + "turnId": capture.turn_id, + }, + })); + valid_interrupt_sent = true; + } else if id == valid_interrupt_request_id { + assert!( + value.get("error").is_none(), + "valid interrupt should be acknowledged: {value}" + ); + valid_interrupt_acknowledged = true; + } + continue; + } + + if let Some(method) = value.get("method").and_then(Value::as_str) { + assert_notification_thread_id(&value, thread_id); + capture.consume_notification(method, &value); + if method == "turn/started" + && capture.turn_id.is_empty() + && let Some(turn_id) = value.pointer("/params/turn/id").and_then(Value::as_str) + { + capture.turn_id = turn_id.to_string(); + } + if method == "turn/started" + && !wrong_thread_interrupt_sent + && !capture.turn_id.is_empty() + { + self.send(json!({ + "id": wrong_thread_interrupt_request_id, + "method": "turn/interrupt", + "params": { + "threadId": "wrong-thread", + "turnId": capture.turn_id, + }, + })); + wrong_thread_interrupt_sent = true; + } + if method == "turn/completed" { + assert!( + wrong_thread_interrupt_rejected, + "turn completed before wrong-thread interrupt rejection" + ); + assert!( + wrong_turn_interrupt_sent && wrong_turn_interrupt_rejected, + "turn completed before wrong-turn interrupt rejection" + ); + assert!(valid_interrupt_sent, "valid interrupt was never sent"); + assert!( + valid_interrupt_acknowledged, + "turn completed before valid interrupt response" + ); + break; + } + } + } + + capture + } + fn run_blocks_user_turn(&mut self, prompt: &str, timeout: Duration) -> TurnCapture { self.run_blocks_user_turn_with_model(prompt, None, timeout) } @@ -1326,6 +1906,54 @@ impl BridgeProcess { capture } + fn run_blocks_interrupted_turn(&mut self, prompt: &str, timeout: Duration) -> TurnCapture { + self.send(json!({ + "type": "user", + "thread_key": "slack:C123:123.456", + "trace_metadata": { + "source": "slackbotv2", + "action": "execute" + }, + "message": { + "role": "user", + "content": [{"type": "text", "text": prompt}], + }, + })); + self.send(json!({ + "type": "interrupt", + "thread_key": "slack:C123:123.456", + "trace_metadata": { + "source": "test", + "action": "interrupt_active_execution" + } + })); + + let deadline = Instant::now() + timeout; + let mut capture = TurnCapture::default(); + + loop { + let value = self.read_json(deadline); + assert!( + response_id(&value).is_none(), + "blocks mode emitted JSON-RPC response: {value}" + ); + if let Some(method) = value.get("method").and_then(Value::as_str) { + capture.consume_notification(method, &value); + if method == "turn/started" + && capture.turn_id.is_empty() + && let Some(turn_id) = value.pointer("/params/turn/id").and_then(Value::as_str) + { + capture.turn_id = turn_id.to_string(); + } + if method == "turn/completed" { + break; + } + } + } + + capture + } + fn send(&mut self, value: Value) { eprintln!("stdin JSON: {value}"); let stdin = self.stdin.as_mut().expect("stdin still open"); @@ -1335,6 +1963,14 @@ impl BridgeProcess { } fn read_json(&mut self, deadline: Instant) -> Value { + self.read_json_checked(deadline, false) + } + + fn read_json_allowing_error(&mut self, deadline: Instant) -> Value { + self.read_json_checked(deadline, true) + } + + fn read_json_checked(&mut self, deadline: Instant, allow_error: bool) -> Value { loop { let now = Instant::now(); assert!(now < deadline, "timed out waiting for app-server stdout"); @@ -1347,7 +1983,7 @@ impl BridgeProcess { self.stdout_lines.push(line.clone()); let value: Value = serde_json::from_str(line.trim()).expect("valid JSON stdout line"); - validate_jsonrpc_value(&value); + validate_jsonrpc_value(&value, allow_error); return value; } Ok(Err(error)) => panic!("read app-server stdout: {error}"), @@ -1656,7 +2292,7 @@ impl RawProcess { } } -fn validate_jsonrpc_value(value: &Value) { +fn validate_jsonrpc_value(value: &Value, allow_error: bool) { let message: JSONRPCMessage = serde_json::from_value(value.clone()).expect("valid JSON-RPC message"); match message { @@ -1671,7 +2307,9 @@ fn validate_jsonrpc_value(value: &Value) { } } JSONRPCMessage::Response(_) => {} - JSONRPCMessage::Error(error) => panic!("app-server returned JSON-RPC error: {error:?}"), + JSONRPCMessage::Error(error) => { + assert!(allow_error, "app-server returned JSON-RPC error: {error:?}"); + } JSONRPCMessage::Request(request) => { panic!("app-server emitted unexpected request: {request:?}") } @@ -1853,6 +2491,59 @@ done script } +fn fake_codex_interruptible_app_server_script(log_path: &Path) -> String { + let mut script = String::new(); + script.push_str("#!/bin/sh\n"); + script.push_str("log="); + script.push_str(&shell_quote(log_path)); + script.push_str( + r#" +touch "$log" +if [ "${1:-}" = "app-server" ] && [ "${2:-}" = "--help" ]; then + printf '%s\n' '--listen stdio://' + exit 0 +fi +if [ "${1:-}" != "app-server" ]; then + printf '%s\n' 'expected app-server command' >&2 + exit 64 +fi + +request_id() { + printf '%s' "$1" | sed -n 's/.*"id":\([0-9][0-9]*\).*/\1/p' +} + +while IFS= read -r line; do + printf '%s\n' "$line" >> "$log" + case "$line" in + *'"method":"initialize"'*) + id=$(request_id "$line") + printf '{"id":%s,"result":{"userAgent":"fake-codex"}}\n' "$id" + ;; + *'"method":"thread/start"'*) + id=$(request_id "$line") + printf '{"id":%s,"result":{"thread":{"id":"thread-1"}}}\n' "$id" + ;; + *'"method":"turn/start"'*) + id=$(request_id "$line") + printf '{"id":%s,"result":{"turn":{"id":"turn-1"}}}\n' "$id" + printf '%s\n' '{"method":"turn/started","params":{"threadId":"thread-1","turn":{"id":"turn-1","items":[],"itemsView":"full","status":"inProgress","error":null,"startedAt":1,"completedAt":null,"durationMs":null}}}' + ;; + *'"method":"turn/interrupt"'*) + id=$(request_id "$line") + printf '{"id":%s,"result":{}}\n' "$id" + printf '%s\n' '{"method":"turn/completed","params":{"threadId":"thread-1","turn":{"id":"turn-1","items":[],"itemsView":"full","status":"interrupted","error":null,"startedAt":1,"completedAt":2,"durationMs":1}}}' + ;; + *) + printf '%s\n' "unexpected request: $line" >&2 + exit 65 + ;; + esac +done +"#, + ); + script +} + fn fake_codex_app_server_steer_script(log_path: &Path) -> String { let mut script = String::new(); script.push_str("#!/bin/sh\n"); diff --git a/docs/UPSTREAM_SYNC_20260711.md b/docs/UPSTREAM_SYNC_20260711.md new file mode 100644 index 000000000..be935ed1e --- /dev/null +++ b/docs/UPSTREAM_SYNC_20260711.md @@ -0,0 +1,183 @@ +# TipLink Centaur upstream alignment (2026-07-11) + +This branch integrates the exact Paradigm tree at `3c6e84d9` with TipLink +`ba2c01f5`. Conflict resolution used the upstream implementation as the +baseline, then reapplied only TipLink behavior that remains deployment- or +runtime-relevant. The reviewed aggregate tree is transported as GitHub-authored +verified commits, then joined to a second GitHub-verified branch rooted at the +upstream SHA. The final head must therefore contain both histories, reproduce +the tested tree exactly, and satisfy the fork's signature policy. + +## Retained on the current upstream architecture + +| TipLink behavior | Current implementation | +| --- | --- | +| Ordered tool overlays | `install_tool_shims.py` uses later-source replacement and matches package, project, and script identifiers for allow/block lists. | +| Warm first-turn context | `harness-server` derives `CENTAUR_THREAD_KEY` from the first blocks-mode user line before spawning Codex app-server. | +| Active Codex steering | A second user line during an active turn becomes `turn/steer`; its response is consumed rather than leaked into blocks output. | +| Reviewed runtime pins | The sandbox retains TipLink's tested Codex `0.144.1` and Claude Code `2.1.198` pins. Paradigm's Codex `0.144.0` already supports GPT-5.6 Sol, while Claude Code `2.1.197` is the documented Sonnet 5 floor; the retained next-patch pins include later fixes and are verified in the image build. | +| Sandbox development capabilities | Terraform and Playwright `1.58.0` with its native headless shell remain available on amd64 and arm64. `agent-browser` uses its installed Chrome on amd64 and an explicit Playwright-shell path on arm64, where its own installer and discovery do not work. Node tools opt into the sandbox's injected proxy without discarding existing `NODE_OPTIONS`. | +| Table-aware Codex configuration | The image-owned harness is the default base. The entrypoint transforms that copied config as TOML, disables both multi-agent feature forms without colliding with a `[features.multi_agent_v2]` table, applies deployment reasoning and Bedrock settings, then lets a valid operator overlay win. It validates before atomically replacing the config. The native amd64/arm64 image gate runs the real packaged Codex app-server without network access and proves both OpenRouter and Meta provider discovery. `CENTAUR_HARNESS_CONFIG_DIR` remains an explicit full replacement, not a merge. | +| Claude model aliases on a Codex default | Slack, Linear, and GitHub aliases map Sonnet to `claude-sonnet-5`. A known Claude alias in `--model` selects Claude only when an explicit harness/provider flag has not already won; full model IDs remain pass-through escape hatches. | +| Canonical session release | `POST /api/session/{thread_key}/release` locks the session row, fences the expected sandbox, rejects active work unless cancellation is explicit, clears stdout ownership on cancellation, and stops only the snapshotted sandbox. | +| Ambient Slack channels | Configured root messages and replies execute without an explicit mention; messages outside the allowlist remain inert. | +| Slack event dedupe | The patched Chat dependency dedupes by actionable bucket so a non-actionable `message` event cannot suppress a later `app_mention`. | +| Durable terminal reconciliation | Slack compares streamed markdown with the durable terminal result and replaces divergent output. | +| Generic HTTP secret scopes | Method/path scopes are retained through discovery, permission translation, and iron-control registration. | +| Least-privilege Slack ETL token | The reviewed TipLink #68 intent is ported to the current `match_headers` manifest schema: `SLACK_ETL_TOKEN` can replace `Authorization` only for the four ETL Slack Web API paths over `GET`/`POST` and for `GET` downloads from `files.slack.com`. A real-manifest translation test locks the resulting iron-control rules. | +| GitHub App installation tokens | The grant is registered in upstream `Broker::CredentialGrants`; the model delegates validation and refresh to that registry. Helm bootstraps the canonical credential before api-rs starts, and the built-in infra role grants a scheme-preserving `GITHUB_TOKEN` replacement for `github.com` and `api.github.com` to sandbox principals. | +| Fork image publication | TipLink GHCR namespace, GitHub-hosted native builders, safe multi-arch assembly, and upstream `githubbot`/harness image inputs are combined. Pull requests build without registry credentials. Publication is globally serialized and accepts only an exact GitHub-signed, ready-or-merged PR head whose aggregate CI, Console CI, and native image-validation runs are green. A newly created full-SHA publication tag or exact dispatch may write packages; `reviewed-` registry tags are single-assignment. The descriptor binds each final arm64 child to the runnable child produced by that exact run. Fineas promotion is owned by the separately reviewed infra PR DAG. | +| GitHub-hosted CI | TipLink's removal of Depot runners remains authoritative across core, Console, docs, audit, and chart workflows. Native arm64 publication continues on GitHub's arm runner. | +| Human-reviewed upstream tracking | The weekly `upstream-sync.yml` workflow opens a draft cross-repository PR directly from `paradigmxyz:main`, after verifying every upstream-only commit. It never copies unreviewed code into a trusted TipLink branch, so PR workflows retain the external-head token/secret boundary. A separate synchronize-time check fails if the moving upstream/base refs no longer match the SHAs and verified count recorded in the PR body. The PR is only an audit signal; an integration branch must pin that recorded upstream SHA. | +| Trusted publication split | PR docs, chart, and image validation have read-only/no-secret jobs. Cloudflare docs and chart publication retain their reviewed-main confirmation. Runtime image publication is a separate package-writing workflow bound to a signed, ready-or-merged PR head and its exact successful aggregate checks. | +| Trust-lane key separation | API and Console startup reject configured control, bot, workflow, feedback, or JWT signing credentials shorter than 32 bytes or reused across trust lanes without printing secret values. | + +## Transitional deployment compatibility + +- `overlay.image` is deprecated for new deployments but remains functional in + both the api-rs pod and Agent Sandbox pods. It provides an init-copy, + read-only mount, tool/workflow wiring, and sandbox prompt path so staged + rollout and rollback do not require an atomic repo-cache cutover. +- When `CENTAUR_OVERLAY_DIR` identifies an available repository root, that root + is authoritative for both prompts and skills. Its + `services/sandbox/SYSTEM_PROMPT.md` wins when present; intentionally omitting + the file disables the overlay prompt rather than resurrecting stale image + instructions. Image-baked prompt/skill fallbacks apply only when the repo + root itself is unavailable. +- `networkPolicy.legacyManagedByApiServerAccess` defaults to `true`. It keeps + API ingress and egress available to pre-capability-label pods carrying only + `centaur.ai/managed-by=api-rs`. New pods always project + `centaur.ai/api-server-enabled` as `true` or `false`, and the legacy selector + requires that label to be absent, so it cannot grant access to a new + capability-disabled pod. Disable it only after legacy ready sandboxes and + assigned sessions have drained; the schema-forward rollback stage restores + it for bridge-created unlabeled pods. +- Warm-pool reconciliation atomically reserves only `status='ready'` rows with + a workload key different from the current spec before stopping their backend + sandboxes. Claimed or otherwise bound work cannot enter that eviction set. +- Every sandbox assignment is stamped with a digest binding the deployment's + full default-spec generation to that sandbox ID. The first owned turn on an + older assigned thread replaces its stale sandbox; the ID binding prevents a + rollback-era reassignment from inheriting a trusted forward stamp. + +## Upstream replacements and dropped patches + +- Upstream's current Slack render/activity pipeline replaces the old + renderer-specific Thinking patches. The retained terminal mismatch check is + layered onto the durable upstream pipeline. +- Upstream's in-process Slack handoff retry replaces TipLink's older dedupe-key + deletion/retry mechanics. +- TipLink's custom `session.delivery_completed` receipt path was used only by + the retired silent-thread trace/capture workflows. Upstream render + obligations, terminal reconciliation, and fallback delivery remain the + active reliability mechanisms. +- Upstream stdout-owner leases, adoption, shutdown handoff, sandbox capacity, + capability labels, and API routing are authoritative. Session release was + redesigned around those ownership fences rather than replaying the old + sequential release patch. +- Upstream repo-cache overlay sources are the target architecture. Image + overlays are explicitly transitional, not a competing long-term source. +- TipLink's old managed-proxy patch is replaced by the current agent-k8s + implementation, which injects + `IRON_PROXY_UPSTREAM_RESPONSE_HEADER_TIMEOUT=120s` into every managed proxy. + Obsolete line-oriented portions of the old Codex bootstrap were replaced by + the retained table-aware transformer. Legacy GitHub identity enrichment, + tool-specific MPP/Preqin/Drive changes, and stale generated docs/workflows + were not carried into core; they are obsolete under the new architecture or + belong in overlay/tool repositories. +- CodeQL findings in unchanged Paradigm code and inherited test fixtures are + treated as upstream baseline, not fork patches. This sync carries neither + analyzer-only rewrites nor `codeql[...]` suppression comments; review and + rollout gates cover integration-owned behavior instead. +- Claude-as-default changes were not retained in base Centaur. Harness defaults + remain upstream-owned; Fineas-specific defaults belong in the deployment + overlay/configuration. This does not remove the newer, reviewed Claude Code + pin used when a deployment explicitly selects Sonnet 5. + +## TipLink-only commit inventory + +`git cherry` identified 101 TipLink-only commits relative to the reviewed +Paradigm baseline. Every commit is classified below; no commit is implicitly +dropped. + +- Retained directly or satisfied by an identified upstream equivalent (36): + `91234d85 2a03d179 755b5f61 5ec8beb9 429e2be9 1d712a5e 6369763a + 26f9db98 4978561a 7aa8f772 d6dcdb4d 567d1abc 6c522c51 46788900 + 65ce9902 9b5a4bbb f5636a0f 1882c8eb 7617924f 1e902a24 60f3272c + c59a82ae ea51b3ee 26527258 226b9dcd 5aa259cb f65fdc0b 3617f569 + 9d00faca 3d34dc7f 410d3769 abc0f356 b93bd640 b1a4569f e00ccb21 + 28bb47ee`. +- Publication and upstream tracking retained through the redesigned trust + lanes (11): `b52e85ed 6de8d862 d4c87aff 5599cc51 093717ef a6a2fb33 + 1105c098 f30034f8 c3bf8c16 bbb543d2 cb100e05`. +- Mixed commits split by behavior (4): `7dcdf72e` retains overlay-image + compatibility while dropping the old Python API; `163a7a88` retains GitHub + App intent through `CredentialGrants`; `79b5926a` retains workflow behavior + on the current host while dropping obsolete API shapes; `1f21791f` retains + Sonnet 5 aliases while dropping Claude-as-default. +- Superseded, obsolete, or migrated out of core (39): `f54bace2 720fa29c + 63576812 c823cd59 fa7940f6 508861b2 8f458c87 95f20ab3 2f0fec99 + c154bde6 2d5fb4ed b8bbe0c1 c452b02b 2d999e0a 86e6ca2d 5da54a5e + ad5f8d34 ed4f52d2 d844ebab 0d3e0c43 f2bb2e46 40d2837c 87f26b5e + d058b5d3 4f007df0 e413aebc 86068044 1dc5130a 4215ada6 930b1a82 + 9895becb f247bdfb ce677374 3254b0a2 cf5b6749 45bb36e9 036e35ed + db630210 13ec857e`. +- Net-reverted add/revert pairs with no baseline tree effect (4): `2595dcf1 + f9d0b765 dda19688 7ac28097`. +- CodeQL-only history intentionally ignored as inherited upstream baseline + (6): `a8adb1f8 9baa30cf 3a886f7f 2dfabef2 f58f1154 6b0c3b09`. +- Historical follow-up rather than active-baseline carry (1): `07bd5f08`. + Its fallback-post retry was already absent from `ba2c01f5`; restoring it + requires a separate exactly-once delivery decision and test. + +Known one-for-one upstream equivalents include `d6dcdb4d` / `0691b1aa`, +`c59a82ae` / `cc0c4c0c`, `f5636a0f` / `f6664689`, `1882c8eb` / +`1c9a5d62`, `7617924f` / `e12b9d93`, `1e902a24` / `1f944521`, +`60f3272c` / `f51239ee`, `26527258` / `2a6b838d`, and `226b9dcd` / +`a90453b8`. The exact managed-proxy replacement for `9895becb` is the +agent-k8s injection of `IRON_PROXY_UPSTREAM_RESPONSE_HEADER_TIMEOUT=120s`. + +### Reviewed post-baseline carry + +The 101-commit inventory above covers the `ba2c01f5` baseline. This sync also +semantically ports the approved least-privilege Slack ETL token work from +TipLink PR #68 (`c9ebdc58`, `68468602`, `31e14d62`, and `2186f3e5`) onto the +current HTTP-secret manifest and documentation. Those four commits are not +counted in the baseline inventory and are not patch-equivalent because the +upstream secret schema and Slack ETL documentation evolved in parallel. + +## Migration boundary + +TipLink SQLx migrations `0001` through `0032` retain their deployed identities. +Upstream migrations that formerly occupied `0032` through `0038` are shifted to +`0033` through `0039`; `0033`–`0038` keep their upstream bodies, while `0039` +adds the independently reviewed fail-closed Fineas privacy/RLS reconciliation. +Upstream `0040`–`0042` retain both numbering and bodies. Rails migration +`20260624000100_add_password_grant_to_broker_credentials.rb` remains compatible +with TipLink's deployed migration record. New fork migration `0043` adds the +nullable, rollback-compatible sandbox content-revision stamp. Immutable +SQLx/Rails checksum manifests and CI guards are included in this branch. + +## Rollout order + +1. After review and all exact-head non-CodeQL checks are green, create a fresh + `reviewed-images-publish--at-` lightweight tag + directly at the signed PR head (or dispatch with that same full SHA). The + globally serialized publisher refuses any pre-existing component tag, + rechecks immediately before every final tag write, and emits the descriptor + only after every final arm64 child matches this run. A PR run or merge alone + does not enter the package/deployment lanes. If a run creates only a subset + of final tags, never delete or move them and never retry that head; supersede + it with a new signed PR commit and repeat the complete gate. +2. Deploy the complete api-rs and Slackbot revision with legacy network-policy + access and `overlay.image` compatibility enabled, then restore ingress. +3. Run one controlled Slack turn and confirm its primary or fallback message + is visible through the retained render-obligation recovery path. +4. Let workload-key reconciliation retire stale unclaimed warm sandboxes; + existing assigned sessions replace their sandbox on their next owned turn, + while explicit cancellation still uses canonical release. +5. Verify new ready pods carry capability labels, repo-backed prompts, and the + expected workload key. +6. Drain all legacy sessions, then disable + `networkPolicy.legacyManagedByApiServerAccess` and eventually remove the + image-overlay values from the Fineas deployment. diff --git a/docs/pages/architecture.mdx b/docs/pages/architecture.mdx index 53a44e0ea..eba179f1d 100644 --- a/docs/pages/architecture.mdx +++ b/docs/pages/architecture.mdx @@ -52,9 +52,17 @@ https://api.acme.com/api/webhooks/slack The webhook does not use a Centaur API key. Slack signs every request with `X-Slack-Signature` and `X-Slack-Request-Timestamp`; the Slackbot validates that HMAC signature with `SLACK_SIGNING_SECRET` before it routes the event to the API. -After validation, the Slackbot calls Centaur's agent API with +After validation, the Slackbot calls Centaur's api-rs session API with `SLACKBOT_API_KEY`. +Ingress bot keys are session-API service credentials; they are not global +operator credentials or per-channel authorization. Administrative data routes, workflow administration, +and fleet-wide operations such as sandbox drain require the distinct +`CENTAUR_CONTROL_API_KEY`. Sandbox tools never receive that key. The optional +feedback tool instead combines `SLACK_FEEDBACK_API_KEY` with its Console JWT; +the API binds each `feedback-improvement:*` session to that caller's existing +principal and capabilities. + During a Slack delivery, the API owns the execution state while Slackbot owns Slack rendering: opening or updating the thread UI, streaming chunks, rendering steps, and posting the final answer. The landing page preview shows that Slack diff --git a/docs/pages/deploying-in-production.mdx b/docs/pages/deploying-in-production.mdx index ead9ffbe0..c0c6d26d6 100644 --- a/docs/pages/deploying-in-production.mdx +++ b/docs/pages/deploying-in-production.mdx @@ -17,7 +17,7 @@ creates sandbox pods for agent work. [iron-proxy](https://docs.iron.sh) handles need credentials:
- Centaur production workflow — Centaur API plus Postgres hands a run to the Kubernetes backend, which attaches a sandbox pod whose outbound HTTP routes through iron-proxy + Centaur production workflow: Centaur API plus Postgres hands a run to the Kubernetes backend, which attaches a sandbox pod whose outbound HTTP routes through iron-proxy
Slackbot and API ingress → Centaur API (Postgres-backed) → Kubernetes sandbox runtime → outbound traffic through iron-proxy.
@@ -62,17 +62,25 @@ Minimum keys: | `DATABASE_URL` | API | Postgres connection string. | | `IRON_MANAGEMENT_API_KEY` | [iron-proxy](https://docs.iron.sh) management API | Generate with `openssl rand -hex 32`. | | `SANDBOX_SIGNING_KEY` | Sandbox API tokens | Generate with `openssl rand -hex 32`; keeps sandbox tokens valid across API restarts. | -| `SLACK_BOT_TOKEN` | Slackbot | Bot User OAuth Token from the Slack app. | -| `SLACK_UPLOAD_TOKEN` | Slack file uploads | Dedicated Slack token for tool-driven file upload and verification. | +| `SLACK_BOT_TOKEN` | Slackbot/API | Bot User OAuth Token from the Slack app. | | `SLACK_SIGNING_SECRET` | Slackbot/API | Used to verify Slack webhook signatures. | | `SLACKBOT_API_KEY` | Slackbot to API | Static service token; API bootstraps it into Postgres on startup with `agent` scope. | +| `CENTAUR_CONTROL_API_KEY` | Console/operator to API | Dedicated high-entropy service token for administrative routes and global controls. Never reuse a bot key or expose this value to sandboxes. | +| `SLACK_FEEDBACK_API_KEY` | Optional sandbox feedback tool to API | Separate narrow token that can create and operate only `feedback-improvement:*` sessions. | | `OP_CONNECT_TOKEN` | [iron-proxy](https://docs.iron.sh) 1Password Connect source (preferred) | Needed when `ironProxy.secretSource` is `onepassword-connect`. | | `OP_SERVICE_ACCOUNT_TOKEN` | [iron-proxy](https://docs.iron.sh) 1Password service-account source | Needed when `ironProxy.secretSource` is `onepassword`. | | `OP_VAULT` | [iron-proxy](https://docs.iron.sh) 1Password source | Vault name or id used for `op://` references (either mode). | -`SLACKBOT_API_KEY` is not created with the admin API during initial boot, because -the API process requires it before it can start. Generate a high-entropy value, -store it in the infra Secret, and reuse the same value in Slackbot. +`SLACKBOT_API_KEY` and `CENTAUR_CONTROL_API_KEY` are not created with the admin +API during initial boot, because the API process requires them before it can +start. Generate distinct high-entropy values, store them in the infra Secret, +and reuse only the Slackbot value in Slackbot. Local bootstrap generates and +preserves the control and feedback keys automatically. + +For upgrades using `secretManager.existingSecretName`, create the prefixed +`CENTAUR_CONTROL_API_KEY` Secret entry before applying the new chart. The API +and Console references are non-optional and otherwise produce a pod startup +failure. This secret prerequisite is the first gate in the upgrade runbook. ## 3. Configure harness credentials @@ -82,6 +90,7 @@ Store one secret per enabled harness credential: |---------|-----------|----------------|---------------------|----------| | Codex default | `codex` | none or `--codex` | `OPENAI_API_KEY` | `api.openai.com` | | Codex with OpenRouter provider | `codex` | none or `--codex` | `OPENROUTER_API_KEY` | `openrouter.ai` | +| Codex with Meta AI direct | `codex` | `--meta` | `META_AI_API_KEY` | `api.ai.meta.com` | | Amp | `amp` | `--amp` | `AMP_API_KEY` | `ampcode.com` | | Claude Code | `claude-code` | `--claude` | `ANTHROPIC_API_KEY` | `api.anthropic.com` | | pi-mono | `pi-mono` | `--pi` | `ANTHROPIC_API_KEY` | `api.anthropic.com` | @@ -100,6 +109,10 @@ alongside `CODEX_MODEL`. Per-turn Codex model overrides with provider-style slugs such as `--model anthropic/claude-fable-5` also select the OpenRouter provider even when `OPENROUTER_MODEL` is unset. +To run Codex through Meta AI direct, store `META_AI_API_KEY` and select the +provider with `--meta`. Pair it with `--model ` when choosing a +provider-specific model for a turn. + Whatever source you pick, the vault is shared across the whole deployment, so any thread can use any configured credential. Per-user and per-channel scoping is on the roadmap; until then, scope tool and harness access @@ -204,10 +217,9 @@ Use the app page to install the bot, copy the Bot User OAuth Token for 1. Add the bot scopes required by the Slackbot features you enable. 2. Install the app to the workspace. 3. Store the Bot User OAuth Token as `SLACK_BOT_TOKEN`. -4. Store a dedicated upload-capable Slack token as `SLACK_UPLOAD_TOKEN` if Slack file uploads are enabled. -5. Store the app Signing Secret as `SLACK_SIGNING_SECRET`. -6. Enable Event Subscriptions. -7. Set the Request URL to `https:///api/webhooks/slack`. +4. Store the app Signing Secret as `SLACK_SIGNING_SECRET`. +5. Enable Event Subscriptions. +6. Set the Request URL to `https:///api/webhooks/slack`. 7. Subscribe to `app_mention` and to the message events you want Centaur to see: `message.channels`, `message.groups`, and `message.im`. @@ -240,6 +252,10 @@ ironProxy: secretSource: onepassword-connect secretTtl: 10m +apiRs: + # Delete any sandbox older than this, running or suspended. + sandboxMaxLifetimeSecs: 259200 + onepasswordConnect: connect: create: true @@ -257,6 +273,17 @@ sandbox: The Kubernetes sandbox backend is the active runtime backend; there is no chart switch named `api.sandboxBackend`. +Sandbox lifecycle has two separate timers: + +- Slackbot v2 sends `idle_timeout_ms` on execute requests, defaulting to up to + 3 hours, so api-rs can pause an idle sandbox after a turn finishes. +- api-rs deletes old sandboxes through `apiRs.sandboxMaxLifetimeSecs`, default + 72 hours, regardless of whether the sandbox is still running or already + suspended. + +There is no suspended-only delete setting. If you want sandboxes gone after N +hours, set `apiRs.sandboxMaxLifetimeSecs` to N hours in seconds. + Install or upgrade: ```bash @@ -285,21 +312,28 @@ Run one agent turn from inside the api-rs deployment: THREAD_KEY=cli:production-smoke-codex THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') -SESSION=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}" \ - -H "Content-Type: application/json" \ - -d '{"harness_type":"codex","on_harness_conflict":"restart"}') - -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/messages" \ - -H "Content-Type: application/json" \ - -d '{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG."}]}]}' +SESSION=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"harness_type":"codex","on_harness_conflict":"restart"}'\''' sh "$THREAD_PATH") -EXECUTE=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/execute" \ - -H "Content-Type: application/json" \ - -d '{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG.\"}]}}"]}') +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1/messages" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG."}]}]}'\''' sh "$THREAD_PATH" + +EXECUTE=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1/execute" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG.\"}]}}"]}'\''' sh "$THREAD_PATH") EXECUTION_ID=$(printf '%s' "$EXECUTE" | jq -r '.execution_id') -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -N \ - "http://localhost:8080/api/session/${THREAD_PATH}/events?execution_id=${EXECUTION_ID}&after_event_id=0" +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -N "http://localhost:8080/api/session/$1/events?execution_id=$2&after_event_id=0" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}"' sh "$THREAD_PATH" "$EXECUTION_ID" ``` Then run the same prompt through Slack: @@ -322,8 +356,9 @@ If a run fails because the sandbox pod exits or is deleted, inspect the durable session and api-rs logs before retrying: ```bash -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/session/${THREAD_PATH}" | jq +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s "http://localhost:8080/api/session/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}"' sh "$THREAD_PATH" | jq kubectl logs -n centaur-system deploy/centaur-centaur-api-rs --tail=200 kubectl get pods -n centaur-system -l centaur.ai/managed=true diff --git a/docs/pages/extend/acme-example.mdx b/docs/pages/extend/acme-example.mdx index 7309446be..ce9374fd1 100644 --- a/docs/pages/extend/acme-example.mdx +++ b/docs/pages/extend/acme-example.mdx @@ -220,11 +220,14 @@ Expected paths include: /home/agent/github/your-org/centaur-acme/.agents/skills ``` -You can also inspect the api-rs session context for a thread: +From a trusted operator shell, you can also inspect the api-rs session context +for a thread. (Inside a sandbox, iron-proxy supplies the narrower per-principal +JWT automatically; never copy the control key into a sandbox.) ```bash THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') -curl -s "$CENTAUR_API_URL/api/session/${THREAD_PATH}" | jq +curl -s "$CENTAUR_API_URL/api/session/${THREAD_PATH}" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?required}" | jq ``` ## What to change first diff --git a/docs/pages/extend/overlay.mdx b/docs/pages/extend/overlay.mdx index c27044e52..33acb6912 100644 --- a/docs/pages/extend/overlay.mdx +++ b/docs/pages/extend/overlay.mdx @@ -48,9 +48,11 @@ overlays: sources: - repo: paradigmxyz/centaur ref: main + visibility: public - repo: your-org/centaur-overlay ref: main + visibility: private ``` `repo` is `owner/name` on GitHub. `ref` can be a branch, tag, or commit SHA; @@ -60,6 +62,11 @@ production rollout, but many overlay repos intentionally track `main` so a reviewed merge is enough for new sandboxes to pick up the change after repo-cache refreshes. +`visibility` controls which sandboxes may receive the repo-cache checkout. +It defaults to `private`. Set `visibility: public` only for repos whose full +contents are safe to expose to principals configured with +`sandbox_repo_cache=public`; invalid or missing values are treated as `private`. + Each source defaults to the conventional layout — `toolsSubdir: tools`, `workflowsSubdir: workflows`, `skillsSubdir: .agents/skills` — and directories a repo does not contain are skipped at runtime, so a skills-only overlay needs @@ -128,15 +135,10 @@ overlay: Add deployment-specific agent guidance here. ``` -When using a legacy overlay image, `overlay.mountPath` is the api-rs mount and -`overlay.sandboxMountPath` is the matching sandbox/workflow-host mount. Keep -them separate unless your deployment intentionally uses the same filesystem -layout in both containers. - -For larger prompt/persona sets, keep files in an overlay repo and expose tool, -workflow, and skill paths through `overlays.sources` where possible. Existing -deployments can continue using `overlay.image.*` while they migrate remaining -prompt, harness, and persona assets onto repo-cache-backed overlay sources. +For larger prompt/persona sets, keep files in an overlay repo and expose their +paths through `overlays.sources` as that surface is wired into your deployment. +Do not rely on `overlay.image.*`; repo-cache-backed overlays are the default +delivery path. ## Verify the overlay diff --git a/docs/pages/extend/workflows-v2.mdx b/docs/pages/extend/workflows-v2.mdx index 7f2d24171..16ae4c515 100644 --- a/docs/pages/extend/workflows-v2.mdx +++ b/docs/pages/extend/workflows-v2.mdx @@ -69,32 +69,28 @@ Supported v2 primitives: ### Keep imports narrow -Workflow files should import only the workflow context compatibility module: +Workflow files should import only the supported workflow-host API surface they +need: ```python from api.workflow_engine import WorkflowContext +from api.runtime_control import ControlPlaneError ``` -Do not import Python API internals such as: +Supported workflow-host modules are `api.workflow_engine`, +`api.runtime_control`, `api.app`, and `api.metrics`. -```python -from api.runtime_control import canonical_json -from api.vm_metrics import workflow_counter -``` - -Those modules were implementation details of the Python API service. In v2, -the workflow host provides a small compatibility surface instead of the whole -Python API package. +Do not import unrelated API-service internals or another workflow domain's local +helpers. Domain-specific helpers should live next to the workflows that own +them, for example `workflows/slack/metrics.py`. If a workflow needs a helper, move it into the workflow file, a shared overlay -module, or a supported workflow-host compatibility shim. +module, or a supported workflow-host API module. -### Make stepped side effects idempotent +### Put side effects behind steps -The handler may be replayed after a crash or retry. `ctx.step(...)` -checkpoints only after the callback returns. If the callback performs an -external write, use a stable provider-side idempotency key or upsert so replay -does not duplicate the side effect: +The handler may be replayed after a crash or retry. Any external write should +be wrapped in `ctx.step(...)` so the result is checkpointed: ```python async def handler(inp: dict, ctx: WorkflowContext) -> dict: @@ -184,8 +180,8 @@ For each existing workflow: `api.workflow_engine.WorkflowContext`. 3. Confirm third-party Python packages are installed in the workflow-host sandbox image. -4. Make Slack posts, database writes, external HTTP calls, and tool calls - idempotent before putting them in `ctx.step(...)`. +4. Wrap Slack posts, database writes, external HTTP calls, and tool calls in + `ctx.step(...)` when they must not repeat. 5. Replace direct agent-control-plane calls with `ctx.agent_turn(...)`. 6. If the workflow uses `ctx._pool`, confirm the workflow-host sandbox receives `DATABASE_URL`. @@ -196,10 +192,10 @@ For each existing workflow: ## Known gaps -The v2 POC supports the workflow model, but it does not yet emulate the full -Python API package. Workflows that import `api.runtime_control`, `api.vm_metrics`, -or other Python API internals need a compatibility shim or a small local helper -before they are v2-ready. +The v2 workflow host intentionally exposes a narrow Python API package. +Workflows that import unrelated API-service internals should move that behavior +into the workflow-host API surface or a small local helper owned by the workflow +domain before they are v2-ready. `ctx.call_tool(...)` is a compatibility surface in the Python workflow host. It uses the generated `centaur-tools call` bridge against the installed tool @@ -221,6 +217,7 @@ Then create a real run: ```bash curl -s "$CENTAUR_API_URL/api/workflows/runs" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_name": "nightly_report", diff --git a/docs/pages/extend/workflows.mdx b/docs/pages/extend/workflows.mdx index d9b1b8b50..d5d6d46fd 100644 --- a/docs/pages/extend/workflows.mdx +++ b/docs/pages/extend/workflows.mdx @@ -1,14 +1,14 @@ --- title: Creating Workflows -description: Add durable Centaur workflows with checkpointed steps, sleeps, schedules, webhooks, Slack posts, tool calls, and agent turns. +description: Add durable Centaur workflows with checkpointed steps, sleeps, events, child workflows, and agent turns. --- # Creating Workflows -Workflows are Python handlers that run through Centaur's durable api-rs -workflow engine. They are useful when the task is longer than one agent turn: -polling, branching, retries, scheduled syncs, webhook handling, or coordinating -agent runs. +Workflows are Python handlers that run through Centaur's durable workflow +engine. They are useful when the task is longer than one agent turn: polling, +branching, retries, waiting for external events, or coordinating multiple agent +runs. Use a workflow when the system needs durable progress rather than a single request-response turn. Common examples include scheduled reports, ETL syncs, @@ -37,6 +37,7 @@ An optional `Input` dataclass gives structured inputs. ```python from dataclasses import dataclass +from datetime import timedelta from typing import Any from api.workflow_engine import WorkflowContext @@ -53,10 +54,10 @@ class Input: async def handler(inp: Input, ctx: WorkflowContext) -> dict[str, Any]: data = await ctx.step("collect", lambda: {"topic": inp.topic}) - await ctx.sleep_for("settle", 30) + await ctx.sleep("settle", timedelta(seconds=30)) result = await ctx.run_agent( - f"Write a short report about {data['topic']}", - thread_key=f"workflow:{ctx.run_id}:nightly_report", + "summarize", + text=f"Write a short report about {data['topic']}", ) return {"channel": inp.channel, "report": result} ``` @@ -65,18 +66,17 @@ async def handler(inp: Input, ctx: WorkflowContext) -> dict[str, Any]: | Primitive | Use it for | |-----------|------------| -| `ctx.step(name, fn)` | Run a deterministic or idempotent operation and cache its result after the callback succeeds. | -| `ctx.sleep_for(name, seconds)` | Suspend and resume later. | +| `ctx.step(name, fn)` | Run a side effect once and cache its result. | +| `ctx.sleep(name, duration)` | Suspend and resume later. | | `ctx.sleep_until(name, when)` | Resume at a specific time. | -| `ctx.agent_turn(text, **kwargs)` / `ctx.run_agent(text, **kwargs)` | Start an agent turn and wait for the result. | -| `ctx.call_tool(tool, method, args)` | Call a tool through the workflow-host `centaur-tools call` bridge. | -| `ctx.post_to_slack(channel, text, **kwargs)` | Post to Slack through the api-rs Slack context path. | -| `ctx._pool` | Access the workflow database pool when the workflow-host sandbox receives `DATABASE_URL`. | +| `ctx.wait_for_event(name, event_type, correlation_id)` | Wait for an external event. | +| `ctx.wait_for_workflow(...)` | Wait for a child workflow to finish. | +| `ctx.run_workflow(...)` | Start and wait in one call. | +| `ctx.start_agent(...)` | Start an agent turn. | +| `ctx.run_agent(...)` | Start an agent turn and wait for the result. | -The handler may re-execute after a restart. `ctx.step(...)` memoizes the result -after its callback returns, so writes and external API calls inside a step must -still be idempotent. If the host crashes after the external side effect but -before the checkpoint commits, the step can replay. +The handler may re-execute after a restart. Put external side effects behind +`ctx.step(...)` so completed work is not repeated. These primitives compose into larger automations: @@ -84,17 +84,25 @@ These primitives compose into larger automations: or business-hours monitor without a human prompt. - **Polling loops**: sleep between checks for CI, blockchain confirmations, billing state, deploy health, or vendor exports. -- **Event-driven flows**: expose a signed webhook and let the handler process a - normalized webhook envelope. +- **Event-driven flows**: wait for a webhook, approval, upload, or callback and + continue from the last checkpoint. +- **Fan-out/fan-in orchestration**: start child workflows for independent work + and wait for all of them before producing a final result. - **Agent orchestration**: use agents for judgment-heavy steps while the workflow owns timing, retries, state, and final delivery. ## Run a workflow -Create a run through the API: +The manual control API requires the trusted `CENTAUR_CONTROL_API_KEY` (or an +optional dedicated `WORKFLOW_API_KEY`). Agent tools use a separate Console JWT +lane: their workflow name must be listed in `WORKFLOW_API_ALLOWED_NAMES`, and +`input.thread_key` must belong to one of the JWT's Slack upload channels. + +Create a run through the trusted operator lane: ```bash curl -s "$CENTAUR_API_URL/api/workflows/runs" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_name": "nightly_report", @@ -106,7 +114,8 @@ curl -s "$CENTAUR_API_URL/api/workflows/runs" \ Inspect it: ```bash -curl -s "$CENTAUR_API_URL/api/workflows/runs/$RUN_ID" | jq +curl -s "$CENTAUR_API_URL/api/workflows/runs/$RUN_ID" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" | jq ``` ## Schedule a workflow @@ -154,9 +163,8 @@ declared in the schedule instead of inferred from wall-clock state when possible. For workflows that may run longer than their schedule interval, make each tick -idempotent. Put writes and external API calls in named `ctx.step(...)` blocks -only when they use stable provider-side idempotency keys or upserts, derive -those keys from the scheduled window, and have the handler detect +idempotent. Put writes and external API calls in named `ctx.step(...)` blocks, +derive stable keys from the scheduled window, and have the handler detect already-processed periods before starting expensive work. Interval schedules are useful when exact wall-clock alignment does not matter: @@ -182,29 +190,24 @@ entrypoints such as GitHub issue triage, billing events, or deploy callbacks. ```python from typing import Any +from api.webhooks import HeaderTriggerKey, HmacAuth, WebhookSpec from api.workflow_engine import WorkflowContext WORKFLOW_NAME = "github_issue_triage" WEBHOOKS = [ - { - "slug": "github-issue-triage", - "provider": "github", - "auth": {"type": "github", "secret_ref": "GITHUB_WEBHOOK_SECRET"}, - "trigger_key": {"type": "header", "header": "X-GitHub-Delivery"}, - "allowed_methods": ["POST"], - "allowed_content_types": [ + WebhookSpec( + slug="github-issue-triage", + provider="github", + auth=HmacAuth.github(secret_ref="GITHUB_WEBHOOK_SECRET"), + trigger_key=HeaderTriggerKey("X-GitHub-Delivery"), + allowed_methods=["POST"], + allowed_content_types=[ "application/json", "application/x-www-form-urlencoded", ], - "filter": { - "all": [ - {"source": "header", "key": "x-github-event", "op": "equals", "value": "issues"}, - {"source": "body", "key": "action", "op": "in", "values": ["opened", "reopened"]}, - ] - }, - } + ) ] @@ -213,6 +216,9 @@ async def handler(inp: dict[str, Any], ctx: WorkflowContext) -> dict[str, Any]: headers = webhook["headers"] payload = webhook["body"] + if headers.get("x-github-event") != "issues": + return {"skipped": True, "reason": "unsupported_event"} + issue = payload["issue"] repo = payload["repository"]["full_name"] result = await ctx.agent_turn( @@ -233,17 +239,11 @@ For GitHub, set the webhook secret to the same value as GitHub's default `application/x-www-form-urlencoded` payloads also work when that content type is listed in `allowed_content_types`. -Use `filter` for provider events that can be rejected from headers or JSON body -fields. The API evaluates the filter before creating a workflow run, which -keeps org-wide webhooks from spawning a sandbox for events the handler would -immediately skip. - Webhook requests do not use Centaur API keys. The API verifies the provider -signature before creating workflow state. Use -`{"type": "github", "secret_ref": "GITHUB_WEBHOOK_SECRET"}` for GitHub -`X-Hub-Signature-256` webhooks, or `{"type": "hmac", ...}` for other SHA-256 -HMAC providers. During local development or for trusted internal routes, use -`{"type": "none"}`. +signature before creating workflow state. `HmacAuth.github(...)` verifies +`X-Hub-Signature-256`; a plain `HmacAuth(...)` can be used for other +SHA-256 HMAC providers. During local development or for trusted internal +routes, `auth="none"` is allowed. The workflow receives input in this shape: diff --git a/docs/pages/operate/slack-etl.mdx b/docs/pages/operate/slack-etl.mdx index 1a9747de8..111dc557a 100644 --- a/docs/pages/operate/slack-etl.mdx +++ b/docs/pages/operate/slack-etl.mdx @@ -6,13 +6,13 @@ description: Sync Slack channel history into Postgres, drain historical backfill # Slack ETL :::warning[Off by default in production] -Slack ETL is disabled unless the API service has `SLACK_ETL_ENABLED=true`. +Slack ETL is disabled unless Helm values set `apiRs.etl.slack.enabled=true`. Production deployments should enable it deliberately after choosing the Slack token, channel scope, exclusion patterns, and data boundary they want agents to use. ::: -Slack ETL keeps an indexed, queryable copy of public Slack history in Postgres +Slack ETL keeps an indexed, queryable copy of Slack channel history in Postgres for agent context and operator workflows. It runs as scheduled Centaur workflows: one workflow keeps recent channel history fresh, one drains deferred historical backfill work, and one turns synced messages into company context @@ -27,7 +27,7 @@ token and writes durable rows into Postgres. | Workflow | Default cadence | Role | |----------|-----------------|------| -| `slack_sync` | 1 hour | Lists public channels, refreshes users, syncs recent root messages, advances per-channel checkpoints, and enqueues backfill jobs. | +| `slack_sync` | 1 hour | Lists channels, refreshes users, syncs recent root messages, advances per-channel checkpoints, and enqueues backfill jobs. | | `slack_backfill` | 10 minutes | Claims queued backfill jobs and drains Slack cursors without slowing the incremental sync. | | `company_context_documents` | 4 hours | Projects changed Slack rows into `company_context_documents` for retrieval. | @@ -39,27 +39,39 @@ posting to Slack. Create a Slack user token for ETL reads and store it as `SLACK_ETL_TOKEN` in the same secret source used by tools. The Slack tool declares it as an optional -HTTP secret for `slack.com` and `files.slack.com`; iron-proxy injects the real -value when the tool calls Slack. +HTTP secret scoped to `GET` and `POST` calls to the Slack Web API endpoints +below, plus `GET` downloads from `files.slack.com`; iron-proxy replaces the +`Authorization` header only for those requests. The token must be able to call: | Slack API | Used for | |-----------|----------| -| `conversations.list` | Discover public channels. | +| `conversations.list` | Discover public channels, and private channels when explicitly enabled. | | `conversations.history` | Read channel root messages. | | `conversations.replies` | Refresh thread replies. | | `users.list` | Resolve Slack user metadata for documents. | | `files:read` / file URL access | Download message attachment bytes from `files.slack.com`. | -Slack ETL currently syncs public channels visible to the configured ETL user -token. It does not sync private channels, DMs, or Slackbot-only live thread -events. +Slack ETL syncs public channels visible to the configured ETL user token. +Set `SLACK_SYNC_INDEX_PRIVATE_CHANNELS=true` to also sync private channels +visible to that token. It does not sync DMs or Slackbot-only live thread events. +Private channel rows are protected by RLS: `centaur_readonly` sees public +channel data and the channel in `centaur.slack_channel_id`. ## Enable the schedules -Set `SLACK_ETL_ENABLED=true` on the API service. The other schedules default on -once Slack ETL is enabled, but can be tuned independently. +Set `apiRs.etl.slack.enabled=true` in Helm values. The chart renders the +corresponding API and workflow-host env automatically; do not set +`SESSION_SANDBOX_PASSTHROUGH_ENV` by hand for these ETLs. The other schedules +default on once Slack ETL is enabled, but can be tuned independently. + +```yaml +apiRs: + etl: + slack: + enabled: true +``` | Environment variable | Default | Effect | |----------------------|---------|--------| @@ -71,15 +83,17 @@ once Slack ETL is enabled, but can be tuned independently. | `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `5` | Maximum Slack history pages drained before a job is requeued. | | `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS` | `30` | Historical window seeded for first-time channel backfills. | | `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `3` | Recent thread window eligible for reply refresh. | +| `SLACK_SYNC_INDEX_PRIVATE_CHANNELS` | `false` | Includes private channels visible to the ETL token in Slack sync and backfill. | | `SLACK_ETL_ATTACHMENTS_ENABLED` | `true` | Download Slack message attachment bytes into Postgres. Metadata rows are still written when downloads are disabled. | | `SLACK_ETL_ATTACHMENT_MAX_BYTES` | `10485760` | Per-file byte cap for Slack attachment downloads. Oversized files keep metadata with `skipped_too_large` status. | | `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | empty | Comma-separated channel-name globs to skip, without needing the leading `#`. | | `SLACK_RETENTION_ENABLED` | `true` | Allows the `slack_retention` schedule to run when at least one Slack retention TTL is positive. | | `SLACK_RETENTION_INTERVAL_MINUTES` | `60` | How often to prune Slack retention-managed rows. | -| `SLACK_ETL_RETENTION_DAYS` | `0` | Deletes public Slack ETL messages, derived Slack documents, and terminal ETL run/job rows older than this many days. `0` disables public ETL retention. | +| `SLACK_ETL_RETENTION_DAYS` | `0` | Deletes Slack ETL messages, derived Slack documents, and terminal ETL run/job rows older than this many days. `0` disables ETL retention. | | `SLACK_DM_RETENTION_DAYS` | `0` | Deletes Slack DM messages, stale empty DM conversations, and terminal DM run/job rows older than this many days. `0` disables DM retention. | | `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `true` | Enables projection from Slack sync rows into company context documents. | | `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `14400` | How often to project changed Slack rows into documents. | +| `COMPANY_CONTEXT_DOCUMENTS_MAX_WINDOW_SECONDS` | `21600` | Maximum source `updated_at` window projected by one company context documents run. | Example exclusion list: @@ -93,7 +107,7 @@ Slack ETL writes normalized Slack data into dedicated tables: | Table | Contents | |-------|----------| -| `slack_sync_channels` | Public channels visible to the ETL token and whether they are currently syncable. | +| `slack_sync_channels` | Channels visible to the ETL token, channel privacy, and whether they are currently syncable. | | `slack_sync_users` | Slack user display metadata used when rendering documents. | | `slack_sync_runs` | One row per incremental or backfill workflow run, with counts and channel outcomes. | | `slack_sync_messages` | Root messages and replies keyed by `(channel_id, message_ts)`. | @@ -125,18 +139,21 @@ TTL is positive. ## Run it manually Use a manual run when enabling the feature or testing a configuration change. -From inside the API deployment, localhost bypass avoids needing an external API -key: +The control endpoint remains authenticated on localhost; run curl inside the +API deployment so the trusted `CENTAUR_CONTROL_API_KEY` never leaves the pod: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "slack_sync", "input": {"metadata": {"reason": "manual_check"}}, "eager_start": true - }' | jq +} +JSON ``` Then inspect the run: @@ -144,34 +161,42 @@ Then inspect the run: ```bash RUN_ID=wfr_... -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/workflows/runs/${RUN_ID}" | jq +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s "http://localhost:8080/api/workflows/runs/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' sh "$RUN_ID" | jq ``` To drain pending historical work immediately: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "slack_backfill", "input": {"channel_batch_limit": 10}, "eager_start": true - }' | jq +} +JSON ``` To force document projection after rows have synced: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "company_context_documents", "input": {}, "eager_start": true - }' | jq +} +JSON ``` ## Verify @@ -179,8 +204,10 @@ kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ Check the workflow schedules: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - http://localhost:8080/api/workflows/schedules | jq \ +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s http://localhost:8080/api/workflows/schedules \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' | jq \ '.schedules[] | select(.schedule_id == "slack_sync" or .schedule_id == "slack_backfill" @@ -191,8 +218,10 @@ kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ Check recent workflow runs: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/workflows/runs?limit=20" | jq \ +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s "http://localhost:8080/api/workflows/runs?limit=20" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' | jq \ '.runs[] | select(.workflow_name == "slack_sync" or .workflow_name == "slack_backfill" @@ -244,8 +273,8 @@ setting alerts. | Symptom | What to check | |---------|---------------| | Schedules are missing | Confirm `WORKFLOW_DIRS` includes `/app/workflows` and the API restarted after the workflow files were deployed. | -| Schedules exist but are disabled | Confirm `SLACK_ETL_ENABLED=true` is present in the API environment. | -| `slack_sync` skips with `no_public_channels` | Confirm the ETL user token can see the expected public channels. | +| Schedules exist but are disabled | Confirm Helm values set `apiRs.etl.slack.enabled=true` and the API pod was restarted. | +| `slack_sync` skips with `no_channels` | Confirm the ETL user token can see the expected public channels, or enable private channel sync when only private channels are in scope. | | Channels are all skipped | Check `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` for broad globs. | | Checkpoints show `missing_scope` or `not_allowed_token_type` | Add the missing Slack OAuth scope or use the expected user-token class. | | Backfill jobs keep failing | Inspect `slack_sync_backfill_jobs.last_error` and the corresponding `slack_sync_runs` row. | diff --git a/docs/pages/quickstart.mdx b/docs/pages/quickstart.mdx index 46d19d907..7faf05946 100644 --- a/docs/pages/quickstart.mdx +++ b/docs/pages/quickstart.mdx @@ -78,11 +78,9 @@ Application-level model and tool secrets, such as `OPENAI_API_KEY`, placeholder values and [iron-proxy](https://docs.iron.sh) injects the real credentials only on approved outbound requests. -The default harness is `claudecode`, so an Anthropic credential -(`ANTHROPIC_API_KEY`, or the brokered subscription token in `access_token` -mode) must exist in the configured secret source before Slack agent turns can -complete. Use explicit harness selectors only when you want a non-default -harness such as Codex or Amp. +The default harness is `codex`, so `OPENAI_API_KEY` must exist in the configured +secret source before Slack agent turns can complete. Use explicit harness +selectors only when you want a non-default harness such as Amp or Claude Code. ## 3. Boot the stack diff --git a/docs/pages/reference/configuration.mdx b/docs/pages/reference/configuration.mdx index a1d3cfe8e..5191c6a55 100644 --- a/docs/pages/reference/configuration.mdx +++ b/docs/pages/reference/configuration.mdx @@ -35,7 +35,8 @@ These must exist for the normal Helm deployment. For local development, | `DATABASE_URL` | `secretManager.existingSecretName`; local bootstrap generates it. | API and Slackbot Postgres connection. | | `SLACK_SIGNING_SECRET` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack request signature verification. | | `SLACKBOT_API_KEY` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Static API key bootstrapped for Slackbot. | -| `SLACK_BOT_TOKEN` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack Web API access for Slackbot. | +| `CENTAUR_CONTROL_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. The key name is configurable with `apiRs.controlApiKeySecretKey`. | Trusted Console/operator authorization for global and administrative API routes. Never expose it to sandboxes or reuse another service key; api-rs fails startup when configured trust-lane credentials collide. | +| `SLACK_BOT_TOKEN` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack Web API access for Slackbot and api-rs Slack helpers. | | `SANDBOX_SIGNING_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Signing key for short-lived sandbox API tokens. | | `IRON_MANAGEMENT_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Management key for API-created iron-proxy pods. | | `IRON_BROKER_TOKEN` | `secretManager.existingSecretName`; required when `tokenBroker.enabled=true`. | Bearer token iron-proxy presents to iron-token-broker and the broker enforces on its HTTP API. | @@ -51,6 +52,7 @@ Optional required-by-mode variables: | `LOCAL_DEV_API_KEY` | API env. | Static local admin/dev key bootstrapped into Postgres. | | `TEAMS_BOT_APP_ID`, `TEAMS_BOT_APP_PASSWORD`, `TEAMS_BOT_APP_TENANT_ID` | Local shell before `just bootstrap-secrets`; production Secret. | Required by Teamsbot when `teamsbot.enabled=true`. | | `TEAMSBOT_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it when Teams credentials are present and it is omitted. | Static API key used by Teamsbot. | +| `SLACK_FEEDBACK_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Optional second factor for the sandbox Slack feedback/improvement tool. The API also requires the caller's Console JWT and preserves that principal's capabilities. Keep it distinct from `CENTAUR_CONTROL_API_KEY`. | ## API @@ -70,7 +72,7 @@ Optional required-by-mode variables: | `SLACKBOT_URL` | Chart-rendered Slackbot service URL. | API callback target for Slack delivery. | | `FINAL_DELIVERY_MAX_ATTEMPTS`, `FINAL_DELIVERY_READY_GRACE_S` | `api.extraEnv`. | Final-delivery retry and claim timing. | | `CENTAUR_ENABLE_GCLOUD_BOOTSTRAP`, `GCP_GCLOUD_CREDENTIAL`, `GCLOUD_PROJECT` | `api.extraEnv` or Secret. | Optional gcloud ADC bootstrap in the API container. | -| `CLAUDE_MODEL`, `CODEX_MODEL` | `api.extraEnv` or request model override. | Harness model selection defaults. | +| `CLAUDE_MODEL`, `CODEX_MODEL` | `api.extraEnv` or request model override. | Harness model selection defaults. When set via `sandbox.extraEnv`, the chart also mirrors them into slackbotv2 and the Console so their model displays track the deployment. | ## API-RS @@ -84,6 +86,24 @@ Optional required-by-mode variables: | `apiRs.metrics.scrapeAnnotations` | Helm value, default `true`. | Adds Prometheus scrape annotations to the API-RS Pod template and Service. | | `apiRs.metrics.path` | Helm value, default `/metrics`. | Metrics scrape path for annotation-based discovery. | | `apiRs.metrics.annotations` | Helm value. | Additional scrape annotations for Prometheus-compatible collectors. | +| `apiRs.activitySummary.*` | Helm values, default disabled. | Enables API-RS to summarize live session activity into durable `session.activity_summary` events. | +| `SLACK_BOT_TOKEN` | Explicit `secretKeyRef` from `secretManager.existingSecretName`. | Slack Web API access for api-rs Slack proxy and workflow Slack helpers. | +| `CENTAUR_CONTROL_API_KEY` | Required `secretKeyRef` from `secretManager.existingSecretName`. | Authorizes admin data routes, global sandbox drain, and trusted workflow/session control. Bot service keys remain session-API credentials but cannot authorize admin routes or global drain. | +| `SLACK_FEEDBACK_API_KEY` | Optional `secretKeyRef` from `secretManager.existingSecretName`. | Combined with the caller JWT, authorizes only principal-bound `feedback-improvement:*` sessions through `X-Centaur-Feedback-Key`; it cannot elevate repo access or operate admin routes. | +| `OPENAI_API_KEY` | Secret mounted into api-rs, or `apiRs.extraEnv` for local/dev overrides. | OpenAI credential for activity summaries; the feature stays disabled when no key is present. | +| `SESSION_ACTIVITY_SUMMARY_MODEL` | `apiRs.activitySummary.model`, default `gpt-5.4-nano`. | Model used for the short live activity sentence. | + +Sandbox lifecycle: + +| Env var or value | Set from | Controls | +| --- | --- | --- | +| `SESSION_IDLE_TIMEOUT_MS` | `slackbotv2.extraEnv`; default is up to 3 hours. | Slackbot v2 execute idle timeout. After an execution reaches a terminal state, api-rs pauses the sandbox if no newer execution has used that sandbox. If `SESSION_MAX_DURATION_MS` is lower than 3 hours and this value is unset, Slackbot v2 caps the default idle timeout to the max duration. | +| `SESSION_MAX_DURATION_MS` | `slackbotv2.extraEnv`. | Optional per-execution max duration forwarded to api-rs. api-rs rejects requests where `idle_timeout_ms` is greater than `max_duration_ms`. | +| `apiRs.sandboxMaxLifetimeSecs` / `SESSION_SANDBOX_MAX_LIFETIME_SECS` | Helm value, default `259200` (72 hours). | Restart-surviving sandbox deletion backstop. The reaper stops any non-terminal sandbox older than this, regardless of whether it is running or suspended. Set `0` to disable max-lifetime reaping. | +| `apiRs.sandboxReapIntervalSecs` / `SESSION_SANDBOX_REAP_INTERVAL_SECS` | Helm value, default `300`. | How often api-rs sweeps observed sandboxes for max-lifetime expiry. | + +There is no separate suspended-only delete timer. Pausing is controlled by the +per-execution idle timeout; deletion is controlled by sandbox max lifetime. Execution tuning: @@ -110,7 +130,6 @@ Execution tuning: | `SLACK_API_URL` | `slackbot.extraEnv`. | Optional Slack Web API base URL override. | | `CENTAUR_API_URL` | Chart-rendered API service URL. | API base URL used by Slackbot. | | `CENTAUR_SLACK_EVENTS_PATH` | `slackbot.extraEnv`. | Slack Events API route; defaults to `/api/webhooks/slack`. | -| `SLACKBOT_AMBIENT_CHANNEL_IDS` | `slackbotv2.extraEnv`. | Comma/space-separated Slack channel ids where messages start sessions without a bot mention. | | `RUNTIME_ERROR_ALERT_CHANNEL` | `slackbot.runtimeErrorAlertChannel`. | Slack channel for runtime error alerts. | | `SLACK_EVENT_DEDUP_TTL_MS` | `slackbot.extraEnv`. | Slack event dedupe window. | | `SLACK_SIGNATURE_MAX_AGE_SECONDS` | `slackbot.extraEnv`. | Maximum accepted Slack signature age. | @@ -164,7 +183,8 @@ Kubernetes backend: | `KUBERNETES_SANDBOX_RUNTIME_CLASS_NAME`, `KUBERNETES_SANDBOX_SERVICE_ACCOUNT_NAME` | `sandbox.runtimeClassName`, `api.extraEnv`. | Pod runtime class and service account. | | `KUBERNETES_SANDBOX_CPU_LIMIT`, `KUBERNETES_SANDBOX_MEMORY_LIMIT`, `KUBERNETES_SANDBOX_CPU_REQUEST`, `KUBERNETES_SANDBOX_MEMORY_REQUEST` | `sandbox.resources.*`. | Sandbox pod resources. | | `KUBERNETES_SANDBOX_READY_TIMEOUT_S`, `KUBERNETES_ATTACH_LOG_TAIL_LINES` | `api.extraEnv`. | Sandbox readiness and attach diagnostics. | -| `SESSION_SANDBOX_CLEANUP_INTERVAL_SECS`, `SESSION_SANDBOX_IDLE_CLEANUP_BACKSTOP_SECS` | `apiRs.sandboxCleanupIntervalSecs`, `apiRs.sandboxIdleCleanupBackstopSecs`. | DB-aware cleanup of unreferenced sandboxes and idle-pause backstop after API restarts. | +| `SESSION_SANDBOX_RUNNING_LIMIT`, `SESSION_SANDBOX_HOT_IDLE_GRACE_SECS` | `apiRs.sandboxRunningLimit`, `apiRs.sandboxHotIdleGraceSecs`. | Capacity admission for running-like sandboxes; discards ready warm sandboxes first, then pauses least-recently-active idle sessions outside the grace window. | +| `SESSION_SANDBOX_CLEANUP_INTERVAL_SECS`, `SESSION_SANDBOX_IDLE_CLEANUP_BACKSTOP_SECS` | `apiRs.sandboxCleanupIntervalSecs`, `apiRs.sandboxIdleCleanupBackstopSecs`. | DB-aware cleanup of unreferenced sandboxes and restart recovery for idle pauses. Persisted `idle_timeout_ms` is honored after restart; the backstop is the fallback for older execution rows without that metadata. | | `KUBERNETES_SANDBOX_EXTRA_ENV` | `sandbox.extraEnv`. | JSON list copied into each sandbox. | | `KUBERNETES_WORKFLOW_DIRS` | Chart-rendered from `overlays.sources[*].workflowsSubdir` (default `workflows`) using the sandbox repo-cache mount prefix. | Workflow-host sandbox discovery paths. | | `KUBERNETES_FIREWALL_CA_SECRET_NAME`, `KUBERNETES_FIREWALL_CA_KEY_SECRET_NAME` | `firewall.existingCa*` or generated CA Secrets. | CA material for sandbox/proxy TLS interception. | @@ -182,14 +202,15 @@ Sandbox entrypoint and wrappers: | Env var | Set from | Controls | | --- | --- | --- | -| `CENTAUR_HARNESS_CONFIG_DIR`, `CENTAUR_HARNESS_ADAPTER` | Sandbox image or `sandbox.extraEnv`. | Harness config directory and optional adapter executable. | +| `CENTAUR_HARNESS_CONFIG_DIR`, `CENTAUR_HARNESS_ADAPTER` | Sandbox image or `sandbox.extraEnv`. | Authoritative harness config directory and optional adapter executable. The entrypoint copies this directory's Codex and Claude files instead of merging them with the image-baked `~/harness` defaults, so an override must carry every required provider, feature, and trust setting. Leave it unset to use the reviewed config packaged in the sandbox image. | | `CENTAUR_SKILL_DIRS` | Chart-rendered from `overlays.sources[*].skillsSubdir` (default `.agents/skills`) through `SESSION_SANDBOX_EXTRA_ENV`. | Ordered skill directories copied into the agent workspace. | | `AGENT_REPO`, `AGENT_PERSONA` | Runtime assignment metadata. | Workspace repo clone and persona prompt. | | `GOOGLE_APPLICATION_CREDENTIALS` | Sandbox entrypoint or `sandbox.extraEnv`. | Google ADC path; entrypoint creates a local stub when unset. | | `CODEX_API_KEY`, `CODEX_HOME`, `CODEX_CONTINUE_THREAD_ID` | `sandbox.extraEnv` or runtime resume. | Codex auth/config/resume behavior. | | `CODEX_AUTH_MODE` | `sandbox.extraEnv`. | Codex auth flow: `api_key` (default, hits `api.openai.com`) or `access_token` (hits `chatgpt.com` via the brokered ChatGPT login). See [Codex Auth Modes](/deploying-in-production#codex-auth-modes). | +| `META_AI_API_KEY` | Secret mounted into api-rs. | Meta AI direct credential for Codex provider `responses` and Slack or Linear `--meta` selection. | | `CODEX_MODEL_REASONING_SUMMARY` | `sandbox.extraEnv`. | Sets `model_reasoning_summary` in the Codex config (`auto`, `concise`, `detailed`, `none`). Codex >= 0.139 emits no reasoning summaries unless this is set, so renderers show no thinking trace. | -| `CODEX_MODEL_REASONING_EFFORT` | `sandbox.extraEnv`. | Overrides the codex `model_reasoning_effort` (baked into `harness/codex/config.toml`) by patching the per-sandbox `~/.codex/config.toml` at boot, without forking the image. One of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`; an unknown value is ignored (the config default stands). | +| `CODEX_MODEL_REASONING_EFFORT` | `sandbox.extraEnv`. | Overrides the codex `model_reasoning_effort` (baked into `harness/codex/config.toml`) by patching the per-sandbox `~/.codex/config.toml` at boot, without forking the image. One of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`; an unknown value is ignored (the config default stands). | | `CLAUDE_MODEL`, `CLAUDE_CONTINUE_SESSION_ID` | `sandbox.extraEnv` or runtime resume. | Claude model and resume behavior. | | `CLAUDE_CODE_AUTH_MODE` | `sandbox.extraEnv`. | Claude Code auth flow: `api_key` (default, uses `ANTHROPIC_API_KEY`) or `access_token` (Claude.ai Pro or Max via the brokered OAuth login). See [Claude Auth Modes](/deploying-in-production#claude-auth-modes). | | `DEPLOY_ENV`, `ENVIRONMENT`, `TRACEPARENT` | Deployment env or wrapper-generated. | Runtime environment and trace context. | @@ -210,23 +231,31 @@ Slack ETL workflows: | Env var | Set from | Controls | | --- | --- | --- | -| `SLACK_ETL_ENABLED` | `api.slackEtlEnabled`. | Master switch for Slack sync/backfill/context schedules. | -| `SLACK_SYNC_INTERVAL_SECONDS`, `SLACK_BACKFILL_INTERVAL_SECONDS`, `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `api.*IntervalSeconds`. | Slack ETL schedule intervals. | -| `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS`, `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `api.slackSync*LookbackDays`. | Slack history/thread lookback windows. | -| `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | `api.slackEtlExcludedChannelPatterns`. | Comma-separated channel-name globs to skip. | -| `SLACK_BACKFILL_ENABLED`, `SLACK_BACKFILL_CHANNEL_BATCH_LIMIT`, `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `api.extraEnv` or chart batch limit. | Backfill enablement and batch sizing. | -| `SLACK_RETENTION_INTERVAL_MINUTES`, `SLACK_ETL_RETENTION_DAYS`, `SLACK_DM_RETENTION_DAYS` | `api.extraEnv`. | Slack retention cadence and separate public ETL/DM TTLs. | -| `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `api.extraEnv`. | Enables company-context projection when Slack ETL is on. | +| `SLACK_ETL_ENABLED` | `apiRs.etl.slack.enabled`. | Master switch for Slack sync/backfill/context schedules. | +| `SLACK_SYNC_INTERVAL_SECONDS`, `SLACK_BACKFILL_INTERVAL_SECONDS`, `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `apiRs.etl.slack.syncIntervalSeconds`, `apiRs.etl.slack.backfill.intervalSeconds`, `apiRs.etl.companyContextDocuments.intervalSeconds`. | Slack ETL schedule intervals. | +| `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS`, `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `apiRs.etl.slack.syncBackfillLookbackDays`, `apiRs.etl.slack.syncThreadLookbackDays`. | Slack history/thread lookback windows. | +| `SLACK_SYNC_INDEX_PRIVATE_CHANNELS` | `apiRs.etl.slack.indexPrivateChannels`. | Includes private channels visible to the ETL token. | +| `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | `apiRs.etl.slack.excludedChannelPatterns`. | Comma-separated channel-name globs to skip. | +| `SLACK_BACKFILL_ENABLED`, `SLACK_BACKFILL_CHANNEL_BATCH_LIMIT`, `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `apiRs.etl.slack.backfill.*`. | Backfill enablement and batch sizing. | +| `SLACK_RETENTION_ENABLED`, `SLACK_RETENTION_INTERVAL_MINUTES`, `SLACK_ETL_RETENTION_DAYS`, `SLACK_DM_RETENTION_DAYS` | `apiRs.etl.slack.retention.*`. | Slack retention enablement, cadence, and separate public ETL/DM TTLs. | +| `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `apiRs.etl.companyContextDocuments.enabled`. | Enables company-context projection when any ETL is on. | +| `COMPANY_CONTEXT_DOCUMENTS_MAX_WINDOW_SECONDS` | `apiRs.etl.companyContextDocuments.maxWindowSeconds`. | Maximum source `updated_at` window projected by one company-context documents run. | Google Workspace ETL workflows: | Env var | Set from | Controls | | --- | --- | --- | -| `GOOGLE_DRIVE_ETL_ENABLED` | `api.googleDriveEtlEnabled`. | Enables Google Drive Docs sync. | -| `GOOGLE_DRIVE_ETL_FOLDER_IDS` | `api.extraEnv`. | Comma- or whitespace-separated Drive folder IDs/URLs to recursively full-scan for Google Docs. | -| `GOOGLE_DRIVE_SYNC_INTERVAL_SECONDS` | `api.googleDriveSyncIntervalSeconds`. | Google Drive Docs sync schedule interval. | -| `GOOGLE_CALENDAR_ETL_ENABLED` | `api.googleCalendarEtlEnabled`. | Enables Google Calendar sync. | -| `GOOGLE_CALENDAR_SYNC_INTERVAL_SECONDS` | `api.googleCalendarSyncIntervalSeconds`. | Google Calendar sync schedule interval. | +| `GOOGLE_DRIVE_ETL_ENABLED` | `apiRs.etl.googleDrive.enabled`. | Enables Google Drive Docs sync. | +| `GOOGLE_DRIVE_SYNC_INTERVAL_SECONDS` | `apiRs.etl.googleDrive.syncIntervalSeconds`. | Google Drive Docs sync schedule interval. | +| `GOOGLE_CALENDAR_ETL_ENABLED` | `apiRs.etl.googleCalendar.enabled`. | Enables Google Calendar sync. | +| `GOOGLE_CALENDAR_SYNC_INTERVAL_SECONDS` | `apiRs.etl.googleCalendar.syncIntervalSeconds`. | Google Calendar sync schedule interval. | + +Linear ETL workflows: + +| Env var | Set from | Controls | +| --- | --- | --- | +| `LINEAR_ETL_ENABLED` | `apiRs.etl.linear.enabled`. | Enables Linear project/issue/comment sync. | +| `LINEAR_SYNC_INTERVAL_SECONDS` | `apiRs.etl.linear.syncIntervalSeconds`. | Linear sync schedule interval. | ## Observability and Retention diff --git a/docs/pages/reference/tool-directory.mdx b/docs/pages/reference/tool-directory.mdx index 903d5dffb..17322fa7f 100644 --- a/docs/pages/reference/tool-directory.mdx +++ b/docs/pages/reference/tool-directory.mdx @@ -31,10 +31,10 @@ These are broadly useful across most deployments and are good candidates to conf |---|---|---| | `linear` | Search, create, update, and comment on Linear issues, projects, cycles, teams, and labels | `LINEAR_API_KEY` | | `notion` | Search and update Notion pages, databases, blocks, and comments | `NOTION_API_KEY` | -| `slack` | Search Slack, read threads, inspect channels/users, and send or upload messages | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_UPLOAD_TOKEN`, `SLACK_ETL_TOKEN` | +| `slack` | Search Slack, read threads, inspect channels/users, and send or upload messages | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_ETL_TOKEN` | | `gsuite` | Use Gmail, Calendar, Drive, Docs, Sheets, Slides, and Google Analytics | `GOOGLE_TOKEN_JSON` | | `websearch` | Free web search via Parallel and deep research | None; `PARALLEL_API_KEY` for `deep_research`; `ANTHROPIC_API_KEY` for search synthesis | -| `company_context` | Search indexed company history across internal sources | None | +| `company_context` | Search indexed company history, Slack DMs, and Google Docs | None | | `grafana` | Query dashboards, alerts, VictoriaMetrics, VictoriaLogs, and annotations | `GRAFANA_URL`, `GRAFANA_API_KEY` | | `posthog` | Query product analytics, events, pageviews, breakdowns, and user agents | `POSTHOG_API_KEY`, `POSTHOG_PROJECT_ID` | | `attio` | Work with CRM objects, records, lists, notes, tasks, calls, and meetings | `ATTIO_API_KEY` | @@ -74,7 +74,7 @@ These are broadly useful across most deployments and are good candidates to conf | Tool | Use | API key / credential | |---|---|---| | `airtable` | Bases, schemas, tables, records, views, and URL parsing | `AIRTABLE_API_KEY` | -| `company_context` | Search indexed company history across internal sources | None | +| `company_context` | Search indexed company history, Slack DMs, and Google Docs | None | | `composio` | Execute tools from third-party services exposed through Composio | `COMPOSIO_API_KEY` | | `figma` | Extract Figma files, nodes, components, styles, and variables | `FIGMA_ACCESS_TOKEN` | | `granola` | Search and read Granola notes and transcripts | `GRANOLA_API_KEY` | @@ -82,7 +82,7 @@ These are broadly useful across most deployments and are good candidates to conf | `linear` | Linear issues, projects, cycles, teams, workflow states, and labels | `LINEAR_API_KEY` | | `notion` | Notion pages, databases, blocks, comments, and users | `NOTION_API_KEY` | | `opentable` | Search OpenTable restaurant reservations | None | -| `slack` | Slack messages, files, channels, threads, users, and usergroups | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_UPLOAD_TOKEN`, `SLACK_ETL_TOKEN` | +| `slack` | Slack messages, files, channels, threads, users, and usergroups | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_ETL_TOKEN` | ## Research @@ -136,7 +136,7 @@ These tools ship in the base repo because many Centaur users need onchain or mar | `kalshi` | Prediction market events, markets, trades, and candlesticks | None | | `karma` | DAO delegate reputation, activity, scores, and governance analytics | None | | `messari` | Crypto asset prices, metrics, profiles, markets, news, and timeseries | `MESSARI_API_KEY` | -| `mpp` | Paid market-data and web-search requests through Machine Payments Protocol | None | +| `mpp` | Paid MPP requests | None | | `nansen` | Wallet labels, smart-money activity, token flows, holders, and PnL | `NANSEN_API_KEY` | | `polymarket` | Prediction market events, markets, prices, books, and trades | None | | `snapshot` | Offchain governance spaces, proposals, votes, and voting power | `SNAPSHOT_API_KEY` | diff --git a/docs/pages/secrets/environment.mdx b/docs/pages/secrets/environment.mdx index 19b871f85..f24c158ac 100644 --- a/docs/pages/secrets/environment.mdx +++ b/docs/pages/secrets/environment.mdx @@ -31,8 +31,9 @@ kubectl create secret generic centaur-infra-env \ --namespace centaur-system \ --from-literal=DATABASE_URL='postgres://...' \ --from-literal=SLACKBOT_API_KEY='...' \ + --from-literal=CENTAUR_CONTROL_API_KEY="$(openssl rand -hex 32)" \ + --from-literal=SLACK_FEEDBACK_API_KEY="$(openssl rand -hex 32)" \ --from-literal=SLACK_BOT_TOKEN='xoxb-...' \ - --from-literal=SLACK_UPLOAD_TOKEN='xoxp-...' \ --from-literal=SLACK_SIGNING_SECRET='...' \ --from-literal=SANDBOX_SIGNING_KEY="$(openssl rand -hex 32)" \ --from-literal=IRON_MANAGEMENT_API_KEY="$(openssl rand -hex 32)" \ @@ -42,6 +43,17 @@ kubectl create secret generic centaur-infra-env \ --from-literal=WAREHOUSE_API_KEY='...' ``` +`CENTAUR_CONTROL_API_KEY` is a required control-plane credential. Use it only +from Console and operator tooling; do not place it in a sandbox. The optional +`SLACK_FEEDBACK_API_KEY` has a separate, narrow feedback-session scope and must +not reuse the control key. + +When upgrading an existing release, add the prefixed control key (for example, +`PREFIX_CENTAUR_CONTROL_API_KEY` when `envPrefix: PREFIX_`) before Helm/Argo +applies the new workloads. The API and Console references are non-optional, so +pods will not start until that Secret key exists. `just bootstrap-secrets` +tops up existing local-development Secrets without rotating an existing key. + For local development, `just bootstrap-secrets` creates the local Kubernetes Secret from your shell environment. diff --git a/docs/pages/secrets/oauth-apps.mdx b/docs/pages/secrets/oauth-apps.mdx index bdde378ff..3422b3676 100644 --- a/docs/pages/secrets/oauth-apps.mdx +++ b/docs/pages/secrets/oauth-apps.mdx @@ -7,203 +7,70 @@ description: Register OAuth clients, collect user consent, and grant refreshed a OAuth apps let users connect their own upstream accounts to Centaur. An operator registers an OAuth client in the console, shares a consent link, and each user -who completes the flow creates or updates a managed broker credential. - -The broker credential owns refresh-token lifecycle. It refreshes access tokens -inside the Centaur Console and exposes only the current access token to iron-proxy -through a `token_broker` secret source. The user's refresh token never leaves -the Centaur Console. - -OAuth apps are separate from console login. Console SSO uses -`/auth//start` and signs operators into the console. OAuth apps use -`/oauth//start` and mint credentials for tools. +who completes the flow gets a managed credential. The Centaur Console keeps the +token fresh and iron-proxy injects it as `Authorization: Bearer ` +into requests to the provider's API hosts. Refresh tokens never leave the +Centaur Console. ## Supported Providers | Provider | Use | |----------|-----| -| `google` | Google API credentials, such as Gmail or Drive scopes. | -| `slack` | Slack user-token credentials with normal Slack API scopes. | - -Google flows request offline access and force consent so the token response -includes a refresh token. Slack OAuth apps should enable token rotation so the -callback also receives a refresh token. - -## Create The Provider App - -Create an OAuth client in the upstream provider first. - -Register this callback URL: - -```text -/oauth//callback -``` - -For example: - -```text -https://control.example.com/oauth/google-drive/callback -``` +| `google` | Google APIs, such as Gmail or Drive scopes. | +| `slack` | Slack user tokens with normal Slack API scopes. | +| `github` | GitHub user tokens for `api.github.com`. | +| `granola` | Granola MCP tokens for `mcp.granola.ai`. | +| `linear` | Linear tokens for `api.linear.app`. | +| `attio` | Attio workspace tokens for `api.attio.com`. | -The slug is the stable name users see in the consent URL. It must contain only -URL-safe characters. +## Set Up An App -For Slack, use normal Slack API scopes such as `channels:history` or -`users:read`. Do not use Sign in with Slack scopes such as `openid`, `email`, or -`profile` for OAuth apps. +1. **Create an OAuth client with the provider** (for example in the Google + Cloud console or the Attio developer dashboard). Register this callback + URL: `/oauth//callback`. +2. **Register it in Centaur.** In the console, open **OAuth Apps**, click + **Add App**, and fill in the slug, provider, client id, client + secret, and allowed scopes (one per line). +3. **Share the consent link** shown on the app page: + `/oauth//start`. Each user who opens it + and approves the provider's consent screen gets a credential, wrapped in a + grantable secret. -## Register The App In Centaur +Re-consenting with the same account updates the existing credential instead of +creating another one. -In the console, open **OAuth Apps**, then create an app with: +## Provider-Specific Setup -| Field | Meaning | -|-------|---------| -| `Slug` | Globally unique consent-link name, for example `google-drive`. | -| `Provider` | `google` or `slack`. | -| `Client ID` | OAuth client id from the provider. | -| `Client Secret` | OAuth client secret from the provider. Stored encrypted. | -| `Credential Namespace` | Namespace for broker credentials minted by this app. | -| `Allowed Scopes` | One scope per line. Consent requests must be a subset. | -| `Enabled` | Disabled apps reject new consent flows. Existing credentials keep refreshing. | +### Granola -You can also create the app through the API: +Granola has no app dashboard; obtain the OAuth client once via dynamic client +registration, then use the returned `client_id` and `client_secret` when adding +the app in the console: ```bash -curl -sS -X POST "$IRON_CONTROL_URL/api/v1/oauth_apps" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" \ +curl -sS -X POST https://mcp-auth.granola.ai/oauth2/register \ -H "Content-Type: application/json" \ -d '{ - "data": { - "slug": "google-drive", - "description": "Google Drive user access", - "provider": "google", - "client_id": "client-id.apps.googleusercontent.com", - "client_secret": "client-secret", - "credential_namespace": "default", - "allowed_scopes": [ - "https://www.googleapis.com/auth/drive.metadata.readonly" - ], - "enabled": true, - "labels": { "team": "platform" } - } + "client_name": "Centaur Console", + "redirect_uris": ["/oauth/granola/callback"], + "grant_types": ["authorization_code", "refresh_token"], + "response_types": ["code"], + "token_endpoint_auth_method": "client_secret_post", + "scope": "openid email profile offline_access mcp" }' ``` -`client_secret` is write-only. API responses never include it. Updating an app -without a new `client_secret` keeps the stored value. - -## Collect User Consent - -Share the app start URL with the user: - -```text -/oauth//start -``` - -Omitting `scopes` requests every allowed scope: - -```text -https://control.example.com/oauth/google-drive/start -``` - -To request a subset, pass scopes as a space-separated or comma-separated query -parameter: - -```text -https://control.example.com/oauth/google-drive/start?scopes=https://www.googleapis.com/auth/drive.metadata.readonly -``` - -The start endpoint rejects unknown slugs, disabled apps, and scopes outside the -app allowlist. After provider consent, the callback exchanges the code, records -the provider account identity, and renders a console result page. - -Re-consenting with the same app and provider account updates the existing broker -credential instead of creating another one. - -## What Gets Created - -A successful consent creates or updates: - -| Resource | Purpose | -|----------|---------| -| Broker credential | Stores provider identity, scopes, current access token, refresh token, expiry, and refresh state. | -| Static secret | Grantable wrapper that injects `Authorization: Bearer `. | - -The static secret uses a `token_broker` source that points at the broker -credential. At proxy sync time, the Centaur Console resolves the broker credential and -sends the current access token to iron-proxy. If the credential is still -bootstrapping or cannot refresh, the secret is omitted from proxy config until -it recovers. +Use `mcp` as the allowed scope for the app. -The auto-created request rules are provider-scoped: +## Grant The Credential -| Provider | Default API host rules | -|----------|------------------------| -| Google | `*.googleapis.com` | -| Slack | `slack.com` | +Consent does not automatically grant the token to every session. In the +console, open **Principals**, choose the user or channel, and use **Direct +Grants** to select the secret created for the credential — or grant it to a +reusable role. -Operators can tighten the static secret's rules in the console if a credential -should only be valid for specific API paths. - -## Grant The OAuth Credential - -OAuth consent does not automatically grant the token to every session. Grant the -auto-created static secret to the correct user, channel, or role. - -You can grant the secret in the Centaur Console. Open **Principals**, choose the -user or channel principal, then use **Direct Grants** to select the static secret -created for the broker credential. The same principal page can assign a role if -you grant the OAuth secret to a reusable role instead. - -For scripted changes, list secrets in the credential namespace and find the -static secret created for the broker credential: - -```bash -curl -sS "$IRON_CONTROL_URL/api/v1/static_secrets?namespace=default" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" | jq -``` - -Then grant the secret with `centaur-perms`: - -```bash -cd services/api-rs -cargo run -p centaur-perms -- \ - principals grant slack-user-u123 \ - --secret ssr_... -``` - -Grant the same credential to a channel when the channel should define access: - -```bash -cargo run -p centaur-perms -- \ - principals grant slack-channel-c456 \ - --secret ssr_... -``` - -Or grant it to a reusable role: - -```bash -cargo run -p centaur-perms -- \ - roles grant tool-google-drive \ - --secret ssr_... -``` - -## Rotate Or Disable - -Rotating the OAuth client's secret on the app updates every credential minted by -that app because minted broker credentials delegate `client_id` and -`client_secret` back to the app. - -Disable an app to stop new consent flows: - -```bash -curl -sS -X PATCH "$IRON_CONTROL_URL/api/v1/oauth_apps/google-drive" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" \ - -H "Content-Type: application/json" \ - -d '{ "data": { "enabled": false } }' -``` +## Disable Or Remove -Existing broker credentials keep refreshing while the app exists. To fully -remove access, revoke grants to the wrapper static secret, delete the wrapper -secret, then delete or unlink the broker credential. An app cannot be deleted -while minted credentials still reference it. +Toggle **Enabled** off on the app page to stop new consent flows; existing +credentials keep working. To fully remove access, revoke grants to the wrapper +secret, delete it, then delete the credential. diff --git a/docs/pages/secrets/onepassword.mdx b/docs/pages/secrets/onepassword.mdx index 87542b6fa..9f95cdf9c 100644 --- a/docs/pages/secrets/onepassword.mdx +++ b/docs/pages/secrets/onepassword.mdx @@ -72,8 +72,8 @@ It must also include infrastructure secrets such as: ```text DATABASE_URL SLACKBOT_API_KEY +CENTAUR_CONTROL_API_KEY SLACK_BOT_TOKEN -SLACK_UPLOAD_TOKEN SLACK_SIGNING_SECRET SANDBOX_SIGNING_KEY IRON_MANAGEMENT_API_KEY @@ -81,6 +81,17 @@ IRON_MANAGEMENT_API_KEY Those are boot-time service secrets, not tool credentials. +`CENTAUR_CONTROL_API_KEY` is required and must be a dedicated high-entropy +control-plane value. It does not belong in the 1Password tool-credential vault +or any sandbox. `SLACK_FEEDBACK_API_KEY` is optional but, when the feedback tool +is enabled, must be a different value because it receives only the +`feedback-improvement:*` session capability. + +For an existing deployment, provision the prefixed control key in +`secretManager.existingSecretName` before upgrading. The API and Console +Secret references are non-optional; adding the value to a 1Password item alone +does not satisfy this boot-time Kubernetes Secret requirement. + ## Name 1Password items For the normal tool declaration: @@ -112,6 +123,7 @@ Store enabled harness credentials the same way: |------------|----------| | `OPENAI_API_KEY` | Codex default | | `OPENROUTER_API_KEY` | OpenRouter via Codex | +| `META_AI_API_KEY` | Meta AI direct via Codex | | `AMP_API_KEY` | Amp | | `ANTHROPIC_API_KEY` | Claude Code and pi-mono | diff --git a/docs/public/md/architecture.md b/docs/public/md/architecture.md index 53a44e0ea..eba179f1d 100644 --- a/docs/public/md/architecture.md +++ b/docs/public/md/architecture.md @@ -52,9 +52,17 @@ https://api.acme.com/api/webhooks/slack The webhook does not use a Centaur API key. Slack signs every request with `X-Slack-Signature` and `X-Slack-Request-Timestamp`; the Slackbot validates that HMAC signature with `SLACK_SIGNING_SECRET` before it routes the event to the API. -After validation, the Slackbot calls Centaur's agent API with +After validation, the Slackbot calls Centaur's api-rs session API with `SLACKBOT_API_KEY`. +Ingress bot keys are session-API service credentials; they are not global +operator credentials or per-channel authorization. Administrative data routes, workflow administration, +and fleet-wide operations such as sandbox drain require the distinct +`CENTAUR_CONTROL_API_KEY`. Sandbox tools never receive that key. The optional +feedback tool instead combines `SLACK_FEEDBACK_API_KEY` with its Console JWT; +the API binds each `feedback-improvement:*` session to that caller's existing +principal and capabilities. + During a Slack delivery, the API owns the execution state while Slackbot owns Slack rendering: opening or updating the thread UI, streaming chunks, rendering steps, and posting the final answer. The landing page preview shows that Slack diff --git a/docs/public/md/deploying-in-production.md b/docs/public/md/deploying-in-production.md index ead9ffbe0..c0c6d26d6 100644 --- a/docs/public/md/deploying-in-production.md +++ b/docs/public/md/deploying-in-production.md @@ -17,7 +17,7 @@ creates sandbox pods for agent work. [iron-proxy](https://docs.iron.sh) handles need credentials:
- Centaur production workflow — Centaur API plus Postgres hands a run to the Kubernetes backend, which attaches a sandbox pod whose outbound HTTP routes through iron-proxy + Centaur production workflow: Centaur API plus Postgres hands a run to the Kubernetes backend, which attaches a sandbox pod whose outbound HTTP routes through iron-proxy
Slackbot and API ingress → Centaur API (Postgres-backed) → Kubernetes sandbox runtime → outbound traffic through iron-proxy.
@@ -62,17 +62,25 @@ Minimum keys: | `DATABASE_URL` | API | Postgres connection string. | | `IRON_MANAGEMENT_API_KEY` | [iron-proxy](https://docs.iron.sh) management API | Generate with `openssl rand -hex 32`. | | `SANDBOX_SIGNING_KEY` | Sandbox API tokens | Generate with `openssl rand -hex 32`; keeps sandbox tokens valid across API restarts. | -| `SLACK_BOT_TOKEN` | Slackbot | Bot User OAuth Token from the Slack app. | -| `SLACK_UPLOAD_TOKEN` | Slack file uploads | Dedicated Slack token for tool-driven file upload and verification. | +| `SLACK_BOT_TOKEN` | Slackbot/API | Bot User OAuth Token from the Slack app. | | `SLACK_SIGNING_SECRET` | Slackbot/API | Used to verify Slack webhook signatures. | | `SLACKBOT_API_KEY` | Slackbot to API | Static service token; API bootstraps it into Postgres on startup with `agent` scope. | +| `CENTAUR_CONTROL_API_KEY` | Console/operator to API | Dedicated high-entropy service token for administrative routes and global controls. Never reuse a bot key or expose this value to sandboxes. | +| `SLACK_FEEDBACK_API_KEY` | Optional sandbox feedback tool to API | Separate narrow token that can create and operate only `feedback-improvement:*` sessions. | | `OP_CONNECT_TOKEN` | [iron-proxy](https://docs.iron.sh) 1Password Connect source (preferred) | Needed when `ironProxy.secretSource` is `onepassword-connect`. | | `OP_SERVICE_ACCOUNT_TOKEN` | [iron-proxy](https://docs.iron.sh) 1Password service-account source | Needed when `ironProxy.secretSource` is `onepassword`. | | `OP_VAULT` | [iron-proxy](https://docs.iron.sh) 1Password source | Vault name or id used for `op://` references (either mode). | -`SLACKBOT_API_KEY` is not created with the admin API during initial boot, because -the API process requires it before it can start. Generate a high-entropy value, -store it in the infra Secret, and reuse the same value in Slackbot. +`SLACKBOT_API_KEY` and `CENTAUR_CONTROL_API_KEY` are not created with the admin +API during initial boot, because the API process requires them before it can +start. Generate distinct high-entropy values, store them in the infra Secret, +and reuse only the Slackbot value in Slackbot. Local bootstrap generates and +preserves the control and feedback keys automatically. + +For upgrades using `secretManager.existingSecretName`, create the prefixed +`CENTAUR_CONTROL_API_KEY` Secret entry before applying the new chart. The API +and Console references are non-optional and otherwise produce a pod startup +failure. This secret prerequisite is the first gate in the upgrade runbook. ## 3. Configure harness credentials @@ -82,6 +90,7 @@ Store one secret per enabled harness credential: |---------|-----------|----------------|---------------------|----------| | Codex default | `codex` | none or `--codex` | `OPENAI_API_KEY` | `api.openai.com` | | Codex with OpenRouter provider | `codex` | none or `--codex` | `OPENROUTER_API_KEY` | `openrouter.ai` | +| Codex with Meta AI direct | `codex` | `--meta` | `META_AI_API_KEY` | `api.ai.meta.com` | | Amp | `amp` | `--amp` | `AMP_API_KEY` | `ampcode.com` | | Claude Code | `claude-code` | `--claude` | `ANTHROPIC_API_KEY` | `api.anthropic.com` | | pi-mono | `pi-mono` | `--pi` | `ANTHROPIC_API_KEY` | `api.anthropic.com` | @@ -100,6 +109,10 @@ alongside `CODEX_MODEL`. Per-turn Codex model overrides with provider-style slugs such as `--model anthropic/claude-fable-5` also select the OpenRouter provider even when `OPENROUTER_MODEL` is unset. +To run Codex through Meta AI direct, store `META_AI_API_KEY` and select the +provider with `--meta`. Pair it with `--model ` when choosing a +provider-specific model for a turn. + Whatever source you pick, the vault is shared across the whole deployment, so any thread can use any configured credential. Per-user and per-channel scoping is on the roadmap; until then, scope tool and harness access @@ -204,10 +217,9 @@ Use the app page to install the bot, copy the Bot User OAuth Token for 1. Add the bot scopes required by the Slackbot features you enable. 2. Install the app to the workspace. 3. Store the Bot User OAuth Token as `SLACK_BOT_TOKEN`. -4. Store a dedicated upload-capable Slack token as `SLACK_UPLOAD_TOKEN` if Slack file uploads are enabled. -5. Store the app Signing Secret as `SLACK_SIGNING_SECRET`. -6. Enable Event Subscriptions. -7. Set the Request URL to `https:///api/webhooks/slack`. +4. Store the app Signing Secret as `SLACK_SIGNING_SECRET`. +5. Enable Event Subscriptions. +6. Set the Request URL to `https:///api/webhooks/slack`. 7. Subscribe to `app_mention` and to the message events you want Centaur to see: `message.channels`, `message.groups`, and `message.im`. @@ -240,6 +252,10 @@ ironProxy: secretSource: onepassword-connect secretTtl: 10m +apiRs: + # Delete any sandbox older than this, running or suspended. + sandboxMaxLifetimeSecs: 259200 + onepasswordConnect: connect: create: true @@ -257,6 +273,17 @@ sandbox: The Kubernetes sandbox backend is the active runtime backend; there is no chart switch named `api.sandboxBackend`. +Sandbox lifecycle has two separate timers: + +- Slackbot v2 sends `idle_timeout_ms` on execute requests, defaulting to up to + 3 hours, so api-rs can pause an idle sandbox after a turn finishes. +- api-rs deletes old sandboxes through `apiRs.sandboxMaxLifetimeSecs`, default + 72 hours, regardless of whether the sandbox is still running or already + suspended. + +There is no suspended-only delete setting. If you want sandboxes gone after N +hours, set `apiRs.sandboxMaxLifetimeSecs` to N hours in seconds. + Install or upgrade: ```bash @@ -285,21 +312,28 @@ Run one agent turn from inside the api-rs deployment: THREAD_KEY=cli:production-smoke-codex THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') -SESSION=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}" \ - -H "Content-Type: application/json" \ - -d '{"harness_type":"codex","on_harness_conflict":"restart"}') - -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/messages" \ - -H "Content-Type: application/json" \ - -d '{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG."}]}]}' +SESSION=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"harness_type":"codex","on_harness_conflict":"restart"}'\''' sh "$THREAD_PATH") -EXECUTE=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -X POST "http://localhost:8080/api/session/${THREAD_PATH}/execute" \ - -H "Content-Type: application/json" \ - -d '{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG.\"}]}}"]}') +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1/messages" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"messages":[{"role":"user","parts":[{"type":"text","text":"Reply with exactly PONG."}]}]}'\''' sh "$THREAD_PATH" + +EXECUTE=$(kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -X POST "http://localhost:8080/api/session/$1/execute" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" \ + -d '\''{"input_lines":["{\"type\":\"user\",\"message\":{\"content\":[{\"type\":\"text\",\"text\":\"Reply with exactly PONG.\"}]}}"]}'\''' sh "$THREAD_PATH") EXECUTION_ID=$(printf '%s' "$EXECUTE" | jq -r '.execution_id') -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s -N \ - "http://localhost:8080/api/session/${THREAD_PATH}/events?execution_id=${EXECUTION_ID}&after_event_id=0" +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s -N "http://localhost:8080/api/session/$1/events?execution_id=$2&after_event_id=0" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}"' sh "$THREAD_PATH" "$EXECUTION_ID" ``` Then run the same prompt through Slack: @@ -322,8 +356,9 @@ If a run fails because the sandbox pod exits or is deleted, inspect the durable session and api-rs logs before retrying: ```bash -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/session/${THREAD_PATH}" | jq +kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- \ + sh -lc 'curl -s "http://localhost:8080/api/session/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}"' sh "$THREAD_PATH" | jq kubectl logs -n centaur-system deploy/centaur-centaur-api-rs --tail=200 kubectl get pods -n centaur-system -l centaur.ai/managed=true diff --git a/docs/public/md/extend/acme-example.md b/docs/public/md/extend/acme-example.md index 7309446be..632489a1e 100644 --- a/docs/public/md/extend/acme-example.md +++ b/docs/public/md/extend/acme-example.md @@ -87,8 +87,12 @@ git -C centaur-acme rev-parse --short HEAD The Centaur chart's repo-cache DaemonSet checks out the overlay repo on each node, so changing tools, workflows, or skills is a Git push — no API, sandbox, or overlay image rebuild is required for overlay-only changes. New sandboxes see -the latest cached checkout; existing sandboxes can run `centaur-tools refresh` -when they need to refresh tool shims from the current repo-cache checkout. +the latest cached checkout. Repo-cache-enabled running sandboxes auto-refresh +their local tool shims and copied skills from the latest cached checkout; use +`centaur-tools refresh` only when you need a manual refresh. This only updates +the runtime catalog and local source copy. Secret grants and proxy credentials +are reconciled separately, so a newly visible tool may still fail normally until +its credential path is available. Configure the ordered overlay sources in Helm values: @@ -220,11 +224,14 @@ Expected paths include: /home/agent/github/your-org/centaur-acme/.agents/skills ``` -You can also inspect the api-rs session context for a thread: +From a trusted operator shell, you can also inspect the api-rs session context +for a thread. (Inside a sandbox, iron-proxy supplies the narrower per-principal +JWT automatically; never copy the control key into a sandbox.) ```bash THREAD_PATH=$(jq -rn --arg v "$THREAD_KEY" '$v|@uri') -curl -s "$CENTAUR_API_URL/api/session/${THREAD_PATH}" | jq +curl -s "$CENTAUR_API_URL/api/session/${THREAD_PATH}" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?required}" | jq ``` ## What to change first diff --git a/docs/public/md/extend/apps.md b/docs/public/md/extend/apps.md index 4cc3095a8..c650b8e22 100644 --- a/docs/public/md/extend/apps.md +++ b/docs/public/md/extend/apps.md @@ -58,8 +58,8 @@ enabled = true [[tools]] name = "research-tool" description = "Search private research data" -scripts = [ - { name = "research-tool", command = "research-tool" }, +methods = [ + { name = "search", path = "/tools/research-tool/search" }, ] [[skills]] @@ -109,7 +109,7 @@ The app plane keeps the API as the registry, auth boundary, and router: | Logs | `GET /apps/{name}/logs` | | Restart | `POST /apps/{name}/restart` | | Delete | `DELETE /apps/{name}` | -| Tool capability | Registered as a sandbox-visible tool script or workflow-host bridge | +| Tool method | Existing `/tools/{tool}/{method}` route proxies to the app | | Skills | Listed through app skill discovery and fetched lazily | | Workflows | Started through the existing workflow run API | diff --git a/docs/public/md/extend/overlay.md b/docs/public/md/extend/overlay.md index c27044e52..33acb6912 100644 --- a/docs/public/md/extend/overlay.md +++ b/docs/public/md/extend/overlay.md @@ -48,9 +48,11 @@ overlays: sources: - repo: paradigmxyz/centaur ref: main + visibility: public - repo: your-org/centaur-overlay ref: main + visibility: private ``` `repo` is `owner/name` on GitHub. `ref` can be a branch, tag, or commit SHA; @@ -60,6 +62,11 @@ production rollout, but many overlay repos intentionally track `main` so a reviewed merge is enough for new sandboxes to pick up the change after repo-cache refreshes. +`visibility` controls which sandboxes may receive the repo-cache checkout. +It defaults to `private`. Set `visibility: public` only for repos whose full +contents are safe to expose to principals configured with +`sandbox_repo_cache=public`; invalid or missing values are treated as `private`. + Each source defaults to the conventional layout — `toolsSubdir: tools`, `workflowsSubdir: workflows`, `skillsSubdir: .agents/skills` — and directories a repo does not contain are skipped at runtime, so a skills-only overlay needs @@ -128,15 +135,10 @@ overlay: Add deployment-specific agent guidance here. ``` -When using a legacy overlay image, `overlay.mountPath` is the api-rs mount and -`overlay.sandboxMountPath` is the matching sandbox/workflow-host mount. Keep -them separate unless your deployment intentionally uses the same filesystem -layout in both containers. - -For larger prompt/persona sets, keep files in an overlay repo and expose tool, -workflow, and skill paths through `overlays.sources` where possible. Existing -deployments can continue using `overlay.image.*` while they migrate remaining -prompt, harness, and persona assets onto repo-cache-backed overlay sources. +For larger prompt/persona sets, keep files in an overlay repo and expose their +paths through `overlays.sources` as that surface is wired into your deployment. +Do not rely on `overlay.image.*`; repo-cache-backed overlays are the default +delivery path. ## Verify the overlay diff --git a/docs/public/md/extend/workflows-v2.md b/docs/public/md/extend/workflows-v2.md index 7f2d24171..610dfa06e 100644 --- a/docs/public/md/extend/workflows-v2.md +++ b/docs/public/md/extend/workflows-v2.md @@ -69,32 +69,28 @@ Supported v2 primitives: ### Keep imports narrow -Workflow files should import only the workflow context compatibility module: +Workflow files should import only the supported workflow-host API surface they +need: ```python from api.workflow_engine import WorkflowContext +from api.runtime_control import ControlPlaneError ``` -Do not import Python API internals such as: +Supported workflow-host modules are `api.workflow_engine`, +`api.runtime_control`, `api.app`, and `api.metrics`. -```python -from api.runtime_control import canonical_json -from api.vm_metrics import workflow_counter -``` - -Those modules were implementation details of the Python API service. In v2, -the workflow host provides a small compatibility surface instead of the whole -Python API package. +Do not import unrelated API-service internals or another workflow domain's local +helpers. Domain-specific helpers should live next to the workflows that own +them, for example `workflows/slack/metrics.py`. If a workflow needs a helper, move it into the workflow file, a shared overlay -module, or a supported workflow-host compatibility shim. +module, or a supported workflow-host API module. -### Make stepped side effects idempotent +### Put side effects behind steps -The handler may be replayed after a crash or retry. `ctx.step(...)` -checkpoints only after the callback returns. If the callback performs an -external write, use a stable provider-side idempotency key or upsert so replay -does not duplicate the side effect: +The handler may be replayed after a crash or retry. Any external write should +be wrapped in `ctx.step(...)` so the result is checkpointed: ```python async def handler(inp: dict, ctx: WorkflowContext) -> dict: @@ -122,6 +118,13 @@ The workflow host sandbox is separate from the agent sandbox. The workflow handler coordinates the run; the agent turn runs through the normal Centaur session runtime. +Production deployments should keep `WORKFLOW_HOST_SANDBOX=true` (the default). +`false` runs Python as a child process of api-rs for local development and is +not an isolation boundary. Centaur removes its known control, bot, feedback, +GitHub, Slack, and model-provider credentials from that child, but arbitrary +ambient process configuration can still be visible; use only trusted workflow +code in local mode. + ### Declare webhook metadata in the workflow Expose a workflow through `WEBHOOKS`: @@ -184,8 +187,8 @@ For each existing workflow: `api.workflow_engine.WorkflowContext`. 3. Confirm third-party Python packages are installed in the workflow-host sandbox image. -4. Make Slack posts, database writes, external HTTP calls, and tool calls - idempotent before putting them in `ctx.step(...)`. +4. Wrap Slack posts, database writes, external HTTP calls, and tool calls in + `ctx.step(...)` when they must not repeat. 5. Replace direct agent-control-plane calls with `ctx.agent_turn(...)`. 6. If the workflow uses `ctx._pool`, confirm the workflow-host sandbox receives `DATABASE_URL`. @@ -196,10 +199,10 @@ For each existing workflow: ## Known gaps -The v2 POC supports the workflow model, but it does not yet emulate the full -Python API package. Workflows that import `api.runtime_control`, `api.vm_metrics`, -or other Python API internals need a compatibility shim or a small local helper -before they are v2-ready. +The v2 workflow host intentionally exposes a narrow Python API package. +Workflows that import unrelated API-service internals should move that behavior +into the workflow-host API surface or a small local helper owned by the workflow +domain before they are v2-ready. `ctx.call_tool(...)` is a compatibility surface in the Python workflow host. It uses the generated `centaur-tools call` bridge against the installed tool @@ -221,6 +224,7 @@ Then create a real run: ```bash curl -s "$CENTAUR_API_URL/api/workflows/runs" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_name": "nightly_report", diff --git a/docs/public/md/extend/workflows.md b/docs/public/md/extend/workflows.md index d9b1b8b50..687cc66a4 100644 --- a/docs/public/md/extend/workflows.md +++ b/docs/public/md/extend/workflows.md @@ -1,14 +1,14 @@ --- title: Creating Workflows -description: Add durable Centaur workflows with checkpointed steps, sleeps, schedules, webhooks, Slack posts, tool calls, and agent turns. +description: Add durable Centaur workflows with checkpointed steps, sleeps, events, child workflows, and agent turns. --- # Creating Workflows -Workflows are Python handlers that run through Centaur's durable api-rs -workflow engine. They are useful when the task is longer than one agent turn: -polling, branching, retries, scheduled syncs, webhook handling, or coordinating -agent runs. +Workflows are Python handlers that run through Centaur's durable workflow +engine. They are useful when the task is longer than one agent turn: polling, +branching, retries, waiting for external events, or coordinating multiple agent +runs. Use a workflow when the system needs durable progress rather than a single request-response turn. Common examples include scheduled reports, ETL syncs, @@ -37,6 +37,7 @@ An optional `Input` dataclass gives structured inputs. ```python from dataclasses import dataclass +from datetime import timedelta from typing import Any from api.workflow_engine import WorkflowContext @@ -53,10 +54,10 @@ class Input: async def handler(inp: Input, ctx: WorkflowContext) -> dict[str, Any]: data = await ctx.step("collect", lambda: {"topic": inp.topic}) - await ctx.sleep_for("settle", 30) + await ctx.sleep("settle", timedelta(seconds=30)) result = await ctx.run_agent( - f"Write a short report about {data['topic']}", - thread_key=f"workflow:{ctx.run_id}:nightly_report", + "summarize", + text=f"Write a short report about {data['topic']}", ) return {"channel": inp.channel, "report": result} ``` @@ -65,18 +66,17 @@ async def handler(inp: Input, ctx: WorkflowContext) -> dict[str, Any]: | Primitive | Use it for | |-----------|------------| -| `ctx.step(name, fn)` | Run a deterministic or idempotent operation and cache its result after the callback succeeds. | -| `ctx.sleep_for(name, seconds)` | Suspend and resume later. | +| `ctx.step(name, fn)` | Run a side effect once and cache its result. | +| `ctx.sleep(name, duration)` | Suspend and resume later. | | `ctx.sleep_until(name, when)` | Resume at a specific time. | -| `ctx.agent_turn(text, **kwargs)` / `ctx.run_agent(text, **kwargs)` | Start an agent turn and wait for the result. | -| `ctx.call_tool(tool, method, args)` | Call a tool through the workflow-host `centaur-tools call` bridge. | -| `ctx.post_to_slack(channel, text, **kwargs)` | Post to Slack through the api-rs Slack context path. | -| `ctx._pool` | Access the workflow database pool when the workflow-host sandbox receives `DATABASE_URL`. | +| `ctx.wait_for_event(name, event_type, correlation_id)` | Wait for an external event. | +| `ctx.wait_for_workflow(...)` | Wait for a child workflow to finish. | +| `ctx.run_workflow(...)` | Start and wait in one call. | +| `ctx.start_agent(...)` | Start an agent turn. | +| `ctx.run_agent(...)` | Start an agent turn and wait for the result. | -The handler may re-execute after a restart. `ctx.step(...)` memoizes the result -after its callback returns, so writes and external API calls inside a step must -still be idempotent. If the host crashes after the external side effect but -before the checkpoint commits, the step can replay. +The handler may re-execute after a restart. Put external side effects behind +`ctx.step(...)` so completed work is not repeated. These primitives compose into larger automations: @@ -84,17 +84,26 @@ These primitives compose into larger automations: or business-hours monitor without a human prompt. - **Polling loops**: sleep between checks for CI, blockchain confirmations, billing state, deploy health, or vendor exports. -- **Event-driven flows**: expose a signed webhook and let the handler process a - normalized webhook envelope. +- **Event-driven flows**: wait for a webhook, approval, upload, or callback and + continue from the last checkpoint. +- **Fan-out/fan-in orchestration**: start child workflows for independent work + and wait for all of them before producing a final result. - **Agent orchestration**: use agents for judgment-heavy steps while the workflow owns timing, retries, state, and final delivery. ## Run a workflow -Create a run through the API: +The manual control API requires the trusted `CENTAUR_CONTROL_API_KEY` (or an +optional dedicated `WORKFLOW_API_KEY`). Agent tools use +a separate Console JWT lane: their workflow name must be listed in +`WORKFLOW_API_ALLOWED_NAMES`, and `input.thread_key` must belong to one of the +JWT's Slack upload channels. + +Create a run through the trusted operator lane: ```bash curl -s "$CENTAUR_API_URL/api/workflows/runs" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "workflow_name": "nightly_report", @@ -106,7 +115,8 @@ curl -s "$CENTAUR_API_URL/api/workflows/runs" \ Inspect it: ```bash -curl -s "$CENTAUR_API_URL/api/workflows/runs/$RUN_ID" | jq +curl -s "$CENTAUR_API_URL/api/workflows/runs/$RUN_ID" \ + -H "Authorization: Bearer $CENTAUR_CONTROL_API_KEY" | jq ``` ## Schedule a workflow @@ -154,9 +164,8 @@ declared in the schedule instead of inferred from wall-clock state when possible. For workflows that may run longer than their schedule interval, make each tick -idempotent. Put writes and external API calls in named `ctx.step(...)` blocks -only when they use stable provider-side idempotency keys or upserts, derive -those keys from the scheduled window, and have the handler detect +idempotent. Put writes and external API calls in named `ctx.step(...)` blocks, +derive stable keys from the scheduled window, and have the handler detect already-processed periods before starting expensive work. Interval schedules are useful when exact wall-clock alignment does not matter: @@ -182,29 +191,24 @@ entrypoints such as GitHub issue triage, billing events, or deploy callbacks. ```python from typing import Any +from api.webhooks import HeaderTriggerKey, HmacAuth, WebhookSpec from api.workflow_engine import WorkflowContext WORKFLOW_NAME = "github_issue_triage" WEBHOOKS = [ - { - "slug": "github-issue-triage", - "provider": "github", - "auth": {"type": "github", "secret_ref": "GITHUB_WEBHOOK_SECRET"}, - "trigger_key": {"type": "header", "header": "X-GitHub-Delivery"}, - "allowed_methods": ["POST"], - "allowed_content_types": [ + WebhookSpec( + slug="github-issue-triage", + provider="github", + auth=HmacAuth.github(secret_ref="GITHUB_WEBHOOK_SECRET"), + trigger_key=HeaderTriggerKey("X-GitHub-Delivery"), + allowed_methods=["POST"], + allowed_content_types=[ "application/json", "application/x-www-form-urlencoded", ], - "filter": { - "all": [ - {"source": "header", "key": "x-github-event", "op": "equals", "value": "issues"}, - {"source": "body", "key": "action", "op": "in", "values": ["opened", "reopened"]}, - ] - }, - } + ) ] @@ -213,6 +217,9 @@ async def handler(inp: dict[str, Any], ctx: WorkflowContext) -> dict[str, Any]: headers = webhook["headers"] payload = webhook["body"] + if headers.get("x-github-event") != "issues": + return {"skipped": True, "reason": "unsupported_event"} + issue = payload["issue"] repo = payload["repository"]["full_name"] result = await ctx.agent_turn( @@ -233,17 +240,11 @@ For GitHub, set the webhook secret to the same value as GitHub's default `application/x-www-form-urlencoded` payloads also work when that content type is listed in `allowed_content_types`. -Use `filter` for provider events that can be rejected from headers or JSON body -fields. The API evaluates the filter before creating a workflow run, which -keeps org-wide webhooks from spawning a sandbox for events the handler would -immediately skip. - Webhook requests do not use Centaur API keys. The API verifies the provider -signature before creating workflow state. Use -`{"type": "github", "secret_ref": "GITHUB_WEBHOOK_SECRET"}` for GitHub -`X-Hub-Signature-256` webhooks, or `{"type": "hmac", ...}` for other SHA-256 -HMAC providers. During local development or for trusted internal routes, use -`{"type": "none"}`. +signature before creating workflow state. `HmacAuth.github(...)` verifies +`X-Hub-Signature-256`; a plain `HmacAuth(...)` can be used for other +SHA-256 HMAC providers. During local development or for trusted internal +routes, `auth="none"` is allowed. The workflow receives input in this shape: diff --git a/docs/public/md/operate/slack-etl.md b/docs/public/md/operate/slack-etl.md index 1a9747de8..111dc557a 100644 --- a/docs/public/md/operate/slack-etl.md +++ b/docs/public/md/operate/slack-etl.md @@ -6,13 +6,13 @@ description: Sync Slack channel history into Postgres, drain historical backfill # Slack ETL :::warning[Off by default in production] -Slack ETL is disabled unless the API service has `SLACK_ETL_ENABLED=true`. +Slack ETL is disabled unless Helm values set `apiRs.etl.slack.enabled=true`. Production deployments should enable it deliberately after choosing the Slack token, channel scope, exclusion patterns, and data boundary they want agents to use. ::: -Slack ETL keeps an indexed, queryable copy of public Slack history in Postgres +Slack ETL keeps an indexed, queryable copy of Slack channel history in Postgres for agent context and operator workflows. It runs as scheduled Centaur workflows: one workflow keeps recent channel history fresh, one drains deferred historical backfill work, and one turns synced messages into company context @@ -27,7 +27,7 @@ token and writes durable rows into Postgres. | Workflow | Default cadence | Role | |----------|-----------------|------| -| `slack_sync` | 1 hour | Lists public channels, refreshes users, syncs recent root messages, advances per-channel checkpoints, and enqueues backfill jobs. | +| `slack_sync` | 1 hour | Lists channels, refreshes users, syncs recent root messages, advances per-channel checkpoints, and enqueues backfill jobs. | | `slack_backfill` | 10 minutes | Claims queued backfill jobs and drains Slack cursors without slowing the incremental sync. | | `company_context_documents` | 4 hours | Projects changed Slack rows into `company_context_documents` for retrieval. | @@ -39,27 +39,39 @@ posting to Slack. Create a Slack user token for ETL reads and store it as `SLACK_ETL_TOKEN` in the same secret source used by tools. The Slack tool declares it as an optional -HTTP secret for `slack.com` and `files.slack.com`; iron-proxy injects the real -value when the tool calls Slack. +HTTP secret scoped to `GET` and `POST` calls to the Slack Web API endpoints +below, plus `GET` downloads from `files.slack.com`; iron-proxy replaces the +`Authorization` header only for those requests. The token must be able to call: | Slack API | Used for | |-----------|----------| -| `conversations.list` | Discover public channels. | +| `conversations.list` | Discover public channels, and private channels when explicitly enabled. | | `conversations.history` | Read channel root messages. | | `conversations.replies` | Refresh thread replies. | | `users.list` | Resolve Slack user metadata for documents. | | `files:read` / file URL access | Download message attachment bytes from `files.slack.com`. | -Slack ETL currently syncs public channels visible to the configured ETL user -token. It does not sync private channels, DMs, or Slackbot-only live thread -events. +Slack ETL syncs public channels visible to the configured ETL user token. +Set `SLACK_SYNC_INDEX_PRIVATE_CHANNELS=true` to also sync private channels +visible to that token. It does not sync DMs or Slackbot-only live thread events. +Private channel rows are protected by RLS: `centaur_readonly` sees public +channel data and the channel in `centaur.slack_channel_id`. ## Enable the schedules -Set `SLACK_ETL_ENABLED=true` on the API service. The other schedules default on -once Slack ETL is enabled, but can be tuned independently. +Set `apiRs.etl.slack.enabled=true` in Helm values. The chart renders the +corresponding API and workflow-host env automatically; do not set +`SESSION_SANDBOX_PASSTHROUGH_ENV` by hand for these ETLs. The other schedules +default on once Slack ETL is enabled, but can be tuned independently. + +```yaml +apiRs: + etl: + slack: + enabled: true +``` | Environment variable | Default | Effect | |----------------------|---------|--------| @@ -71,15 +83,17 @@ once Slack ETL is enabled, but can be tuned independently. | `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `5` | Maximum Slack history pages drained before a job is requeued. | | `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS` | `30` | Historical window seeded for first-time channel backfills. | | `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `3` | Recent thread window eligible for reply refresh. | +| `SLACK_SYNC_INDEX_PRIVATE_CHANNELS` | `false` | Includes private channels visible to the ETL token in Slack sync and backfill. | | `SLACK_ETL_ATTACHMENTS_ENABLED` | `true` | Download Slack message attachment bytes into Postgres. Metadata rows are still written when downloads are disabled. | | `SLACK_ETL_ATTACHMENT_MAX_BYTES` | `10485760` | Per-file byte cap for Slack attachment downloads. Oversized files keep metadata with `skipped_too_large` status. | | `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | empty | Comma-separated channel-name globs to skip, without needing the leading `#`. | | `SLACK_RETENTION_ENABLED` | `true` | Allows the `slack_retention` schedule to run when at least one Slack retention TTL is positive. | | `SLACK_RETENTION_INTERVAL_MINUTES` | `60` | How often to prune Slack retention-managed rows. | -| `SLACK_ETL_RETENTION_DAYS` | `0` | Deletes public Slack ETL messages, derived Slack documents, and terminal ETL run/job rows older than this many days. `0` disables public ETL retention. | +| `SLACK_ETL_RETENTION_DAYS` | `0` | Deletes Slack ETL messages, derived Slack documents, and terminal ETL run/job rows older than this many days. `0` disables ETL retention. | | `SLACK_DM_RETENTION_DAYS` | `0` | Deletes Slack DM messages, stale empty DM conversations, and terminal DM run/job rows older than this many days. `0` disables DM retention. | | `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `true` | Enables projection from Slack sync rows into company context documents. | | `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `14400` | How often to project changed Slack rows into documents. | +| `COMPANY_CONTEXT_DOCUMENTS_MAX_WINDOW_SECONDS` | `21600` | Maximum source `updated_at` window projected by one company context documents run. | Example exclusion list: @@ -93,7 +107,7 @@ Slack ETL writes normalized Slack data into dedicated tables: | Table | Contents | |-------|----------| -| `slack_sync_channels` | Public channels visible to the ETL token and whether they are currently syncable. | +| `slack_sync_channels` | Channels visible to the ETL token, channel privacy, and whether they are currently syncable. | | `slack_sync_users` | Slack user display metadata used when rendering documents. | | `slack_sync_runs` | One row per incremental or backfill workflow run, with counts and channel outcomes. | | `slack_sync_messages` | Root messages and replies keyed by `(channel_id, message_ts)`. | @@ -125,18 +139,21 @@ TTL is positive. ## Run it manually Use a manual run when enabling the feature or testing a configuration change. -From inside the API deployment, localhost bypass avoids needing an external API -key: +The control endpoint remains authenticated on localhost; run curl inside the +API deployment so the trusted `CENTAUR_CONTROL_API_KEY` never leaves the pod: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "slack_sync", "input": {"metadata": {"reason": "manual_check"}}, "eager_start": true - }' | jq +} +JSON ``` Then inspect the run: @@ -144,34 +161,42 @@ Then inspect the run: ```bash RUN_ID=wfr_... -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/workflows/runs/${RUN_ID}" | jq +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s "http://localhost:8080/api/workflows/runs/$1" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' sh "$RUN_ID" | jq ``` To drain pending historical work immediately: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "slack_backfill", "input": {"channel_batch_limit": 10}, "eager_start": true - }' | jq +} +JSON ``` To force document projection after rows have synced: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ - http://localhost:8080/api/workflows/runs \ - -H "Content-Type: application/json" \ - -d '{ +kubectl exec -i -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s -X POST http://localhost:8080/api/workflows/runs \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" \ + -H "Content-Type: application/json" --data-binary @- +' <<'JSON' +{ "workflow_name": "company_context_documents", "input": {}, "eager_start": true - }' | jq +} +JSON ``` ## Verify @@ -179,8 +204,10 @@ kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s -X POST \ Check the workflow schedules: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - http://localhost:8080/api/workflows/schedules | jq \ +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s http://localhost:8080/api/workflows/schedules \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' | jq \ '.schedules[] | select(.schedule_id == "slack_sync" or .schedule_id == "slack_backfill" @@ -191,8 +218,10 @@ kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ Check recent workflow runs: ```bash -kubectl exec -n centaur deploy/centaur-centaur-api-rs -- curl -s \ - "http://localhost:8080/api/workflows/runs?limit=20" | jq \ +kubectl exec -n centaur deploy/centaur-centaur-api-rs -- sh -lc ' + curl -s "http://localhost:8080/api/workflows/runs?limit=20" \ + -H "Authorization: Bearer ${CENTAUR_CONTROL_API_KEY:?}" +' | jq \ '.runs[] | select(.workflow_name == "slack_sync" or .workflow_name == "slack_backfill" @@ -244,8 +273,8 @@ setting alerts. | Symptom | What to check | |---------|---------------| | Schedules are missing | Confirm `WORKFLOW_DIRS` includes `/app/workflows` and the API restarted after the workflow files were deployed. | -| Schedules exist but are disabled | Confirm `SLACK_ETL_ENABLED=true` is present in the API environment. | -| `slack_sync` skips with `no_public_channels` | Confirm the ETL user token can see the expected public channels. | +| Schedules exist but are disabled | Confirm Helm values set `apiRs.etl.slack.enabled=true` and the API pod was restarted. | +| `slack_sync` skips with `no_channels` | Confirm the ETL user token can see the expected public channels, or enable private channel sync when only private channels are in scope. | | Channels are all skipped | Check `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` for broad globs. | | Checkpoints show `missing_scope` or `not_allowed_token_type` | Add the missing Slack OAuth scope or use the expected user-token class. | | Backfill jobs keep failing | Inspect `slack_sync_backfill_jobs.last_error` and the corresponding `slack_sync_runs` row. | diff --git a/docs/public/md/quickstart.md b/docs/public/md/quickstart.md index 46d19d907..7faf05946 100644 --- a/docs/public/md/quickstart.md +++ b/docs/public/md/quickstart.md @@ -78,11 +78,9 @@ Application-level model and tool secrets, such as `OPENAI_API_KEY`, placeholder values and [iron-proxy](https://docs.iron.sh) injects the real credentials only on approved outbound requests. -The default harness is `claudecode`, so an Anthropic credential -(`ANTHROPIC_API_KEY`, or the brokered subscription token in `access_token` -mode) must exist in the configured secret source before Slack agent turns can -complete. Use explicit harness selectors only when you want a non-default -harness such as Codex or Amp. +The default harness is `codex`, so `OPENAI_API_KEY` must exist in the configured +secret source before Slack agent turns can complete. Use explicit harness +selectors only when you want a non-default harness such as Amp or Claude Code. ## 3. Boot the stack diff --git a/docs/public/md/reference/configuration.md b/docs/public/md/reference/configuration.md index a1d3cfe8e..6b3349324 100644 --- a/docs/public/md/reference/configuration.md +++ b/docs/public/md/reference/configuration.md @@ -35,7 +35,8 @@ These must exist for the normal Helm deployment. For local development, | `DATABASE_URL` | `secretManager.existingSecretName`; local bootstrap generates it. | API and Slackbot Postgres connection. | | `SLACK_SIGNING_SECRET` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack request signature verification. | | `SLACKBOT_API_KEY` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Static API key bootstrapped for Slackbot. | -| `SLACK_BOT_TOKEN` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack Web API access for Slackbot. | +| `CENTAUR_CONTROL_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. The key name is configurable with `apiRs.controlApiKeySecretKey`. | Trusted Console/operator authorization for global and administrative API routes. Never expose it to sandboxes or reuse another service key; api-rs fails startup when configured trust-lane credentials collide. | +| `SLACK_BOT_TOKEN` | `secretManager.existingSecretName`; local bootstrap reads shell env. | Slack Web API access for Slackbot and api-rs Slack helpers. | | `SANDBOX_SIGNING_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Signing key for short-lived sandbox API tokens. | | `IRON_MANAGEMENT_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Management key for API-created iron-proxy pods. | | `IRON_BROKER_TOKEN` | `secretManager.existingSecretName`; required when `tokenBroker.enabled=true`. | Bearer token iron-proxy presents to iron-token-broker and the broker enforces on its HTTP API. | @@ -51,6 +52,7 @@ Optional required-by-mode variables: | `LOCAL_DEV_API_KEY` | API env. | Static local admin/dev key bootstrapped into Postgres. | | `TEAMS_BOT_APP_ID`, `TEAMS_BOT_APP_PASSWORD`, `TEAMS_BOT_APP_TENANT_ID` | Local shell before `just bootstrap-secrets`; production Secret. | Required by Teamsbot when `teamsbot.enabled=true`. | | `TEAMSBOT_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it when Teams credentials are present and it is omitted. | Static API key used by Teamsbot. | +| `SLACK_FEEDBACK_API_KEY` | `secretManager.existingSecretName`; local bootstrap generates it. | Optional second factor for the sandbox Slack feedback/improvement tool. The API also requires the caller's Console JWT and preserves that principal's capabilities. Keep it distinct from `CENTAUR_CONTROL_API_KEY`. | ## API @@ -70,7 +72,7 @@ Optional required-by-mode variables: | `SLACKBOT_URL` | Chart-rendered Slackbot service URL. | API callback target for Slack delivery. | | `FINAL_DELIVERY_MAX_ATTEMPTS`, `FINAL_DELIVERY_READY_GRACE_S` | `api.extraEnv`. | Final-delivery retry and claim timing. | | `CENTAUR_ENABLE_GCLOUD_BOOTSTRAP`, `GCP_GCLOUD_CREDENTIAL`, `GCLOUD_PROJECT` | `api.extraEnv` or Secret. | Optional gcloud ADC bootstrap in the API container. | -| `CLAUDE_MODEL`, `CODEX_MODEL` | `api.extraEnv` or request model override. | Harness model selection defaults. | +| `CLAUDE_MODEL`, `CODEX_MODEL` | `api.extraEnv` or request model override. | Harness model selection defaults. When set via `sandbox.extraEnv`, the chart also mirrors them into slackbotv2 and the Console so their model displays track the deployment. | ## API-RS @@ -84,6 +86,24 @@ Optional required-by-mode variables: | `apiRs.metrics.scrapeAnnotations` | Helm value, default `true`. | Adds Prometheus scrape annotations to the API-RS Pod template and Service. | | `apiRs.metrics.path` | Helm value, default `/metrics`. | Metrics scrape path for annotation-based discovery. | | `apiRs.metrics.annotations` | Helm value. | Additional scrape annotations for Prometheus-compatible collectors. | +| `apiRs.activitySummary.*` | Helm values, default disabled. | Enables API-RS to summarize live session activity into durable `session.activity_summary` events. | +| `SLACK_BOT_TOKEN` | Explicit `secretKeyRef` from `secretManager.existingSecretName`. | Slack Web API access for api-rs Slack proxy and workflow Slack helpers. | +| `CENTAUR_CONTROL_API_KEY` | Required `secretKeyRef` from `secretManager.existingSecretName`. | Authorizes admin data routes, global sandbox drain, and trusted workflow/session control. Bot service keys remain session-API credentials but cannot authorize admin routes or global drain. | +| `SLACK_FEEDBACK_API_KEY` | Optional `secretKeyRef` from `secretManager.existingSecretName`. | Combined with the caller JWT, authorizes only principal-bound `feedback-improvement:*` sessions through `X-Centaur-Feedback-Key`; it cannot elevate repo access or operate admin routes. | +| `OPENAI_API_KEY` | Secret mounted into api-rs, or `apiRs.extraEnv` for local/dev overrides. | OpenAI credential for activity summaries; the feature stays disabled when no key is present. | +| `SESSION_ACTIVITY_SUMMARY_MODEL` | `apiRs.activitySummary.model`, default `gpt-5.4-nano`. | Model used for the short live activity sentence. | + +Sandbox lifecycle: + +| Env var or value | Set from | Controls | +| --- | --- | --- | +| `SESSION_IDLE_TIMEOUT_MS` | `slackbotv2.extraEnv`; default is up to 3 hours. | Slackbot v2 execute idle timeout. After an execution reaches a terminal state, api-rs pauses the sandbox if no newer execution has used that sandbox. If `SESSION_MAX_DURATION_MS` is lower than 3 hours and this value is unset, Slackbot v2 caps the default idle timeout to the max duration. | +| `SESSION_MAX_DURATION_MS` | `slackbotv2.extraEnv`. | Optional per-execution max duration forwarded to api-rs. api-rs rejects requests where `idle_timeout_ms` is greater than `max_duration_ms`. | +| `apiRs.sandboxMaxLifetimeSecs` / `SESSION_SANDBOX_MAX_LIFETIME_SECS` | Helm value, default `259200` (72 hours). | Restart-surviving sandbox deletion backstop. The reaper stops any non-terminal sandbox older than this, regardless of whether it is running or suspended. Set `0` to disable max-lifetime reaping. | +| `apiRs.sandboxReapIntervalSecs` / `SESSION_SANDBOX_REAP_INTERVAL_SECS` | Helm value, default `300`. | How often api-rs sweeps observed sandboxes for max-lifetime expiry. | + +There is no separate suspended-only delete timer. Pausing is controlled by the +per-execution idle timeout; deletion is controlled by sandbox max lifetime. Execution tuning: @@ -110,7 +130,6 @@ Execution tuning: | `SLACK_API_URL` | `slackbot.extraEnv`. | Optional Slack Web API base URL override. | | `CENTAUR_API_URL` | Chart-rendered API service URL. | API base URL used by Slackbot. | | `CENTAUR_SLACK_EVENTS_PATH` | `slackbot.extraEnv`. | Slack Events API route; defaults to `/api/webhooks/slack`. | -| `SLACKBOT_AMBIENT_CHANNEL_IDS` | `slackbotv2.extraEnv`. | Comma/space-separated Slack channel ids where messages start sessions without a bot mention. | | `RUNTIME_ERROR_ALERT_CHANNEL` | `slackbot.runtimeErrorAlertChannel`. | Slack channel for runtime error alerts. | | `SLACK_EVENT_DEDUP_TTL_MS` | `slackbot.extraEnv`. | Slack event dedupe window. | | `SLACK_SIGNATURE_MAX_AGE_SECONDS` | `slackbot.extraEnv`. | Maximum accepted Slack signature age. | @@ -164,7 +183,7 @@ Kubernetes backend: | `KUBERNETES_SANDBOX_RUNTIME_CLASS_NAME`, `KUBERNETES_SANDBOX_SERVICE_ACCOUNT_NAME` | `sandbox.runtimeClassName`, `api.extraEnv`. | Pod runtime class and service account. | | `KUBERNETES_SANDBOX_CPU_LIMIT`, `KUBERNETES_SANDBOX_MEMORY_LIMIT`, `KUBERNETES_SANDBOX_CPU_REQUEST`, `KUBERNETES_SANDBOX_MEMORY_REQUEST` | `sandbox.resources.*`. | Sandbox pod resources. | | `KUBERNETES_SANDBOX_READY_TIMEOUT_S`, `KUBERNETES_ATTACH_LOG_TAIL_LINES` | `api.extraEnv`. | Sandbox readiness and attach diagnostics. | -| `SESSION_SANDBOX_CLEANUP_INTERVAL_SECS`, `SESSION_SANDBOX_IDLE_CLEANUP_BACKSTOP_SECS` | `apiRs.sandboxCleanupIntervalSecs`, `apiRs.sandboxIdleCleanupBackstopSecs`. | DB-aware cleanup of unreferenced sandboxes and idle-pause backstop after API restarts. | +| `SESSION_SANDBOX_CLEANUP_INTERVAL_SECS`, `SESSION_SANDBOX_IDLE_CLEANUP_BACKSTOP_SECS` | `apiRs.sandboxCleanupIntervalSecs`, `apiRs.sandboxIdleCleanupBackstopSecs`. | DB-aware cleanup of unreferenced sandboxes and restart recovery for idle pauses. Persisted `idle_timeout_ms` is honored after restart; the backstop is the fallback for older execution rows without that metadata. | | `KUBERNETES_SANDBOX_EXTRA_ENV` | `sandbox.extraEnv`. | JSON list copied into each sandbox. | | `KUBERNETES_WORKFLOW_DIRS` | Chart-rendered from `overlays.sources[*].workflowsSubdir` (default `workflows`) using the sandbox repo-cache mount prefix. | Workflow-host sandbox discovery paths. | | `KUBERNETES_FIREWALL_CA_SECRET_NAME`, `KUBERNETES_FIREWALL_CA_KEY_SECRET_NAME` | `firewall.existingCa*` or generated CA Secrets. | CA material for sandbox/proxy TLS interception. | @@ -182,14 +201,19 @@ Sandbox entrypoint and wrappers: | Env var | Set from | Controls | | --- | --- | --- | -| `CENTAUR_HARNESS_CONFIG_DIR`, `CENTAUR_HARNESS_ADAPTER` | Sandbox image or `sandbox.extraEnv`. | Harness config directory and optional adapter executable. | +| `CENTAUR_HARNESS_CONFIG_DIR`, `CENTAUR_HARNESS_ADAPTER` | Sandbox image or `sandbox.extraEnv`. | Authoritative harness config directory and optional adapter executable. The entrypoint copies this directory's Codex and Claude files instead of merging them with the image-baked `~/harness` defaults, so an override must carry every required provider, feature, and trust setting. Leave it unset to use the reviewed config packaged in the sandbox image. | | `CENTAUR_SKILL_DIRS` | Chart-rendered from `overlays.sources[*].skillsSubdir` (default `.agents/skills`) through `SESSION_SANDBOX_EXTRA_ENV`. | Ordered skill directories copied into the agent workspace. | +| `CENTAUR_TOOLS_AUTO_RELOAD` | `repoCache.autoReload` via api-rs tools config; defaults to `true`. | Enables repo-cache-backed auto-refresh of local tool shims and copied skills in running sandboxes. Runtime catalog only; secret grants/proxy credentials reconcile separately. | +| `CENTAUR_TOOLS_RELOAD_INTERVAL_SECONDS` | `sandbox.extraEnv`. | Poll interval for the repo-cache checkout watchdog. | | `AGENT_REPO`, `AGENT_PERSONA` | Runtime assignment metadata. | Workspace repo clone and persona prompt. | | `GOOGLE_APPLICATION_CREDENTIALS` | Sandbox entrypoint or `sandbox.extraEnv`. | Google ADC path; entrypoint creates a local stub when unset. | | `CODEX_API_KEY`, `CODEX_HOME`, `CODEX_CONTINUE_THREAD_ID` | `sandbox.extraEnv` or runtime resume. | Codex auth/config/resume behavior. | | `CODEX_AUTH_MODE` | `sandbox.extraEnv`. | Codex auth flow: `api_key` (default, hits `api.openai.com`) or `access_token` (hits `chatgpt.com` via the brokered ChatGPT login). See [Codex Auth Modes](/deploying-in-production#codex-auth-modes). | +| `META_AI_API_KEY` | Secret mounted into api-rs. | Meta AI direct credential for Codex provider `responses` and Slack or Linear `--meta` selection. | | `CODEX_MODEL_REASONING_SUMMARY` | `sandbox.extraEnv`. | Sets `model_reasoning_summary` in the Codex config (`auto`, `concise`, `detailed`, `none`). Codex >= 0.139 emits no reasoning summaries unless this is set, so renderers show no thinking trace. | -| `CODEX_MODEL_REASONING_EFFORT` | `sandbox.extraEnv`. | Overrides the codex `model_reasoning_effort` (baked into `harness/codex/config.toml`) by patching the per-sandbox `~/.codex/config.toml` at boot, without forking the image. One of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`; an unknown value is ignored (the config default stands). | +| `CODEX_MODEL_REASONING_EFFORT` | `sandbox.extraEnv`. | Overrides the codex `model_reasoning_effort` (baked into `harness/codex/config.toml`) by patching the per-sandbox `~/.codex/config.toml` at boot, without forking the image. One of `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`; an unknown value is ignored (the config default stands). | +| `CODEX_BEDROCK_REGION` | `sandbox.extraEnv`. | Opt-in switch and single source of truth for the Bedrock region. When set, the control plane registers the AWS SigV4 re-signing credential (scoped to the `bedrock` service and this region, upstream `bedrock-mantle..api.aws`), injects the placeholder `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` env so codex can sign requests iron-proxy re-signs with the real IAM keys, and pins codex's `amazon-bedrock` provider to this region at sandbox boot (so the in-sandbox client and the proxy agree). Unset disables Bedrock; defaults to `us-east-1`. See [Codex with Amazon Bedrock](/deploying-in-production#codex-with-amazon-bedrock). | +| `CODEX_BEDROCK_SESSION_TOKEN` | `sandbox.extraEnv`. | Set truthy when the Bedrock IAM credentials are temporary (STS) and carry a session token, so the `AWS_SESSION_TOKEN` placeholder is declared and injected. Omit for long-term IAM user keys. | | `CLAUDE_MODEL`, `CLAUDE_CONTINUE_SESSION_ID` | `sandbox.extraEnv` or runtime resume. | Claude model and resume behavior. | | `CLAUDE_CODE_AUTH_MODE` | `sandbox.extraEnv`. | Claude Code auth flow: `api_key` (default, uses `ANTHROPIC_API_KEY`) or `access_token` (Claude.ai Pro or Max via the brokered OAuth login). See [Claude Auth Modes](/deploying-in-production#claude-auth-modes). | | `DEPLOY_ENV`, `ENVIRONMENT`, `TRACEPARENT` | Deployment env or wrapper-generated. | Runtime environment and trace context. | @@ -210,23 +234,31 @@ Slack ETL workflows: | Env var | Set from | Controls | | --- | --- | --- | -| `SLACK_ETL_ENABLED` | `api.slackEtlEnabled`. | Master switch for Slack sync/backfill/context schedules. | -| `SLACK_SYNC_INTERVAL_SECONDS`, `SLACK_BACKFILL_INTERVAL_SECONDS`, `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `api.*IntervalSeconds`. | Slack ETL schedule intervals. | -| `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS`, `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `api.slackSync*LookbackDays`. | Slack history/thread lookback windows. | -| `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | `api.slackEtlExcludedChannelPatterns`. | Comma-separated channel-name globs to skip. | -| `SLACK_BACKFILL_ENABLED`, `SLACK_BACKFILL_CHANNEL_BATCH_LIMIT`, `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `api.extraEnv` or chart batch limit. | Backfill enablement and batch sizing. | -| `SLACK_RETENTION_INTERVAL_MINUTES`, `SLACK_ETL_RETENTION_DAYS`, `SLACK_DM_RETENTION_DAYS` | `api.extraEnv`. | Slack retention cadence and separate public ETL/DM TTLs. | -| `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `api.extraEnv`. | Enables company-context projection when Slack ETL is on. | +| `SLACK_ETL_ENABLED` | `apiRs.etl.slack.enabled`. | Master switch for Slack sync/backfill/context schedules. | +| `SLACK_SYNC_INTERVAL_SECONDS`, `SLACK_BACKFILL_INTERVAL_SECONDS`, `COMPANY_CONTEXT_DOCUMENTS_INTERVAL_SECONDS` | `apiRs.etl.slack.syncIntervalSeconds`, `apiRs.etl.slack.backfill.intervalSeconds`, `apiRs.etl.companyContextDocuments.intervalSeconds`. | Slack ETL schedule intervals. | +| `SLACK_SYNC_BACKFILL_LOOKBACK_DAYS`, `SLACK_SYNC_THREAD_LOOKBACK_DAYS` | `apiRs.etl.slack.syncBackfillLookbackDays`, `apiRs.etl.slack.syncThreadLookbackDays`. | Slack history/thread lookback windows. | +| `SLACK_SYNC_INDEX_PRIVATE_CHANNELS` | `apiRs.etl.slack.indexPrivateChannels`. | Includes private channels visible to the ETL token. | +| `SLACK_ETL_EXCLUDED_CHANNEL_PATTERNS` | `apiRs.etl.slack.excludedChannelPatterns`. | Comma-separated channel-name globs to skip. | +| `SLACK_BACKFILL_ENABLED`, `SLACK_BACKFILL_CHANNEL_BATCH_LIMIT`, `SLACK_BACKFILL_CHANNEL_PAGES_PER_JOB` | `apiRs.etl.slack.backfill.*`. | Backfill enablement and batch sizing. | +| `SLACK_RETENTION_ENABLED`, `SLACK_RETENTION_INTERVAL_MINUTES`, `SLACK_ETL_RETENTION_DAYS`, `SLACK_DM_RETENTION_DAYS` | `apiRs.etl.slack.retention.*`. | Slack retention enablement, cadence, and separate public ETL/DM TTLs. | +| `COMPANY_CONTEXT_DOCUMENTS_ENABLED` | `apiRs.etl.companyContextDocuments.enabled`. | Enables company-context projection when any ETL is on. | +| `COMPANY_CONTEXT_DOCUMENTS_MAX_WINDOW_SECONDS` | `apiRs.etl.companyContextDocuments.maxWindowSeconds`. | Maximum source `updated_at` window projected by one company-context documents run. | Google Workspace ETL workflows: | Env var | Set from | Controls | | --- | --- | --- | -| `GOOGLE_DRIVE_ETL_ENABLED` | `api.googleDriveEtlEnabled`. | Enables Google Drive Docs sync. | -| `GOOGLE_DRIVE_ETL_FOLDER_IDS` | `api.extraEnv`. | Comma- or whitespace-separated Drive folder IDs/URLs to recursively full-scan for Google Docs. | -| `GOOGLE_DRIVE_SYNC_INTERVAL_SECONDS` | `api.googleDriveSyncIntervalSeconds`. | Google Drive Docs sync schedule interval. | -| `GOOGLE_CALENDAR_ETL_ENABLED` | `api.googleCalendarEtlEnabled`. | Enables Google Calendar sync. | -| `GOOGLE_CALENDAR_SYNC_INTERVAL_SECONDS` | `api.googleCalendarSyncIntervalSeconds`. | Google Calendar sync schedule interval. | +| `GOOGLE_DRIVE_ETL_ENABLED` | `apiRs.etl.googleDrive.enabled`. | Enables Google Drive Docs sync. | +| `GOOGLE_DRIVE_SYNC_INTERVAL_SECONDS` | `apiRs.etl.googleDrive.syncIntervalSeconds`. | Google Drive Docs sync schedule interval. | +| `GOOGLE_CALENDAR_ETL_ENABLED` | `apiRs.etl.googleCalendar.enabled`. | Enables Google Calendar sync. | +| `GOOGLE_CALENDAR_SYNC_INTERVAL_SECONDS` | `apiRs.etl.googleCalendar.syncIntervalSeconds`. | Google Calendar sync schedule interval. | + +Linear ETL workflows: + +| Env var | Set from | Controls | +| --- | --- | --- | +| `LINEAR_ETL_ENABLED` | `apiRs.etl.linear.enabled`. | Enables Linear project/issue/comment sync. | +| `LINEAR_SYNC_INTERVAL_SECONDS` | `apiRs.etl.linear.syncIntervalSeconds`. | Linear sync schedule interval. | ## Observability and Retention diff --git a/docs/public/md/reference/tool-directory.md b/docs/public/md/reference/tool-directory.md index 903d5dffb..b5c044f2f 100644 --- a/docs/public/md/reference/tool-directory.md +++ b/docs/public/md/reference/tool-directory.md @@ -12,13 +12,13 @@ Centaur ships with a set of tool integrations under `tools/`. Deployments can en The repo inventory is not the same as a live deployment. To see what an agent can use in a running sandbox, ask it to run: ```bash -centaur-tools list +call tools ``` -To inspect a specific tool's CLI: +To inspect a specific tool's methods and parameters: ```bash -linear --help +call discover linear ``` The `API key / credential` column uses the secret names declared by each tool's `[tool.centaur]` config. `None` means the base tool declares no required tool-specific credential; optional credentials are called out separately. @@ -31,12 +31,13 @@ These are broadly useful across most deployments and are good candidates to conf |---|---|---| | `linear` | Search, create, update, and comment on Linear issues, projects, cycles, teams, and labels | `LINEAR_API_KEY` | | `notion` | Search and update Notion pages, databases, blocks, and comments | `NOTION_API_KEY` | -| `slack` | Search Slack, read threads, inspect channels/users, and send or upload messages | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_UPLOAD_TOKEN`, `SLACK_ETL_TOKEN` | +| `slack` | Search Slack, read threads, inspect channels/users, and send or upload messages | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_ETL_TOKEN` | | `gsuite` | Use Gmail, Calendar, Drive, Docs, Sheets, Slides, and Google Analytics | `GOOGLE_TOKEN_JSON` | | `websearch` | Free web search via Parallel and deep research | None; `PARALLEL_API_KEY` for `deep_research`; `ANTHROPIC_API_KEY` for search synthesis | -| `company_context` | Search indexed company history across internal sources | None | +| `company_context` | Search indexed company history, Slack DMs, and Google Docs | None | | `grafana` | Query dashboards, alerts, VictoriaMetrics, VictoriaLogs, and annotations | `GRAFANA_URL`, `GRAFANA_API_KEY` | | `posthog` | Query product analytics, events, pageviews, breakdowns, and user agents | `POSTHOG_API_KEY`, `POSTHOG_PROJECT_ID` | +| `amplitude` | Query product analytics — event segmentation, funnels, retention, user activity, and taxonomy | `AMPLITUDE_API_KEY`, `AMPLITUDE_SECRET_KEY` | | `attio` | Work with CRM objects, records, lists, notes, tasks, calls, and meetings | `ATTIO_API_KEY` | | `pylon` | Read and manage support issues, accounts, contacts, teams, tags, and users | `PYLON_API_KEY` | @@ -63,6 +64,7 @@ These are broadly useful across most deployments and are good candidates to conf | `demo` | Test tool hot-reload and basic tool plumbing | None | | `grafana` | Grafana dashboards, alerts, VictoriaMetrics, VictoriaLogs, and annotations | `GRAFANA_URL`, `GRAFANA_API_KEY` | | `posthog` | Product analytics through HogQL, events, pageviews, and breakdowns | `POSTHOG_API_KEY`, `POSTHOG_PROJECT_ID` | +| `amplitude` | Amplitude event segmentation, funnels, retention, user activity, realtime, and taxonomy | `AMPLITUDE_API_KEY`, `AMPLITUDE_SECRET_KEY` | | `profslice` | Extract Firefox Profiler data for analysis | None | | `reth` | Reth execution timing and performance metrics | None | | `reth-log-analyzer` | Parse Reth logs and generate performance graphs | None | @@ -74,7 +76,7 @@ These are broadly useful across most deployments and are good candidates to conf | Tool | Use | API key / credential | |---|---|---| | `airtable` | Bases, schemas, tables, records, views, and URL parsing | `AIRTABLE_API_KEY` | -| `company_context` | Search indexed company history across internal sources | None | +| `company_context` | Search indexed company history, Slack DMs, and Google Docs | None | | `composio` | Execute tools from third-party services exposed through Composio | `COMPOSIO_API_KEY` | | `figma` | Extract Figma files, nodes, components, styles, and variables | `FIGMA_ACCESS_TOKEN` | | `granola` | Search and read Granola notes and transcripts | `GRANOLA_API_KEY` | @@ -82,7 +84,7 @@ These are broadly useful across most deployments and are good candidates to conf | `linear` | Linear issues, projects, cycles, teams, workflow states, and labels | `LINEAR_API_KEY` | | `notion` | Notion pages, databases, blocks, comments, and users | `NOTION_API_KEY` | | `opentable` | Search OpenTable restaurant reservations | None | -| `slack` | Slack messages, files, channels, threads, users, and usergroups | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_UPLOAD_TOKEN`, `SLACK_ETL_TOKEN` | +| `slack` | Slack messages, files, channels, threads, users, and usergroups | `SLACK_BOT_TOKEN`; optional: `SLACK_SEARCH_TOKEN`, `SLACK_ETL_TOKEN` | ## Research @@ -136,7 +138,7 @@ These tools ship in the base repo because many Centaur users need onchain or mar | `kalshi` | Prediction market events, markets, trades, and candlesticks | None | | `karma` | DAO delegate reputation, activity, scores, and governance analytics | None | | `messari` | Crypto asset prices, metrics, profiles, markets, news, and timeseries | `MESSARI_API_KEY` | -| `mpp` | Paid market-data and web-search requests through Machine Payments Protocol | None | +| `mpp` | Paid MPP requests | None | | `nansen` | Wallet labels, smart-money activity, token flows, holders, and PnL | `NANSEN_API_KEY` | | `polymarket` | Prediction market events, markets, prices, books, and trades | None | | `snapshot` | Offchain governance spaces, proposals, votes, and voting power | `SNAPSHOT_API_KEY` | diff --git a/docs/public/md/secrets/environment.md b/docs/public/md/secrets/environment.md index 19b871f85..4a3aabcd0 100644 --- a/docs/public/md/secrets/environment.md +++ b/docs/public/md/secrets/environment.md @@ -31,8 +31,9 @@ kubectl create secret generic centaur-infra-env \ --namespace centaur-system \ --from-literal=DATABASE_URL='postgres://...' \ --from-literal=SLACKBOT_API_KEY='...' \ + --from-literal=CENTAUR_CONTROL_API_KEY="$(openssl rand -hex 32)" \ + --from-literal=SLACK_FEEDBACK_API_KEY="$(openssl rand -hex 32)" \ --from-literal=SLACK_BOT_TOKEN='xoxb-...' \ - --from-literal=SLACK_UPLOAD_TOKEN='xoxp-...' \ --from-literal=SLACK_SIGNING_SECRET='...' \ --from-literal=SANDBOX_SIGNING_KEY="$(openssl rand -hex 32)" \ --from-literal=IRON_MANAGEMENT_API_KEY="$(openssl rand -hex 32)" \ @@ -42,6 +43,17 @@ kubectl create secret generic centaur-infra-env \ --from-literal=WAREHOUSE_API_KEY='...' ``` +`CENTAUR_CONTROL_API_KEY` is a required control-plane credential. Use it only +from Console and operator tooling; do not place it in a sandbox. The optional +`SLACK_FEEDBACK_API_KEY` has a separate, narrow feedback-session scope and must +not reuse the control key. + +When upgrading an existing release, add the prefixed control key (for example, +`PREFIX_CENTAUR_CONTROL_API_KEY` when `envPrefix: PREFIX_`) before Helm/Argo +applies the new workloads. The API and Console references are non-optional, so +pods will not start until that Secret key exists. `just bootstrap-secrets` +tops up existing local-development Secrets without rotating an existing key. + For local development, `just bootstrap-secrets` creates the local Kubernetes Secret from your shell environment. @@ -91,7 +103,7 @@ endpoint, and inject a short-lived bearer token for matching API hosts. Check the API pod environment: ```bash -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- env | \ +kubectl exec -n centaur-system deploy/centaur-centaur-api -- env | \ grep -E 'FIREWALL_MANAGER_SECRET_SOURCE|WAREHOUSE_API_KEY' ``` diff --git a/docs/public/md/secrets/oauth-apps.md b/docs/public/md/secrets/oauth-apps.md index bdde378ff..3422b3676 100644 --- a/docs/public/md/secrets/oauth-apps.md +++ b/docs/public/md/secrets/oauth-apps.md @@ -7,203 +7,70 @@ description: Register OAuth clients, collect user consent, and grant refreshed a OAuth apps let users connect their own upstream accounts to Centaur. An operator registers an OAuth client in the console, shares a consent link, and each user -who completes the flow creates or updates a managed broker credential. - -The broker credential owns refresh-token lifecycle. It refreshes access tokens -inside the Centaur Console and exposes only the current access token to iron-proxy -through a `token_broker` secret source. The user's refresh token never leaves -the Centaur Console. - -OAuth apps are separate from console login. Console SSO uses -`/auth//start` and signs operators into the console. OAuth apps use -`/oauth//start` and mint credentials for tools. +who completes the flow gets a managed credential. The Centaur Console keeps the +token fresh and iron-proxy injects it as `Authorization: Bearer ` +into requests to the provider's API hosts. Refresh tokens never leave the +Centaur Console. ## Supported Providers | Provider | Use | |----------|-----| -| `google` | Google API credentials, such as Gmail or Drive scopes. | -| `slack` | Slack user-token credentials with normal Slack API scopes. | - -Google flows request offline access and force consent so the token response -includes a refresh token. Slack OAuth apps should enable token rotation so the -callback also receives a refresh token. - -## Create The Provider App - -Create an OAuth client in the upstream provider first. - -Register this callback URL: - -```text -/oauth//callback -``` - -For example: - -```text -https://control.example.com/oauth/google-drive/callback -``` +| `google` | Google APIs, such as Gmail or Drive scopes. | +| `slack` | Slack user tokens with normal Slack API scopes. | +| `github` | GitHub user tokens for `api.github.com`. | +| `granola` | Granola MCP tokens for `mcp.granola.ai`. | +| `linear` | Linear tokens for `api.linear.app`. | +| `attio` | Attio workspace tokens for `api.attio.com`. | -The slug is the stable name users see in the consent URL. It must contain only -URL-safe characters. +## Set Up An App -For Slack, use normal Slack API scopes such as `channels:history` or -`users:read`. Do not use Sign in with Slack scopes such as `openid`, `email`, or -`profile` for OAuth apps. +1. **Create an OAuth client with the provider** (for example in the Google + Cloud console or the Attio developer dashboard). Register this callback + URL: `/oauth//callback`. +2. **Register it in Centaur.** In the console, open **OAuth Apps**, click + **Add App**, and fill in the slug, provider, client id, client + secret, and allowed scopes (one per line). +3. **Share the consent link** shown on the app page: + `/oauth//start`. Each user who opens it + and approves the provider's consent screen gets a credential, wrapped in a + grantable secret. -## Register The App In Centaur +Re-consenting with the same account updates the existing credential instead of +creating another one. -In the console, open **OAuth Apps**, then create an app with: +## Provider-Specific Setup -| Field | Meaning | -|-------|---------| -| `Slug` | Globally unique consent-link name, for example `google-drive`. | -| `Provider` | `google` or `slack`. | -| `Client ID` | OAuth client id from the provider. | -| `Client Secret` | OAuth client secret from the provider. Stored encrypted. | -| `Credential Namespace` | Namespace for broker credentials minted by this app. | -| `Allowed Scopes` | One scope per line. Consent requests must be a subset. | -| `Enabled` | Disabled apps reject new consent flows. Existing credentials keep refreshing. | +### Granola -You can also create the app through the API: +Granola has no app dashboard; obtain the OAuth client once via dynamic client +registration, then use the returned `client_id` and `client_secret` when adding +the app in the console: ```bash -curl -sS -X POST "$IRON_CONTROL_URL/api/v1/oauth_apps" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" \ +curl -sS -X POST https://mcp-auth.granola.ai/oauth2/register \ -H "Content-Type: application/json" \ -d '{ - "data": { - "slug": "google-drive", - "description": "Google Drive user access", - "provider": "google", - "client_id": "client-id.apps.googleusercontent.com", - "client_secret": "client-secret", - "credential_namespace": "default", - "allowed_scopes": [ - "https://www.googleapis.com/auth/drive.metadata.readonly" - ], - "enabled": true, - "labels": { "team": "platform" } - } + "client_name": "Centaur Console", + "redirect_uris": ["/oauth/granola/callback"], + "grant_types": ["authorization_code", "refresh_token"], + "response_types": ["code"], + "token_endpoint_auth_method": "client_secret_post", + "scope": "openid email profile offline_access mcp" }' ``` -`client_secret` is write-only. API responses never include it. Updating an app -without a new `client_secret` keeps the stored value. - -## Collect User Consent - -Share the app start URL with the user: - -```text -/oauth//start -``` - -Omitting `scopes` requests every allowed scope: - -```text -https://control.example.com/oauth/google-drive/start -``` - -To request a subset, pass scopes as a space-separated or comma-separated query -parameter: - -```text -https://control.example.com/oauth/google-drive/start?scopes=https://www.googleapis.com/auth/drive.metadata.readonly -``` - -The start endpoint rejects unknown slugs, disabled apps, and scopes outside the -app allowlist. After provider consent, the callback exchanges the code, records -the provider account identity, and renders a console result page. - -Re-consenting with the same app and provider account updates the existing broker -credential instead of creating another one. - -## What Gets Created - -A successful consent creates or updates: - -| Resource | Purpose | -|----------|---------| -| Broker credential | Stores provider identity, scopes, current access token, refresh token, expiry, and refresh state. | -| Static secret | Grantable wrapper that injects `Authorization: Bearer `. | - -The static secret uses a `token_broker` source that points at the broker -credential. At proxy sync time, the Centaur Console resolves the broker credential and -sends the current access token to iron-proxy. If the credential is still -bootstrapping or cannot refresh, the secret is omitted from proxy config until -it recovers. +Use `mcp` as the allowed scope for the app. -The auto-created request rules are provider-scoped: +## Grant The Credential -| Provider | Default API host rules | -|----------|------------------------| -| Google | `*.googleapis.com` | -| Slack | `slack.com` | +Consent does not automatically grant the token to every session. In the +console, open **Principals**, choose the user or channel, and use **Direct +Grants** to select the secret created for the credential — or grant it to a +reusable role. -Operators can tighten the static secret's rules in the console if a credential -should only be valid for specific API paths. - -## Grant The OAuth Credential - -OAuth consent does not automatically grant the token to every session. Grant the -auto-created static secret to the correct user, channel, or role. - -You can grant the secret in the Centaur Console. Open **Principals**, choose the -user or channel principal, then use **Direct Grants** to select the static secret -created for the broker credential. The same principal page can assign a role if -you grant the OAuth secret to a reusable role instead. - -For scripted changes, list secrets in the credential namespace and find the -static secret created for the broker credential: - -```bash -curl -sS "$IRON_CONTROL_URL/api/v1/static_secrets?namespace=default" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" | jq -``` - -Then grant the secret with `centaur-perms`: - -```bash -cd services/api-rs -cargo run -p centaur-perms -- \ - principals grant slack-user-u123 \ - --secret ssr_... -``` - -Grant the same credential to a channel when the channel should define access: - -```bash -cargo run -p centaur-perms -- \ - principals grant slack-channel-c456 \ - --secret ssr_... -``` - -Or grant it to a reusable role: - -```bash -cargo run -p centaur-perms -- \ - roles grant tool-google-drive \ - --secret ssr_... -``` - -## Rotate Or Disable - -Rotating the OAuth client's secret on the app updates every credential minted by -that app because minted broker credentials delegate `client_id` and -`client_secret` back to the app. - -Disable an app to stop new consent flows: - -```bash -curl -sS -X PATCH "$IRON_CONTROL_URL/api/v1/oauth_apps/google-drive" \ - -H "Authorization: Bearer $IRON_CONTROL_API_KEY" \ - -H "Content-Type: application/json" \ - -d '{ "data": { "enabled": false } }' -``` +## Disable Or Remove -Existing broker credentials keep refreshing while the app exists. To fully -remove access, revoke grants to the wrapper static secret, delete the wrapper -secret, then delete or unlink the broker credential. An app cannot be deleted -while minted credentials still reference it. +Toggle **Enabled** off on the app page to stop new consent flows; existing +credentials keep working. To fully remove access, revoke grants to the wrapper +secret, delete it, then delete the credential. diff --git a/docs/public/md/secrets/onepassword.md b/docs/public/md/secrets/onepassword.md index 87542b6fa..2d24b85e6 100644 --- a/docs/public/md/secrets/onepassword.md +++ b/docs/public/md/secrets/onepassword.md @@ -72,8 +72,8 @@ It must also include infrastructure secrets such as: ```text DATABASE_URL SLACKBOT_API_KEY +CENTAUR_CONTROL_API_KEY SLACK_BOT_TOKEN -SLACK_UPLOAD_TOKEN SLACK_SIGNING_SECRET SANDBOX_SIGNING_KEY IRON_MANAGEMENT_API_KEY @@ -81,6 +81,17 @@ IRON_MANAGEMENT_API_KEY Those are boot-time service secrets, not tool credentials. +`CENTAUR_CONTROL_API_KEY` is required and must be a dedicated high-entropy +control-plane value. It does not belong in the 1Password tool-credential vault +or any sandbox. `SLACK_FEEDBACK_API_KEY` is optional but, when the feedback tool +is enabled, must be a different value because it receives only the +`feedback-improvement:*` session capability. + +For an existing deployment, provision the prefixed control key in +`secretManager.existingSecretName` before upgrading. The API and Console +Secret references are non-optional; adding the value to a 1Password item alone +does not satisfy this boot-time Kubernetes Secret requirement. + ## Name 1Password items For the normal tool declaration: @@ -112,6 +123,7 @@ Store enabled harness credentials the same way: |------------|----------| | `OPENAI_API_KEY` | Codex default | | `OPENROUTER_API_KEY` | OpenRouter via Codex | +| `META_AI_API_KEY` | Meta AI direct via Codex | | `AMP_API_KEY` | Amp | | `ANTHROPIC_API_KEY` | Claude Code and pi-mono | @@ -122,7 +134,7 @@ Each item should live in `OP_VAULT` with its value in `credential`. Check that the API and [iron-proxy](https://docs.iron.sh) received the expected source mode: ```bash -kubectl exec -n centaur-system deploy/centaur-centaur-api-rs -- env | \ +kubectl exec -n centaur-system deploy/centaur-centaur-api -- env | \ grep -E 'FIREWALL_MANAGER_SECRET_SOURCE|OP_VAULT' ``` diff --git a/docs/public/md/what-is-centaur.md b/docs/public/md/what-is-centaur.md index f97ef287b..b5dc97ecc 100644 --- a/docs/public/md/what-is-centaur.md +++ b/docs/public/md/what-is-centaur.md @@ -23,9 +23,7 @@ Sandboxes speak a stable Anthropic-style message format with the API. Harness-sp ## Approved Tools -Agents call approved tool CLIs inside the sandbox, not ad hoc local -credentials. Tool plugins are discovered by api-rs for metadata and secret -grants, then installed in sandboxes as local CLI shims by `centaur-tools`. +Agents call tools through Centaur's API, not through ad hoc local credentials. Tool plugins expose typed REST endpoints, are discovered by the API, and can be extended without changing the core control plane. This creates a narrow and auditable boundary for agent capabilities. Teams decide which tools exist, how they authenticate, and what methods are available. @@ -35,15 +33,9 @@ Sandboxes only ever see placeholder strings for upstream credentials. Real value ## Durable Workflows -Centaur includes a durable workflow runtime for long-running automation: -api-rs owns the Absurd-backed state machine, while `services/workflow-python` -runs Python workflow handlers. Handlers checkpoint each step, sleep or wait for -external events, start child workflows, and run agent turns as part of larger -processes. +Centaur includes a Python workflow engine for long-running automation. Workflow handlers checkpoint each step, sleep or wait for external events, start child workflows, and run agent turns as part of larger processes. -This lets teams move beyond one-off prompts. A workflow can poll, branch, -retry, invoke tools, wait for a signal, delegate to an agent turn, and resume -after process restarts without rebuilding orchestration from scratch. +This lets teams move beyond one-off prompts. A workflow can poll, branch, retry, call tools, wait for a signal, delegate to an agent turn, and resume after process restarts without rebuilding orchestration from scratch. ## Slack And API Surfaces diff --git a/harness/claude/settings.json b/harness/claude/settings.json index bd1a5bbce..de16ceac2 100644 --- a/harness/claude/settings.json +++ b/harness/claude/settings.json @@ -1,6 +1,6 @@ { - "model": "claude-sonnet-5", - "effortLevel": "xhigh", + "model": "claude-opus-4-8", + "alwaysThinkingEnabled": true, "permissions": { "defaultMode": "bypassPermissions", "additionalDirectories": [ diff --git a/harness/codex/config.toml b/harness/codex/config.toml index 9598f43e3..ef621948c 100644 --- a/harness/codex/config.toml +++ b/harness/codex/config.toml @@ -1,5 +1,5 @@ -model = "gpt-5.5" -model_reasoning_effort = "medium" +model = "gpt-5.6-sol" +model_reasoning_effort = "low" personality = "pragmatic" model_verbosity = "low" service_tier = "fast" @@ -31,5 +31,22 @@ env_key = "OPENROUTER_API_KEY" wire_api = "responses" requires_openai_auth = false +[model_providers.responses] +# The name "azure" is load-bearing, not cosmetic: codex strips the `id` field +# from Responses input items (`#[serde(skip_serializing)]` on ResponseItem ids) +# and only re-attaches them via `attach_item_ids` when +# `is_azure_responses_provider(name, base_url)` matches — i.e. name == "azure" +# or an Azure base URL. Meta's Responses endpoint, like Azure's, rejects +# replayed history items without ids (`input[N] missing required field id`), +# so long --meta turns fail once the model round-trips its own prior +# reasoning/tool items. Rename back to a display name once we're on +# codex >= 0.142.0 with `[features] item_ids = true`, which preserves ids for +# all Responses providers. +name = "azure" +base_url = "https://api.ai.meta.com/v1" +env_key = "META_AI_API_KEY" +wire_api = "responses" +requires_openai_auth = false + [projects."/"] trust_level = "trusted" diff --git a/packages/api-client/package.json b/packages/api-client/package.json index 9fe1ff397..57cea2d3e 100644 --- a/packages/api-client/package.json +++ b/packages/api-client/package.json @@ -9,9 +9,7 @@ "test:watch": "vitest" }, "dependencies": { - "@centaur/harness-events": "workspace:*", - "axios": "^1.13.6", - "eventsource-parser": "^3.0.6" + "axios": "^1.13.6" }, "devDependencies": { "typescript": "5.9.3", diff --git a/packages/api-client/src/client.ts b/packages/api-client/src/client.ts index 527d5d781..c7d795625 100644 --- a/packages/api-client/src/client.ts +++ b/packages/api-client/src/client.ts @@ -1,130 +1,62 @@ -import { EventSourceParserStream, type EventSourceMessage } from "eventsource-parser/stream"; import axios, { type AxiosInstance } from "axios"; -export type InputContentBlock = - | { type: "text"; text: string } - | { - type: "image"; - source_path?: string; - source: { type: "base64"; media_type: string; data: string }; - } - | { - type: "document"; - source_path?: string; - source: { type: "base64"; media_type: string; data: string }; - }; - -export interface SpawnOptions { - threadKey: string; - spawnId?: string; - harness?: string; - engine?: string; - personaId?: string; - agentsMdOverride?: string; -} - -export interface SpawnResult { - thread_key: string; - sandbox_id?: string | null; - sandbox_capabilities?: { - repo_cache_enabled: boolean; - observability_enabled: boolean; - } | null; - harness_type: string; - harness_thread_id?: string | null; - persona_id?: string | null; - status: string; - iron_control_principal?: string | null; - created_at?: string | null; - updated_at?: string | null; - harness_switched: boolean; -} - -export interface MessageOptions { - threadKey: string; - assignmentGeneration: number; - messageId?: string; - role?: string; - parts?: InputContentBlock[]; - userId?: string; - metadata?: Record; -} - -export interface ExecuteOptions { - threadKey: string; - assignmentGeneration: number; - executeId?: string; - harness?: string; - platform?: string; - userId?: string; - metadata?: Record; - delivery?: Record; -} - -export interface ExecutionAccepted { - ok: boolean; - execution_id: string; - status: string; - thread_key: string; -} - export interface WorkflowRunOptions { workflowName: string; - triggerKey?: string; + idempotencyKey?: string; input?: Record; - eagerStart?: boolean; + harnessType?: "codex" | "amp" | "claudecode"; + maxAttempts?: number; timeoutMs?: number; } -export interface WorkflowRunAccepted { +export interface WorkflowRunCreated { ok: boolean; run_id: string; + task_id: string; + status: string; + created: boolean; +} + +export interface WorkflowRun { + run_id: string; + task_id: string; workflow_name: string; - workflow_version?: string; - workflow_source_path?: string | null; - parent_run_id?: string | null; - root_run_id?: string | null; status: string; - thread_key?: string | null; - execution_id?: string | null; - output_json?: Record | null; - error_text?: string | null; - latest_checkpoint_name?: string | null; - latest_step_kind?: string | null; - waiting_on?: Record | null; - child_runs_count?: number; - created_at?: string | null; - started_at?: string | null; - completed_at?: string | null; - idempotent?: boolean; + input: unknown; + result: unknown | null; + failure: unknown | null; + attempts: number; + created_at: string; + updated_at: string; } -export interface ThreadMessageRecord { - id: string; - role: string; - parts: Array>; - user_id?: string | null; - metadata?: Record | null; - created_at?: string | null; +export interface ReleaseThreadOptions { + releaseId?: string; + expectedSandboxId?: string; + cancelInflight?: boolean; } -export interface StreamEvent { - eventId: number; - eventKind: string; - data: Record; +export interface ReleaseThreadResponse { + ok: boolean; + thread_key: string; + sandbox_id?: string | null; + release_id?: string | null; + expected_sandbox_id?: string | null; + cancel_inflight: boolean; + sandbox_released: boolean; + sandbox_release_error?: string | null; + execution_id?: string | null; + execution_cancelled: boolean; } export class CentaurClient { readonly http: AxiosInstance; - private log?: { info: Function; warn: Function; error: Function }; constructor(opts: { apiUrl: string; apiKey: string; timeoutMs?: number; - logger?: { info: Function; warn: Function; error: Function }; }) { - this.log = opts.logger; this.http = axios.create({ baseURL: opts.apiUrl, headers: { Authorization: `Bearer ${opts.apiKey}` }, @@ -132,193 +64,71 @@ export class CentaurClient { }); } - private get authHeader(): string { - return (this.http.defaults.headers["Authorization"] ?? - this.http.defaults.headers.common?.["Authorization"]) as string; - } - - async spawn(opts: SpawnOptions): Promise { - const metadata: Record = { - ...(opts.spawnId === undefined ? {} : { spawn_id: opts.spawnId }), - ...(opts.agentsMdOverride === undefined - ? {} - : { agents_md_override: opts.agentsMdOverride }), - }; - const { data } = await this.http.post(`/api/session/${encodeURIComponent(opts.threadKey)}`, { - harness_type: opts.harness ?? opts.engine ?? "codex", - persona_id: opts.personaId, - metadata, - }); - return data as SpawnResult; - } - - async message(opts: MessageOptions): Promise<{ ok: boolean; message_ids: string[] }> { + async startWorkflowRun(opts: WorkflowRunOptions): Promise { const { data } = await this.http.post( - `/api/session/${encodeURIComponent(opts.threadKey)}/messages`, + "/api/workflows/runs", { - messages: [ - { - client_message_id: opts.messageId, - role: opts.role ?? "user", - parts: opts.parts ?? [], - metadata: { - ...(opts.metadata ?? {}), - ...(opts.assignmentGeneration === undefined - ? {} - : { assignment_generation: opts.assignmentGeneration }), - ...(opts.userId === undefined ? {} : { user_id: opts.userId }), - }, - }, - ], + workflow_name: opts.workflowName, + idempotency_key: opts.idempotencyKey, + input: opts.input ?? {}, + harness_type: opts.harnessType, + max_attempts: opts.maxAttempts, }, - ); - return data as { ok: boolean; message_ids: string[] }; - } - - async execute(opts: ExecuteOptions): Promise { - const metadata = { - ...(opts.metadata ?? {}), - ...(opts.assignmentGeneration === undefined - ? {} - : { assignment_generation: opts.assignmentGeneration }), - ...(opts.harness === undefined ? {} : { harness: opts.harness }), - ...(opts.platform === undefined ? {} : { platform: opts.platform }), - ...(opts.userId === undefined ? {} : { user_id: opts.userId }), - ...(opts.delivery === undefined ? {} : { delivery: opts.delivery }), - }; - const { data } = await this.http.post( - `/api/session/${encodeURIComponent(opts.threadKey)}/execute`, { - idempotency_key: opts.executeId, - metadata, - input_lines: [], + timeout: opts.timeoutMs, }, ); - return data as ExecutionAccepted; + return data as WorkflowRunCreated; } - async startWorkflowRun(opts: WorkflowRunOptions): Promise { - const { data } = await this.http.post("/workflows/runs", { - workflow_name: opts.workflowName, - trigger_key: opts.triggerKey, - input: opts.input ?? {}, - eager_start: opts.eagerStart ?? false, - }, { - timeout: opts.timeoutMs, - }); - return data as WorkflowRunAccepted; - } - - async getWorkflowRun(runId: string): Promise { - const { data } = await this.http.get(`/workflows/runs/${encodeURIComponent(runId)}`); - return data as WorkflowRunAccepted; + async getWorkflowRun(runId: string): Promise<{ ok: boolean; run: WorkflowRun }> { + const { data } = await this.http.get(`/api/workflows/runs/${encodeURIComponent(runId)}`); + return data as { ok: boolean; run: WorkflowRun }; } async listWorkflowRuns(opts?: { workflowName?: string; threadKey?: string; - status?: string; - parentRunId?: string; limit?: number; - }): Promise<{ ok: boolean; items: WorkflowRunAccepted[] }> { - const { data } = await this.http.get("/workflows/runs", { + }): Promise<{ ok: boolean; runs: WorkflowRun[] }> { + const { data } = await this.http.get("/api/workflows/runs", { params: { workflow_name: opts?.workflowName, thread_key: opts?.threadKey, - status: opts?.status, - parent_run_id: opts?.parentRunId, limit: opts?.limit, }, }); - return data as { ok: boolean; items: WorkflowRunAccepted[] }; + return data as { ok: boolean; runs: WorkflowRun[] }; } - async getWorkflowChildren(runId: string, limit = 200): Promise<{ ok: boolean; items: WorkflowRunAccepted[] }> { - return this.listWorkflowRuns({ parentRunId: runId, limit }); + async cancelWorkflowRun(runId: string): Promise<{ ok: boolean; status: "cancelled" }> { + const { data } = await this.http.post(`/api/workflows/runs/${encodeURIComponent(runId)}/cancel`); + return data as { ok: boolean; status: "cancelled" }; } - async cancelWorkflowRun(runId: string): Promise { - const { data } = await this.http.post(`/workflows/runs/${encodeURIComponent(runId)}/cancel`); - return data as WorkflowRunAccepted; + async releaseThread( + threadKey: string, + opts: ReleaseThreadOptions = {}, + ): Promise { + const { data } = await this.http.post( + `/api/session/${encodeURIComponent(threadKey)}/release`, + { + release_id: opts.releaseId, + expected_sandbox_id: opts.expectedSandboxId, + cancel_inflight: opts.cancelInflight ?? false, + }, + ); + return data as ReleaseThreadResponse; } async sendWorkflowEvent(opts: { eventName: string; payload?: Record; }): Promise> { - const { data } = await this.http.post("/workflows/events", { + const { data } = await this.http.post("/api/workflows/events", { event_name: opts.eventName, payload: opts.payload ?? {}, }); return data as Record; } - - async *streamEvents(opts: { - threadKey: string; - afterEventId?: number; - executionId?: string; - pollMs?: number; - signal?: AbortSignal; - }): AsyncGenerator { - const params = new URLSearchParams(); - if (opts.afterEventId !== undefined) params.set("after_event_id", String(opts.afterEventId)); - if (opts.executionId) params.set("execution_id", opts.executionId); - if (opts.pollMs !== undefined) params.set("poll_ms", String(opts.pollMs)); - - const url = new URL( - `/api/session/${encodeURIComponent(opts.threadKey)}/events`, - this.ensureBaseUrl(), - ); - for (const [key, value] of params) url.searchParams.set(key, value); - const res = await fetch(url.toString(), { - method: "GET", - headers: { - Authorization: this.authHeader, - "X-Centaur-Thread-Key": opts.threadKey, - }, - signal: opts.signal, - }); - - if (!res.ok) { - const text = await res.text().catch(() => ""); - throw new Error(`/api/session/{thread}/events failed (${res.status}): ${text.slice(0, 300)}`); - } - if (!res.body) return; - - const stream = (res.body as ReadableStream) - .pipeThrough(new TextDecoderStream() as unknown as TransformStream) - .pipeThrough(new EventSourceParserStream()); - - for await (const event of stream as unknown as AsyncIterable) { - if (!event.data || event.data === "[DONE]") continue; - let parsed: Record = { type: "unknown", raw: event.data }; - try { - parsed = JSON.parse(event.data) as Record; - } catch { - // keep raw fallback - } - yield { - eventId: Number(event.id || 0), - eventKind: event.event || "message", - data: parsed, - }; - } - } - - async releaseThread(threadKey: string, opts?: { releaseId?: string; cancelInflight?: boolean }) { - const { data } = await this.http.post( - `/api/session/${encodeURIComponent(threadKey)}/release`, - { - release_id: opts?.releaseId, - cancel_inflight: opts?.cancelInflight ?? false, - }, - ); - return data as Record; - } - - private ensureBaseUrl(): string { - const baseURL = this.http.defaults.baseURL; - if (!baseURL) throw new Error("CentaurClient apiUrl is required"); - return baseURL.endsWith("/") ? baseURL : `${baseURL}/`; - } } diff --git a/packages/api-client/src/index.ts b/packages/api-client/src/index.ts index 9a341652c..447ee3559 100644 --- a/packages/api-client/src/index.ts +++ b/packages/api-client/src/index.ts @@ -1,10 +1,9 @@ export { ApiError } from "./types"; export { CentaurClient } from "./client"; export type { - ExecuteOptions, - MessageOptions, - InputContentBlock, - ThreadMessageRecord, + ReleaseThreadOptions, + ReleaseThreadResponse, WorkflowRunOptions, - WorkflowRunAccepted, + WorkflowRun, + WorkflowRunCreated, } from "./client"; diff --git a/packages/api-client/test/client.test.ts b/packages/api-client/test/client.test.ts index 39ed55d35..30bc8cf32 100644 --- a/packages/api-client/test/client.test.ts +++ b/packages/api-client/test/client.test.ts @@ -1,206 +1,122 @@ import { afterEach, describe, expect, it, vi } from "vitest"; -import { CentaurClient, type StreamEvent } from "../src/client"; - -async function collectEvents(events: AsyncIterable): Promise { - const collected: StreamEvent[] = []; - for await (const event of events) { - collected.push(event); - } - return collected; -} - -function sseResponse(body: string, init?: ResponseInit): Response { - return new Response( - new ReadableStream({ - start(controller) { - controller.enqueue(new TextEncoder().encode(body)); - controller.close(); - }, - }), - { - status: 200, - headers: { "Content-Type": "text/event-stream" }, - ...init, - }, - ); -} +import { CentaurClient } from "../src/client"; describe("CentaurClient", () => { afterEach(() => { vi.restoreAllMocks(); - vi.unstubAllGlobals(); }); - it("parses SSE ids, events, JSON data, [DONE], and invalid JSON payloads", async () => { - const fetchMock = vi.fn(async () => sseResponse([ - "id: 11", - "event: amp_raw_event", - 'data: {"type":"assistant","message":{"content":"hello"}}', - "", - "id: 12", - "event: done", - "data: [DONE]", - "", - "id: 13", - "data: not-json", - "", - "", - ].join("\n"))); - vi.stubGlobal("fetch", fetchMock); - + it("starts workflow runs through the workflow API", async () => { const client = new CentaurClient({ apiUrl: "http://api.local", apiKey: "test-key", }); + const postMock = vi.spyOn(client.http, "post").mockResolvedValue({ + data: { ok: true, run_id: "run-123", task_id: "task-123", status: "queued", created: true }, + }); - await expect(collectEvents(client.streamEvents({ threadKey: "thread-1" }))).resolves.toEqual([ - { - eventId: 11, - eventKind: "amp_raw_event", - data: { type: "assistant", message: { content: "hello" } }, - }, + await expect( + client.startWorkflowRun({ + workflowName: "nightly", + idempotencyKey: "trigger-1", + input: { topic: "incidents" }, + harnessType: "codex", + maxAttempts: 3, + timeoutMs: 5000, + }), + ).resolves.toMatchObject({ run_id: "run-123" }); + + expect(postMock).toHaveBeenCalledWith( + "/api/workflows/runs", { - eventId: 13, - eventKind: "message", - data: { type: "unknown", raw: "not-json" }, + workflow_name: "nightly", + idempotency_key: "trigger-1", + input: { topic: "incidents" }, + harness_type: "codex", + max_attempts: 3, }, - ]); + { timeout: 5000 }, + ); }); - it("URL encodes Slack thread keys in event stream URLs", async () => { - const fetchMock = vi.fn(async () => sseResponse("")); - vi.stubGlobal("fetch", fetchMock); + it("reads and mutates workflow runs through workflow endpoints", async () => { const client = new CentaurClient({ apiUrl: "http://api.local", apiKey: "test-key", }); + const getMock = vi.spyOn(client.http, "get").mockResolvedValue({ + data: { ok: true, run_id: "run:123", workflow_name: "nightly", status: "completed" }, + }); + const postMock = vi.spyOn(client.http, "post").mockResolvedValue({ + data: { ok: true, run_id: "run:123", workflow_name: "nightly", status: "cancelled" }, + }); - await collectEvents(client.streamEvents({ - threadKey: "slack:T123:C123:1700000000.000100", - executionId: "exe-1", - afterEventId: 42, - pollMs: 250, - })); + await client.getWorkflowRun("run:123"); + await client.listWorkflowRuns({ + workflowName: "nightly", + threadKey: "slack:C:1", + limit: 5, + }); + await client.cancelWorkflowRun("run:123"); - expect(fetchMock).toHaveBeenCalledWith( - "http://api.local/api/session/slack%3AT123%3AC123%3A1700000000.000100/events?after_event_id=42&execution_id=exe-1&poll_ms=250", - expect.objectContaining({ - method: "GET", - headers: { - Authorization: "Bearer test-key", - "X-Centaur-Thread-Key": "slack:T123:C123:1700000000.000100", - }, - }), - ); + expect(getMock).toHaveBeenNthCalledWith(1, "/api/workflows/runs/run%3A123"); + expect(getMock).toHaveBeenNthCalledWith(2, "/api/workflows/runs", { + params: { + workflow_name: "nightly", + thread_key: "slack:C:1", + limit: 5, + }, + }); + expect(postMock).toHaveBeenCalledWith("/api/workflows/runs/run%3A123/cancel"); }); - it("uses session API routes for path-based session calls", async () => { + it("sends workflow events", async () => { const client = new CentaurClient({ apiUrl: "http://api.local", apiKey: "test-key", }); const postMock = vi.spyOn(client.http, "post").mockResolvedValue({ data: { ok: true } }); - const threadKey = "slack:T123:C123:1700000000.000100"; - await client.spawn({ - threadKey, - harness: "codex", - spawnId: "spawn:1", - personaId: "persona-1", - agentsMdOverride: "custom instructions", - }); - await client.message({ - threadKey, - assignmentGeneration: 3, - messageId: "msg:1", - parts: [{ type: "text", text: "hello" }], - userId: "U123", - metadata: { platform: "slack" }, - }); - await client.execute({ - threadKey, - assignmentGeneration: 3, - executeId: "exec:1", - harness: "codex", - platform: "slack", - userId: "U123", - metadata: { source: "test" }, - }); - await client.releaseThread(threadKey, { - releaseId: "release:1", - cancelInflight: true, + await client.sendWorkflowEvent({ + eventName: "approval.received", + payload: { approved: true, correlation_id: "corr-1" }, }); - expect(postMock).toHaveBeenNthCalledWith( - 1, - "/api/session/slack%3AT123%3AC123%3A1700000000.000100", - { - harness_type: "codex", - persona_id: "persona-1", - metadata: { - spawn_id: "spawn:1", - agents_md_override: "custom instructions", - }, - }, - ); - expect(postMock).toHaveBeenNthCalledWith( - 2, - "/api/session/slack%3AT123%3AC123%3A1700000000.000100/messages", - { - messages: [ - { - client_message_id: "msg:1", - role: "user", - parts: [{ type: "text", text: "hello" }], - metadata: { - platform: "slack", - assignment_generation: 3, - user_id: "U123", - }, - }, - ], - }, - ); - expect(postMock).toHaveBeenNthCalledWith( - 3, - "/api/session/slack%3AT123%3AC123%3A1700000000.000100/execute", - { - idempotency_key: "exec:1", - metadata: { - source: "test", - assignment_generation: 3, - harness: "codex", - platform: "slack", - user_id: "U123", - }, - input_lines: [], - }, - ); - expect(postMock).toHaveBeenNthCalledWith( - 4, - "/api/session/slack%3AT123%3AC123%3A1700000000.000100/release", - { - release_id: "release:1", - cancel_inflight: true, - }, - ); + expect(postMock).toHaveBeenCalledWith("/api/workflows/events", { + event_name: "approval.received", + payload: { approved: true, correlation_id: "corr-1" }, + }); }); - it("throws useful errors for non-OK event stream responses", async () => { - vi.stubGlobal("fetch", vi.fn(async () => new Response( - "upstream unavailable", - { status: 503, statusText: "Service Unavailable" }, - ))); + it("releases a session through the canonical owner-fenced endpoint", async () => { const client = new CentaurClient({ apiUrl: "http://api.local", apiKey: "test-key", }); + const postMock = vi.spyOn(client.http, "post").mockResolvedValue({ + data: { + ok: true, + thread_key: "slack:T:C:1.2", + cancel_inflight: true, + sandbox_released: true, + execution_cancelled: true, + }, + }); - await expect( - collectEvents(client.streamEvents({ threadKey: "slack:T123:C123:1700000000.000100" })), - ).rejects.toThrow( - "/api/session/{thread}/events failed (503): upstream unavailable", + await client.releaseThread("slack:T:C:1.2", { + releaseId: "rel-123", + expectedSandboxId: "asbx-reviewed", + cancelInflight: true, + }); + + expect(postMock).toHaveBeenCalledWith( + "/api/session/slack%3AT%3AC%3A1.2/release", + { + release_id: "rel-123", + expected_sandbox_id: "asbx-reviewed", + cancel_inflight: true, + }, ); }); }); diff --git a/packages/rendering/src/chat-sdk.test.ts b/packages/rendering/src/chat-sdk.test.ts index e0bd2e53a..4d66c2008 100644 --- a/packages/rendering/src/chat-sdk.test.ts +++ b/packages/rendering/src/chat-sdk.test.ts @@ -61,6 +61,17 @@ describe('ChatSDKRenderer', () => { ]) }) + it('treats status updates as renderer side effects only', () => { + const renderer = new ChatSDKRenderer() + + expect( + renderer.render('session-1', { + type: 'renderer.status', + status: 'The agent is inspecting events.' + }) + ).toEqual([]) + }) + it('bounds large task details while preserving full task output', () => { const renderer = new ChatSDKRenderer() const largeDetails = 'd'.repeat(10000) diff --git a/packages/rendering/src/chat-sdk.ts b/packages/rendering/src/chat-sdk.ts index 0be7f10a0..a6eb57c95 100644 --- a/packages/rendering/src/chat-sdk.ts +++ b/packages/rendering/src/chat-sdk.ts @@ -61,7 +61,7 @@ export class ChatSDKRenderer implements RendererInterface { return [] } if (event.type === 'renderer.status') { - return [{ type: 'chat.message.upsert', message: { text: event.status } }] + return [] } if (event.type === 'renderer.message.delta') { return [ diff --git a/packages/rendering/src/codex-app-server.test.ts b/packages/rendering/src/codex-app-server.test.ts index 2df4cf634..7b3b117a3 100644 --- a/packages/rendering/src/codex-app-server.test.ts +++ b/packages/rendering/src/codex-app-server.test.ts @@ -4,7 +4,6 @@ import { codexAppServerToChatSdkStream, codexAppServerToRendererEvents } from './codex-app-server' -import type { RendererTaskBlock } from './types' describe('CodexAppServerRendererEventMapper', () => { it('maps final answer deltas to generic renderer message deltas after activity exists', () => { @@ -53,18 +52,18 @@ describe('CodexAppServerRendererEventMapper', () => { }) }) - it('maps commentary to Thinking task updates instead of message deltas', () => { + it('suppresses commentary thinking blocks', () => { const mapper = new CodexAppServerRendererEventMapper() - mapper.process({ + expect(mapper.process({ type: 'item.started', item: { id: 'thinking-1', type: 'agentMessage', phase: 'commentary' } - }) - mapper.process({ + })).toEqual([]) + expect(mapper.process({ type: 'item.agentMessage.delta', itemId: 'thinking-1', delta: 'Checking the runtime.' - }) + })).toEqual([]) const events = mapper.process({ type: 'item.completed', @@ -77,31 +76,18 @@ describe('CodexAppServerRendererEventMapper', () => { }) expect(events.some(event => event.type === 'renderer.message.delta')).toBe(false) - const task = events.find(event => event.type === 'renderer.task.update') - // Sealed commentary stays in_progress until the next activity starts so - // "Thinking completed" never headlines the Slack plan card mid-turn. - expect(task).toMatchObject({ - type: 'renderer.task.update', - task: { - id: 'thinking-thinking-1', - title: 'Thinking', - status: 'in_progress' - } - }) - expect(plain(task?.type === 'renderer.task.update' ? task.task.details : undefined)).toContain( - 'Checking the runtime.' - ) + expect(events.some(event => event.type === 'renderer.task.update')).toBe(false) const next = mapper.process({ type: 'item.started', item: { id: 'cmd-1', type: 'commandExecution', command: 'pnpm test' } }) expect(next.find(event => event.type === 'renderer.task.update')).toMatchObject({ - task: { id: 'thinking-thinking-1', title: 'Thinking', status: 'complete' } + task: { id: 'cmd-1', title: '1. Command execution', status: 'in_progress' } }) }) - it('keeps one Thinking task in_progress across reasoning deltas until the item seals', () => { + it('suppresses reasoning thinking blocks', () => { const mapper = new CodexAppServerRendererEventMapper() const first = mapper.process({ @@ -109,47 +95,28 @@ describe('CodexAppServerRendererEventMapper', () => { itemId: 'reasoning-1', delta: 'Inspecting the ' }) - expect(first).toContainEqual({ - type: 'renderer.task.update', - task: { - id: 'reasoning-1', - title: 'Thinking', - status: 'in_progress', - details: [{ type: 'text', text: 'Inspecting the' }], - output: undefined - }, - flush: true - }) + expect(first).toEqual([]) - // A command starting mid-thought must not flip the Thinking task to complete. - mapper.process({ + const command = mapper.process({ type: 'item.started', item: { id: 'cmd-1', type: 'commandExecution', command: 'pnpm test' } }) + expect(command.find(event => event.type === 'renderer.task.update')).toMatchObject({ + task: { id: 'cmd-1', title: '1. Command execution', status: 'in_progress' } + }) const second = mapper.process({ type: 'item.reasoning.textDelta', itemId: 'reasoning-1', delta: 'event stream' }) - const secondUpdate = second.find(event => event.type === 'renderer.task.update') - expect(secondUpdate).toMatchObject({ - task: { id: 'reasoning-1', title: 'Thinking', status: 'in_progress' } - }) - expect( - plain(secondUpdate?.type === 'renderer.task.update' ? secondUpdate.task.details : undefined) - ).toContain('Inspecting the event stream') + expect(second.some(event => event.type === 'renderer.task.update')).toBe(false) - // Sealing completes the Thinking task; the still-running command keeps - // the plan in an in-progress state so the Slack header tracks it. const sealed = mapper.process({ type: 'item.completed', item: { id: 'reasoning-1', type: 'reasoning', content: ['Inspecting the event stream'] } }) - const sealedUpdate = sealed.find(event => event.type === 'renderer.task.update') - expect(sealedUpdate).toMatchObject({ - task: { id: 'reasoning-1', title: 'Thinking', status: 'complete' } - }) + expect(sealed.some(event => event.type === 'renderer.task.update')).toBe(false) }) it('holds the last finished task in_progress so the Slack header never claims completion mid-turn', () => { @@ -161,8 +128,8 @@ describe('CodexAppServerRendererEventMapper', () => { }) // The command finishes, leaving nothing else running. Slack would show - // "Thinking completed" for an all-complete plan, so the completion is - // held back and the task stays presented as in_progress. + // a completed-task header, so the completion is held back and the task + // stays presented as in_progress. const completed = mapper.process({ type: 'item.completed', item: { @@ -205,30 +172,22 @@ describe('CodexAppServerRendererEventMapper', () => { ) }) - it('separates Codex reasoning summary sections within one Thinking task', () => { + it('suppresses Codex reasoning summary sections', () => { const mapper = new CodexAppServerRendererEventMapper() - mapper.process({ + expect(mapper.process({ type: 'item.reasoning.summaryTextDelta', itemId: 'reasoning-1', summaryIndex: 0, delta: 'First section.' - }) + })).toEqual([]) const events = mapper.process({ type: 'item.reasoning.summaryTextDelta', itemId: 'reasoning-1', summaryIndex: 1, delta: 'Second section.' }) - const update = events.find(event => event.type === 'renderer.task.update') - expect(update).toMatchObject({ - task: { - id: 'reasoning-1', - title: 'Thinking', - status: 'in_progress', - details: [{ type: 'text', text: 'First section.\n\nSecond section.' }] - } - }) + expect(events).toEqual([]) }) it('parses Rust session output lines before mapping app-server notifications', () => { @@ -290,6 +249,24 @@ describe('CodexAppServerRendererEventMapper', () => { }) }) + it('maps Rust activity summary events to renderer status updates', () => { + const mapper = new CodexAppServerRendererEventMapper() + const events = mapper.process({ + eventKind: 'session.activity_summary', + data: { + execution_id: 'exe-1', + summary: 'The agent is inspecting App Server events.' + } + }) + + expect(events).toEqual([ + { + type: 'renderer.status', + status: 'The agent is inspecting App Server events.' + } + ]) + }) + it('maps app-server agent message deltas keyed by turnId', () => { const mapper = new CodexAppServerRendererEventMapper() const events = mapper.process({ @@ -408,23 +385,13 @@ describe('CodexAppServerRendererEventMapper', () => { title: 'Inspect App Server events', status: 'complete' }) - expect(chunks).toContainEqual({ - type: 'task_update', - id: 'reasoning-1', - title: 'Thinking', - status: 'in_progress', - details: 'Inspecting the event stream' - }) - expect(chunks).toContainEqual({ - type: 'task_update', - id: 'reasoning-1', - title: 'Thinking', - status: 'complete' - }) + expect(chunks.some(chunk => chunk.type === 'task_update' && chunk.title === 'Thinking')).toBe( + false + ) expect(chunks).toContainEqual({ type: 'markdown_text', text: 'Done.' }) }) - it('coalesces repeated reasoning deltas into one Thinking task', async () => { + it('suppresses repeated reasoning deltas from Chat SDK output', async () => { const chunks = await collect( codexAppServerToChatSdkStream( toAsyncIterable([ @@ -452,24 +419,9 @@ describe('CodexAppServerRendererEventMapper', () => { ) ) - const thinkingChunks = chunks.filter( - (chunk): chunk is Extract<(typeof chunks)[number], { type: 'task_update' }> => - chunk.type === 'task_update' && chunk.title === 'Thinking' + expect(chunks.some(chunk => chunk.type === 'task_update' && chunk.title === 'Thinking')).toBe( + false ) - expect(new Set(thinkingChunks.map(chunk => chunk.id))).toEqual(new Set(['reasoning-1'])) - expect(thinkingChunks).toContainEqual({ - type: 'task_update', - id: 'reasoning-1', - title: 'Thinking', - status: 'in_progress', - details: 'Inspecting the event stream' - }) - expect(thinkingChunks).toContainEqual({ - type: 'task_update', - id: 'reasoning-1', - title: 'Thinking', - status: 'complete' - }) }) it('streams command details once and command output incrementally', async () => { @@ -783,13 +735,31 @@ describe('CodexAppServerRendererEventMapper', () => { error: 'sandbox exited' }) }) -}) -function plain(elements: RendererTaskBlock[] | undefined): string { - return (elements ?? []) - .map(element => element.text) - .join('') -} + it('emits interrupted final text for cancelled Rust sessions', async () => { + const chunks = await collect( + codexAppServerToChatSdkStream( + toAsyncIterable([ + { + type: 'item.started', + item: { id: 'cmd-1', type: 'commandExecution', command: 'sleep 60' } + }, + { + eventKind: 'session.execution_cancelled', + data: { error: 'Execution interrupted' } + } + ]) + ) + ) + + expect(chunks.filter(chunk => chunk.type === 'markdown_text')).toEqual([ + { + type: 'markdown_text', + text: 'Execution interrupted' + } + ]) + }) +}) async function collect(source: AsyncIterable): Promise { const out: T[] = [] diff --git a/packages/rendering/src/codex-app-server.ts b/packages/rendering/src/codex-app-server.ts index f6ce04025..a8fe0885a 100644 --- a/packages/rendering/src/codex-app-server.ts +++ b/packages/rendering/src/codex-app-server.ts @@ -59,8 +59,6 @@ type CodexMapperState = { agentMessagePhase: AgentMessagePhase | null agentMessagePhaseByItemId: Map planText: string - reasoningTextByItemId: Map - reasoningSummaryIndexByItemId: Map taskByUseId: Map commandOutputById: Map emittedActivityRunByTaskId: Map @@ -74,6 +72,12 @@ export type CodexAppServerRendererEventMapperOptions = { logInfo?: RendererLogInfo unknownAgentMessagePhase?: AgentMessagePhase taskOutput?: 'full' | 'omit' + // How long buffered assistant text waits for plan/tasks to arrive before it + // streams anyway. The check is event-driven — with no tasks and no further + // events, buffered text sits until the next event or stream end — so + // consumers that stream text into their own surface (discordbot) pass 0 to + // emit deltas immediately. Default 500ms (Slack card-first rendering). + preStreamGraceMs?: number } export type CodexAppServerToChatStreamOptions = CodexAppServerRendererEventMapperOptions & { @@ -89,12 +93,14 @@ export class CodexAppServerRendererEventMapper private readonly logInfo?: RendererLogInfo private readonly unknownAgentMessagePhase: AgentMessagePhase private readonly includeTaskOutput: boolean + private readonly preStreamGraceMs: number constructor(options: CodexAppServerRendererEventMapperOptions = {}) { this.sessionId = options.sessionId ?? '' this.logInfo = options.logInfo this.unknownAgentMessagePhase = options.unknownAgentMessagePhase ?? 'final_answer' this.includeTaskOutput = options.taskOutput === 'full' + this.preStreamGraceMs = options.preStreamGraceMs ?? PRE_STREAM_GRACE_MS } process(source: ServerNotification | RustSessionStreamEvent | unknown): RendererEvent[] { @@ -103,6 +109,8 @@ export class CodexAppServerRendererEventMapper const rustMapped = rustSessionEventToServerNotification(source) if (rustMapped?.kind === 'failed') return this.fail(rustMapped.error) if (rustMapped?.kind === 'completed') return this.complete(rustMapped.resultText) + if (rustMapped?.kind === 'status') + return [{ type: 'renderer.status', status: rustMapped.status }] if (rustMapped?.kind === 'notification') return this.processNotification(rustMapped.notification) if (!isRecord(source)) return [] @@ -114,7 +122,6 @@ export class CodexAppServerRendererEventMapper if (this.state.done) return [] this.state.done = true const out: RendererEvent[] = [] - completeThinkingTasks(this.state) completeOpenTasks(this.state) this.emitActivitySummary(out, { final: true }) this.ensureFinalAnswerText() @@ -171,9 +178,6 @@ export class CodexAppServerRendererEventMapper trackAgentMessageLifecycle(event, this.state) ensureCommentarySegmentBreak(event, this.state) - if (startThinkingTask(this.state, event)) { - this.emitActivitySummary(out) - } const structuredPlan = structuredPlanUpdate(event) if (structuredPlan) { @@ -295,71 +299,6 @@ export class CodexAppServerRendererEventMapper if (update.correction) { this.logCanonicalCorrection(event, update.correction) } - if (buffer === 'commentary' && event?.type === 'item.completed') { - upsertThinkingTask(this.state, event) - this.emitActivitySummary(out) - } - } - - const reasoningMessage = reasoningText(event) - if (reasoningMessage.trim()) { - const itemId = reasoningEventItemId(event) - if (isReasoningDeltaEvent(event) && itemId) { - // Accumulate deltas into one task per reasoning item and keep it - // in_progress until the item seals (item.completed) or the - // execution finishes (flush). Completing earlier makes the Slack - // plan card flip between "Thinking", "Thinking completed", and the - // running command. - const previous = this.state.reasoningTextByItemId.get(itemId) ?? '' - const summaryIndex = reasoningSummaryIndex(event) - const needsBreak = - summaryIndex !== undefined && - this.state.reasoningSummaryIndexByItemId.get(itemId) !== undefined && - this.state.reasoningSummaryIndexByItemId.get(itemId) !== summaryIndex && - previous.trim() !== '' - if (summaryIndex !== undefined) { - this.state.reasoningSummaryIndexByItemId.set(itemId, summaryIndex) - } - const accumulated = previous + (needsBreak ? '\n\n' : '') + reasoningMessage - this.state.reasoningTextByItemId.set(itemId, accumulated) - this.state.taskByUseId.set(itemId, { - id: itemId, - title: 'Thinking', - status: 'in_progress', - details: [section([text(accumulated.trim())])], - output: [] - }) - } else { - const id = itemId || `reasoning-${++this.state.stepCounter}` - this.state.taskByUseId.set(id, { - id, - title: 'Thinking', - status: 'complete', - details: [section([text(reasoningMessage.trim())])], - output: [] - }) - } - this.emitActivitySummary(out) - } - - const sealedReasoning = completedReasoningItem(event) - if (sealedReasoning) { - const id = String(sealedReasoning.id ?? '') - const accumulated = id ? this.state.reasoningTextByItemId.get(id) ?? '' : '' - const finalText = (reasoningItemText(sealedReasoning) || accumulated).trim() - const existing = id ? this.state.taskByUseId.get(id) : undefined - if (id && (existing || finalText)) { - this.state.taskByUseId.set(id, { - id, - title: 'Thinking', - status: 'complete', - details: finalText ? [section([text(finalText)])] : existing?.details ?? [], - output: [] - }) - this.state.reasoningTextByItemId.delete(id) - this.state.reasoningSummaryIndexByItemId.delete(id) - this.emitActivitySummary(out) - } } if (isTerminalCodexAppServerEvent(event)) { @@ -456,7 +395,7 @@ export class CodexAppServerRendererEventMapper const hasPlan = this.state.taskByUseId.size > 0 const graceExpired = this.state.firstBufferedTextAt !== null && - Date.now() - this.state.firstBufferedTextAt >= PRE_STREAM_GRACE_MS + Date.now() - this.state.firstBufferedTextAt >= this.preStreamGraceMs const canStream = hasPlan || opts.force || graceExpired if (!canStream) return @@ -694,6 +633,7 @@ export type RustSessionMappingResult = | { kind: 'notification'; notification: ServerNotification } | { kind: 'failed'; error: string } | { kind: 'completed'; resultText?: string } + | { kind: 'status'; status: string } | null export function rustSessionEventToServerNotification(source: unknown): RustSessionMappingResult { @@ -721,6 +661,12 @@ export function rustSessionEventToServerNotification(source: unknown): RustSessi } } + if (eventKind === 'session.activity_summary') { + const data = isRecord(source.data) ? source.data : source + const status = String(data.summary ?? data.status ?? '').trim() + return status ? { kind: 'status', status } : null + } + if ( eventKind === 'session.execution_failed' || eventKind === 'session.stream_error' || @@ -730,10 +676,17 @@ export function rustSessionEventToServerNotification(source: unknown): RustSessi return { kind: 'failed', error: String(data.error ?? 'Execution failed') } } - if ( - eventKind === 'session.execution_completed' || - eventKind === 'session.execution_cancelled' - ) { + if (eventKind === 'session.execution_cancelled') { + const data = isRecord(source.data) ? source.data : source + const resultText = + terminalResultText(data).trim() || String(data.error ?? 'Execution interrupted').trim() + return { + kind: 'completed', + ...(resultText ? { resultText } : {}) + } + } + + if (eventKind === 'session.execution_completed') { const data = isRecord(source.data) ? source.data : source const resultText = terminalResultText(data).trim() return { @@ -772,8 +725,6 @@ function newState(): CodexMapperState { agentMessagePhase: null, agentMessagePhaseByItemId: new Map(), planText: '', - reasoningTextByItemId: new Map(), - reasoningSummaryIndexByItemId: new Map(), taskByUseId: new Map(), commandOutputById: new Map(), emittedActivityRunByTaskId: new Map(), @@ -888,49 +839,6 @@ function lastInsertedKey(map: Map): K | undefined { return last } -function commentaryItemId(event: any): string { - return String(event?.itemId ?? event?.item_id ?? event?.item?.id ?? '') -} - -function startThinkingTask(state: CodexMapperState, event: any): boolean { - if (event?.type !== 'item.started') return false - if (agentMessageItemPhase(event?.item) !== 'commentary') return false - const id = commentaryItemId(event) - if (!id || state.taskByUseId.has(`thinking-${id}`)) return false - state.taskByUseId.set(`thinking-${id}`, { - id: `thinking-${id}`, - title: 'Thinking', - status: 'in_progress', - details: [], - output: [] - }) - return true -} - -function upsertThinkingTask(state: CodexMapperState, event: any): void { - const id = commentaryItemId(event) - if (!id) return - const body = String(event?.item?.text ?? state.commentaryByItemId.get(id) ?? '').trim() - if (!body) return - if (state.commentaryByItemId.get(id) !== body) { - state.commentaryByItemId.set(id, body) - recomposeBuffers(state) - } - state.taskByUseId.set(`thinking-${id}`, { - id: `thinking-${id}`, - title: 'Thinking', - status: 'complete', - details: [section([text(body)])], - output: [] - }) -} - -function completeThinkingTasks(state: CodexMapperState): void { - for (const [id, body] of state.commentaryByItemId) { - upsertThinkingTask(state, { item: { id, text: body } }) - } -} - function eventCarriesAgentMessageText(event: any): boolean { if (event?.type === 'item.agentMessage.delta') return Boolean(extractDeltaText(event)) if (event?.type === 'assistant') return Boolean(assistantTextFromAssistantEvent(event)) @@ -987,52 +895,6 @@ function textHash(value: string): string { return (hash >>> 0).toString(16).padStart(8, '0') } -function reasoningText(event: any): string { - if ( - event?.type === 'item.reasoning.summaryTextDelta' || - event?.type === 'item.reasoning.textDelta' - ) { - return String(event.delta ?? '') - } - if (event?.type !== 'reasoning') return '' - return String(event.text ?? event.thinking ?? '') -} - -function isReasoningDeltaEvent(event: any): boolean { - return ( - event?.type === 'item.reasoning.summaryTextDelta' || - event?.type === 'item.reasoning.textDelta' - ) -} - -function reasoningEventItemId(event: any): string { - return String(event?.itemId ?? event?.item_id ?? '') -} - -function reasoningSummaryIndex(event: any): number | undefined { - const value = event?.summaryIndex ?? event?.summary_index - return typeof value === 'number' ? value : undefined -} - -function completedReasoningItem(event: any): Record | null { - if (event?.type !== 'item.completed') return null - const item = event.item - if (!item || item.type !== 'reasoning') return null - return item -} - -function reasoningItemText(item: any): string { - const parts = [ - ...(Array.isArray(item?.content) ? item.content : []), - ...(Array.isArray(item?.summary) ? item.summary : []) - ] - const texts = parts - .map(part => (typeof part === 'string' ? part : String(part?.text ?? ''))) - .filter(part => part.trim()) - if (texts.length) return texts.join('\n\n') - return String(item?.text ?? '') -} - function terminalResultText(event: any): string { for (const key of ['result', 'result_text', 'text', 'final_text']) { const value = event?.[key] @@ -1118,7 +980,7 @@ function parsePlanText(value: string): Array<{ step: string; status: RendererTas .split('\n') .map(line => { const trimmed = line.trim() - if (!isPlanListLine(trimmed)) return null + if (!/^[-*]\s+|\d+[.)]\s+/.test(trimmed)) return null return { step: trimmed, status: /\[[xX]\]/.test(trimmed) ? ('complete' as const) : ('pending' as const) @@ -1127,23 +989,6 @@ function parsePlanText(value: string): Array<{ step: string; status: RendererTas .filter(item => item !== null) } -function isPlanListLine(value: string): boolean { - if (value.startsWith('- ') || value.startsWith('* ')) return true - let index = 0 - while (index < value.length && isAsciiDigit(value.charCodeAt(index))) index += 1 - if (index === 0) return false - const marker = value[index] - return (marker === '.' || marker === ')') && isWhitespace(value.charCodeAt(index + 1)) -} - -function isAsciiDigit(code: number): boolean { - return code >= 48 && code <= 57 -} - -function isWhitespace(code: number): boolean { - return code === 32 || code === 9 -} - function planStatus(value: string | undefined): RendererTaskStatus { const status = String(value ?? '').toLowerCase() if (status === 'inprogress' || status === 'in_progress' || status === 'running') @@ -1203,12 +1048,10 @@ function changedActivityTaskUpdates( output?: RendererTaskBlock[] }> = [] // Slack derives the plan card header from task statuses: it shows the - // current in_progress task, and falls back to "Thinking completed" when - // nothing is in progress — even mid-turn (e.g. while the model thinks - // between commands without emitting reasoning events). Mid-turn, present - // the most recent finished task as still in progress so the header never - // claims completion; its true status is emitted with the next batch or at - // the final flush. + // current in_progress task, and falls back to a completed-task header when + // nothing is in progress. Mid-turn, present the most recent finished task as + // still in progress so the header never claims completion; its true status + // is emitted with the next batch or at the final flush. const report = opts.final ? tasks : holdLastFinishedTask(tasks) for (const task of report) { let details: RendererTaskBlock[] | undefined @@ -1263,9 +1106,6 @@ function holdLastFinishedTask(tasks: HarnessTask[]): HarnessTask[] { } function activityRunBlock(task: HarnessTask): RendererTaskBlock[] { - if (task.title === 'Thinking' && task.details.length) { - return task.details - } const command = firstPreformattedBody(task.details) if (command) { return [pre(command, shellLanguage(firstPreformattedLanguage(task.details)))] @@ -1574,53 +1414,16 @@ function languageFromPath(path: string): string { function languageFromContent(value: string): string { const trimmed = value.trim() - if (trimmed.split('\n').some(line => isTypescriptDeclaration(line.trimStart()))) return 'ts' + if ( + /^(export\s+)?(async\s+)?function\s|^type\s+\w+\s*=|^interface\s+\w+|^const\s+\w+\s*[:=]/m.test( + trimmed + ) + ) + return 'ts' if (trimmed.startsWith('{') || trimmed.startsWith('[')) return 'json' return 'text' } -function isTypescriptDeclaration(line: string): boolean { - return ( - startsWithNamedDeclaration(line, 'function') - || startsWithNamedDeclaration(line, 'async function') - || startsWithNamedDeclaration(line, 'export function') - || startsWithNamedDeclaration(line, 'export async function') - || startsWithNamedDeclaration(line, 'interface') - || startsWithAliasDeclaration(line, 'type', '=') - || startsWithAliasDeclaration(line, 'const', '=') - || startsWithAliasDeclaration(line, 'const', ':') - ) -} - -function startsWithNamedDeclaration(line: string, keyword: string): boolean { - if (!line.startsWith(`${keyword} `)) return false - return readIdentifier(line.slice(keyword.length).trimStart()).length > 0 -} - -function startsWithAliasDeclaration(line: string, keyword: string, marker: string): boolean { - if (!line.startsWith(`${keyword} `)) return false - const rest = line.slice(keyword.length).trimStart() - const identifier = readIdentifier(rest) - if (!identifier) return false - return rest.slice(identifier.length).trimStart().startsWith(marker) -} - -function readIdentifier(value: string): string { - let index = 0 - while (index < value.length) { - const code = value.charCodeAt(index) - const isIdentifierChar = - (code >= 65 && code <= 90) - || (code >= 97 && code <= 122) - || (code >= 48 && code <= 57) - || code === 36 - || code === 95 - if (!isIdentifierChar) break - index += 1 - } - return value.slice(0, index) -} - function oneLine(value: string, max: number = limits.finalPlan.taskTitleChars): string { const normalized = value.replace(/\s+/g, ' ').trim() return normalized.length > max ? `${normalized.slice(0, max - 3)}...` : normalized @@ -1630,13 +1433,10 @@ function unwrapShellCommand(command: string): string { const trimmed = command.trim() if (!trimmed) return trimmed - const prefix = '/bin/bash' - if (!trimmed.toLowerCase().startsWith(prefix)) return trimmed - const afterShell = trimmed.slice(prefix.length).trimStart() - if (!afterShell.toLowerCase().startsWith('-lc')) return trimmed + const bashLc = /^\/bin\/bash\s+-lc\s+([\s\S]+)$/i.exec(trimmed) + if (!bashLc?.[1]) return trimmed - let inner = afterShell.slice(3).trim() - if (!inner) return trimmed + let inner = bashLc[1].trim() if ( (inner.startsWith("'") && inner.endsWith("'")) || (inner.startsWith('"') && inner.endsWith('"')) diff --git a/packages/rendering/src/index.ts b/packages/rendering/src/index.ts index 1b03f25e7..6301a36a2 100644 --- a/packages/rendering/src/index.ts +++ b/packages/rendering/src/index.ts @@ -5,7 +5,7 @@ export { isTerminalCodexAppServerEvent, rustSessionEventToServerNotification } from './codex-app-server' -export { ChatSDKRenderer } from './chat-sdk' +export { ChatSDKRenderer, EMPTY_FINAL_ANSWER_TEXT } from './chat-sdk' export type { CodexAppServerToChatStreamOptions } from './codex-app-server' export type { RendererInterface, RendererSession } from './interface' export { rendererEventTypes } from './schema' diff --git a/patches/@chat-adapter__slack@4.31.0.patch b/patches/@chat-adapter__slack@4.31.0.patch index c6086d360..609dd35e6 100644 --- a/patches/@chat-adapter__slack@4.31.0.patch +++ b/patches/@chat-adapter__slack@4.31.0.patch @@ -1,41 +1,5 @@ -diff --git a/dist/index.d.ts b/dist/index.d.ts -index 2a2330e6fd397fb18b42242fedef682fd57122e8..ce6f6569198bcb81708cdc7e4f68d8edb117f594 100644 ---- a/dist/index.d.ts -+++ b/dist/index.d.ts -@@ -232,6 +232,9 @@ interface SlackOAuthCallbackOptions { - } - /** Slack-specific thread ID data */ - interface SlackThreadId { -+ teamId?: string; -+ team?: string; -+ team_id?: string; - channel: string; - threadTs: string; - } -@@ -904,6 +907,10 @@ declare class SlackAdapter implements Adapter { - */ - getChannelVisibility(threadId: string): ChannelVisibility; - decodeThreadId(threadId: string): SlackThreadId; -+ protected decodeChannelId(channelId: string): { -+ teamId?: string; -+ channel: string; -+ }; - parseMessage(raw: SlackEvent): Message; - /** - * Synchronous message parsing without user lookup. -@@ -919,8 +926,8 @@ declare class SlackAdapter implements Adapter { - * Fetch channel-level messages (conversations.history, not thread replies). - */ - fetchChannelMessages(channelId: string, options?: FetchOptions): Promise>; -- protected fetchChannelMessagesForward(channel: string, limit: number, cursor?: string): Promise>; -- protected fetchChannelMessagesBackward(channel: string, limit: number, cursor?: string): Promise>; -+ protected fetchChannelMessagesForward(channel: string, teamId: string | undefined, limit: number, cursor?: string): Promise>; -+ protected fetchChannelMessagesBackward(channel: string, teamId: string | undefined, limit: number, cursor?: string): Promise>; - /** - * List threads in a Slack channel. - * Fetches channel history and filters for messages with replies. diff --git a/dist/index.js b/dist/index.js -index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a3fb0efba 100644 +index a7048fd884020cfd96a51c48b7071fa7293f3dc4..e5bd30bb8f2d41f8f836e7dee742a67f32ccc951 100644 --- a/dist/index.js +++ b/dist/index.js @@ -31,6 +31,216 @@ import { @@ -255,19 +219,41 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a // src/cards.ts import { -@@ -1472,6 +1682,7 @@ var SlackAdapter = class _SlackAdapter { - return; - } - const threadId = channel && (threadTs || messageTs) ? this.encodeThreadId({ -+ teamId: payload.team?.id || payload.container?.team_id, - channel, - threadTs: threadTs || messageTs || "" - }) : ""; -@@ -2001,10 +2212,14 @@ var SlackAdapter = class _SlackAdapter { - const isDM = event.channel_type === "im"; - const threadTs = isDM ? event.thread_ts || "" : event.thread_ts || event.ts; - const threadId = this.encodeThreadId({ -+ teamId: event.team_id || event.team, +@@ -352,6 +562,15 @@ import { + } from "chat"; + var BARE_MENTION_PATTERN = /(?]+/g; ++function plainTextPreservingBlocks(node) { ++ if (node.type === "root" || node.type === "blockquote") { ++ return getNodeChildren(node).map(plainTextPreservingBlocks).filter(Boolean).join(node.type === "root" ? "\n\n" : "\n"); ++ } ++ if (node.type === "list" || node.type === "listItem") { ++ return getNodeChildren(node).map(plainTextPreservingBlocks).filter(Boolean).join("\n"); ++ } ++ return toPlainText(node); ++} + var SlackFormatConverter = class extends BaseFormatConverter { + /** + * Render an AST to standard markdown. Slack accepts this directly via +@@ -366,6 +585,17 @@ var SlackFormatConverter = class extends BaseFormatConverter { + toAst(mrkdwn) { + return parseMarkdown(slackMrkdwnToMarkdown(mrkdwn)); + } ++ /** ++ * Extract plain text for incoming `message` events. The base implementation ++ * flattens the whole AST with mdast-util-to-string, which concatenates ++ * sibling block nodes with NO separator: `--model=fable\n\nexamine ...` ++ * became `--model=fableexamine ...`, gluing every paragraph boundary in the ++ * message. Preserve block boundaries instead — paragraphs join with a blank ++ * line, list items and blockquote lines with a newline. ++ */ ++ extractPlainText(mrkdwn) { ++ return plainTextPreservingBlocks(this.toAst(mrkdwn)); ++ } + /** + * Build the Slack API payload fields for a message. + * +@@ -2004,7 +2234,10 @@ var SlackAdapter = class _SlackAdapter { channel: event.channel, threadTs }); @@ -279,39 +265,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a const factory = async () => { const msg = await this.parseSlackMessage(event, threadId); if (isMention) { -@@ -2090,6 +2305,7 @@ var SlackAdapter = class _SlackAdapter { - ); - } - const threadId = this.encodeThreadId({ -+ teamId: event.team_id || event.team, - channel: event.item.channel, - threadTs: parentTs - }); -@@ -2137,6 +2353,7 @@ var SlackAdapter = class _SlackAdapter { - } - const { channel_id, thread_ts, user_id, context } = event.assistant_thread; - const threadId = this.encodeThreadId({ -+ teamId: context?.team_id, - channel: channel_id, - threadTs: thread_ts - }); -@@ -2177,6 +2394,7 @@ var SlackAdapter = class _SlackAdapter { - } - const { channel_id, thread_ts, user_id, context } = event.assistant_thread; - const threadId = this.encodeThreadId({ -+ teamId: context?.team_id, - channel: channel_id, - threadTs: thread_ts - }); -@@ -2233,6 +2451,7 @@ var SlackAdapter = class _SlackAdapter { - { - userId: event.user, - channelId: this.encodeThreadId({ -+ teamId: event.team, - channel: event.channel, - threadTs: "" - }), -@@ -2520,10 +2739,10 @@ var SlackAdapter = class _SlackAdapter { +@@ -2520,10 +2753,10 @@ var SlackAdapter = class _SlackAdapter { formatted: this.formatConverter.toAst(text), raw: event, author: { @@ -324,7 +278,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a isMe }, metadata: { -@@ -3452,26 +3671,251 @@ var SlackAdapter = class _SlackAdapter { +@@ -3452,26 +3685,251 @@ var SlackAdapter = class _SlackAdapter { } this.logger.debug("Slack: starting stream", { channel, threadTs }); const token = await this.getToken(); @@ -585,7 +539,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a const sendStructuredChunk = async (chunk) => { if (!structuredChunksSupported) { return; -@@ -3481,8 +3925,45 @@ var SlackAdapter = class _SlackAdapter { +@@ -3481,8 +3939,45 @@ var SlackAdapter = class _SlackAdapter { await flushMarkdownDelta(delta); lastAppended = committable; try { @@ -632,7 +586,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a structuredChunksSupported = false; this.logger.warn( "Structured streaming chunk failed, falling back to text-only streaming. Ensure your Slack app manifest includes assistant_view, assistant:write scope, and @slack/web-api >= 7.14.0", -@@ -3497,31 +3978,91 @@ var SlackAdapter = class _SlackAdapter { +@@ -3497,31 +3992,91 @@ var SlackAdapter = class _SlackAdapter { await flushMarkdownDelta(delta); lastAppended = committable; }; @@ -652,7 +606,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a + } else { + await sendStructuredChunk(chunk); + } -+ } + } + renderer.finish(); + const finalCommittable = renderer.getCommittableText(); + const finalDelta = finalCommittable.slice(lastAppended.length); @@ -715,7 +669,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a + error: stopError + }); + } - } ++ } + annotateSlackAnswerLost(error, !sourceConsumed || answerStopFailed); + throw error; } @@ -743,67 +697,7 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a }; } /** -@@ -3729,7 +4270,7 @@ var SlackAdapter = class _SlackAdapter { - } - } - encodeThreadId(platformData) { -- return `slack:${platformData.channel}:${platformData.threadTs}`; -+ return `slack:${platformData.channel}:${platformData.threadTs}`; - } - /** - * Check if a thread is a direct message conversation. -@@ -3762,21 +4304,49 @@ var SlackAdapter = class _SlackAdapter { - } - decodeThreadId(threadId) { - const parts = threadId.split(":"); -- if (parts.length < 2 || parts.length > 3 || parts[0] !== "slack") { -+ if (parts.length < 2 || parts.length > 4 || parts[0] !== "slack") { - throw new ValidationError( - "slack", - `Invalid Slack thread ID: ${threadId}` - ); - } -+ if (parts.length === 4) { -+ return { -+ teamId: parts[1], -+ team: parts[1], -+ team_id: parts[1], -+ channel: parts[2], -+ threadTs: parts[3] -+ }; -+ } - return { - channel: parts[1], - threadTs: parts.length === 3 ? parts[2] : "" - }; - } -+ decodeChannelId(channelId) { -+ const parts = channelId.split(":"); -+ if (parts.length === 3 && parts[0] === "slack") { -+ return { -+ teamId: parts[1], -+ channel: parts[2] -+ }; -+ } -+ if (parts.length === 2 && parts[0] === "slack") { -+ return { -+ channel: parts[1] -+ }; -+ } -+ throw new ValidationError( -+ "slack", -+ `Invalid Slack channel ID: ${channelId}` -+ ); -+ } - parseMessage(raw) { - const event = raw; - const threadTs = event.thread_ts || event.ts || ""; - const threadId = this.encodeThreadId({ -+ teamId: event.team_id || event.team, - channel: event.channel || "", - threadTs - }); -@@ -3798,10 +4368,10 @@ var SlackAdapter = class _SlackAdapter { +@@ -3798,10 +4353,10 @@ var SlackAdapter = class _SlackAdapter { formatted: this.formatConverter.toAst(text), raw: event, author: { @@ -816,145 +710,3 @@ index a7048fd884020cfd96a51c48b7071fa7293f3dc4..82c1259a0726d66da5bb2665d8708e8a isMe }, metadata: { -@@ -3823,32 +4393,28 @@ var SlackAdapter = class _SlackAdapter { - * Slack thread IDs are "slack:CHANNEL:THREAD_TS", channel ID is "slack:CHANNEL". - */ - channelIdFromThreadId(threadId) { -- const { channel } = this.decodeThreadId(threadId); -- return `slack:${channel}`; -+ const { teamId, channel } = this.decodeThreadId(threadId); -+ return teamId ? `slack:${teamId}:${channel}` : `slack:${channel}`; - } - /** - * Fetch channel-level messages (conversations.history, not thread replies). - */ - async fetchChannelMessages(channelId, options = {}) { -- const channel = channelId.split(":")[1]; -- if (!channel) { -- throw new ValidationError( -- "slack", -- `Invalid Slack channel ID: ${channelId}` -- ); -- } -+ const { teamId, channel } = this.decodeChannelId(channelId); - const direction = options.direction ?? "backward"; - const limit = options.limit || 100; - try { - if (direction === "forward") { - return await this.fetchChannelMessagesForward( - channel, -+ teamId, - limit, - options.cursor - ); - } - return await this.fetchChannelMessagesBackward( - channel, -+ teamId, - limit, - options.cursor - ); -@@ -3856,7 +4422,7 @@ var SlackAdapter = class _SlackAdapter { - this.handleSlackError(error); - } - } -- async fetchChannelMessagesForward(channel, limit, cursor) { -+ async fetchChannelMessagesForward(channel, teamId, limit, cursor) { - this.logger.debug("Slack API: conversations.history (forward)", { - channel, - limit, -@@ -3874,7 +4440,11 @@ var SlackAdapter = class _SlackAdapter { - const messages = await Promise.all( - slackMessages.map((msg) => { - const threadTs = msg.thread_ts || msg.ts || ""; -- const threadId = `slack:${channel}:${threadTs}`; -+ const threadId = this.encodeThreadId({ -+ teamId: teamId || msg.team_id || msg.team, -+ channel, -+ threadTs -+ }); - return this.parseSlackMessage(msg, threadId, { - skipSelfMention: false - }); -@@ -3892,7 +4462,7 @@ var SlackAdapter = class _SlackAdapter { - nextCursor - }; - } -- async fetchChannelMessagesBackward(channel, limit, cursor) { -+ async fetchChannelMessagesBackward(channel, teamId, limit, cursor) { - this.logger.debug("Slack API: conversations.history (backward)", { - channel, - limit, -@@ -3911,7 +4481,11 @@ var SlackAdapter = class _SlackAdapter { - const messages = await Promise.all( - chronological.map((msg) => { - const threadTs = msg.thread_ts || msg.ts || ""; -- const threadId = `slack:${channel}:${threadTs}`; -+ const threadId = this.encodeThreadId({ -+ teamId: teamId || msg.team_id || msg.team, -+ channel, -+ threadTs -+ }); - return this.parseSlackMessage(msg, threadId, { - skipSelfMention: false - }); -@@ -3934,13 +4508,7 @@ var SlackAdapter = class _SlackAdapter { - * Fetches channel history and filters for messages with replies. - */ - async listThreads(channelId, options = {}) { -- const channel = channelId.split(":")[1]; -- if (!channel) { -- throw new ValidationError( -- "slack", -- `Invalid Slack channel ID: ${channelId}` -- ); -- } -+ const { teamId, channel } = this.decodeChannelId(channelId); - const limit = options.limit || 50; - try { - this.logger.debug("Slack API: conversations.history (listThreads)", { -@@ -3964,7 +4532,11 @@ var SlackAdapter = class _SlackAdapter { - const threads = await Promise.all( - selected.map(async (msg) => { - const threadTs = msg.ts || ""; -- const threadId = `slack:${channel}:${threadTs}`; -+ const threadId = this.encodeThreadId({ -+ teamId: teamId || msg.team_id || msg.team, -+ channel, -+ threadTs -+ }); - const rootMessage = await this.parseSlackMessage(msg, threadId, { - skipSelfMention: false - }); -@@ -3989,13 +4561,7 @@ var SlackAdapter = class _SlackAdapter { - * Fetch Slack channel info/metadata. - */ - async fetchChannelInfo(channelId) { -- const channel = channelId.split(":")[1]; -- if (!channel) { -- throw new ValidationError( -- "slack", -- `Invalid Slack channel ID: ${channelId}` -- ); -- } -+ const { channel } = this.decodeChannelId(channelId); - try { - this.logger.debug("Slack API: conversations.info (channel)", { channel }); - const result = await this._client.conversations.info( -@@ -4032,14 +4598,8 @@ var SlackAdapter = class _SlackAdapter { - * Post a top-level message to a channel (not in a thread). - */ - async postChannelMessage(channelId, message) { -- const channel = channelId.split(":")[1]; -- if (!channel) { -- throw new ValidationError( -- "slack", -- `Invalid Slack channel ID: ${channelId}` -- ); -- } -- const syntheticThreadId = `slack:${channel}:`; -+ const { teamId, channel } = this.decodeChannelId(channelId); -+ const syntheticThreadId = this.encodeThreadId({ teamId, channel, threadTs: "" }); - return await this.postMessage(syntheticThreadId, message); - } - renderFormatted(content) { diff --git a/patches/chat@4.31.0.patch b/patches/chat@4.31.0.patch index 8c46cefdb..1001c40ea 100644 --- a/patches/chat@4.31.0.patch +++ b/patches/chat@4.31.0.patch @@ -2,7 +2,7 @@ diff --git a/dist/index.js b/dist/index.js index 6af9d7fec34adeb8a7092a3ba0077d9f327a44d7..252b031dd39e65cb6c85d3b0e5c23eb1a3cc41c2 100644 --- a/dist/index.js +++ b/dist/index.js -@@ -3443,7 +3443,20 @@ var Chat = class { +@@ -3443,7 +3443,27 @@ var Chat = class { }); return; } @@ -10,8 +10,15 @@ index 6af9d7fec34adeb8a7092a3ba0077d9f327a44d7..252b031dd39e65cb6c85d3b0e5c23eb1 + message.isMention = message.isMention || this.detectMention(adapter, message); + const isSubscribedForDedupe = await this._stateAdapter.isSubscribed(threadId); + const isDMForDedupe = adapter.isDM?.(threadId) ?? false; ++ const matchesPatternForDedupe = this.messagePatterns.some(({ pattern }) => { ++ const lastIndex = pattern.lastIndex; ++ const matches = pattern.test(message.text); ++ pattern.lastIndex = lastIndex; ++ return matches; ++ }); ++ const hasAction = message.isMention || isDMForDedupe || isSubscribedForDedupe || matchesPatternForDedupe; + const hasHistoryPersistence = adapter.persistThreadHistory || adapter.persistMessageHistory; -+ const dedupeBucket = message.isMention ? "mention" : isDMForDedupe ? "dm" : isSubscribedForDedupe ? "subscribed" : this.messagePatterns.length > 0 ? "pattern" : hasHistoryPersistence ? "history" : null; ++ const dedupeBucket = hasAction ? "action" : hasHistoryPersistence ? "history" : null; + if (!dedupeBucket) { + this.logger.debug("Skipping non-actionable message before dedupe", { + adapter: adapter.name, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index abbf73768..93ccf5341 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -12,13 +12,13 @@ patchedDependencies: hash: fce7a692b030cfe3d325b020a4472b9424b8976aa2f7faded6ad4c83421e9132 path: patches/@chat-adapter__linear@4.31.0.patch '@chat-adapter/slack@4.31.0': - hash: fab60ea727e32c0113ae183fe326244df85c5ac8903ff22fb3956cf9c38e5fbd + hash: b005d7fc3498bc499bfd30ff79be29138a48b63944761da6f1b68bec59ef9d68 path: patches/@chat-adapter__slack@4.31.0.patch '@chat-adapter/state-pg@4.31.0': hash: 69262b03c278ca9d4af0bed7b4e05f9d7a5a36d3d79fad31df2a85da4d349274 path: patches/@chat-adapter__state-pg@4.31.0.patch chat@4.31.0: - hash: af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c + hash: 378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd path: patches/chat@4.31.0.patch importers: @@ -27,15 +27,9 @@ importers: packages/api-client: dependencies: - '@centaur/harness-events': - specifier: workspace:* - version: link:../harness-events axios: specifier: ^1.13.6 version: 1.18.0 - eventsource-parser: - specifier: ^3.0.6 - version: 3.1.0 devDependencies: typescript: specifier: 5.9.3 @@ -94,7 +88,7 @@ importers: version: 4.31.0(patch_hash=69262b03c278ca9d4af0bed7b4e05f9d7a5a36d3d79fad31df2a85da4d349274)(zod@4.4.3) chat: specifier: ^4.31.0 - version: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + version: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) discord.js: specifier: ^14.25.1 version: 14.26.4 @@ -124,6 +118,55 @@ importers: specifier: ^6.0.3 version: 6.0.3 + services/githubbot: + dependencies: + '@centaur/harness-events': + specifier: workspace:* + version: link:../../packages/harness-events + '@centaur/rendering': + specifier: workspace:* + version: link:../../packages/rendering + '@chat-adapter/github': + specifier: ^4.31.0 + version: 4.31.0(zod@4.4.3) + '@chat-adapter/state-pg': + specifier: ^4.31.0 + version: 4.31.0(patch_hash=69262b03c278ca9d4af0bed7b4e05f9d7a5a36d3d79fad31df2a85da4d349274)(zod@4.4.3) + '@octokit/auth-app': + specifier: ^7.1.5 + version: 7.2.2 + '@octokit/rest': + specifier: ^21.1.1 + version: 21.1.1 + chat: + specifier: ^4.31.0 + version: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) + hono: + specifier: ^4.12.18 + version: 4.12.25 + pg: + specifier: ^8.21.0 + version: 8.21.0 + devDependencies: + '@chat-adapter/state-memory': + specifier: ^4.31.0 + version: 4.31.0(zod@4.4.3) + '@types/bun': + specifier: ^1.3.13 + version: 1.3.14 + '@types/node': + specifier: ^25.7.0 + version: 25.9.3 + '@types/pg': + specifier: ^8.15.5 + version: 8.20.0 + '@typescript/native-preview': + specifier: ^7.0.0-dev.20260512.1 + version: 7.0.0-dev.20260616.1 + typescript: + specifier: ^6.0.3 + version: 6.0.3 + services/linearbot: dependencies: '@centaur/harness-events': @@ -143,7 +186,7 @@ importers: version: 76.0.0(graphql@17.0.0) chat: specifier: ^4.31.0 - version: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + version: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) hono: specifier: ^4.12.18 version: 4.12.25 @@ -180,13 +223,13 @@ importers: version: link:../../packages/rendering '@chat-adapter/slack': specifier: ^4.31.0 - version: 4.31.0(patch_hash=fab60ea727e32c0113ae183fe326244df85c5ac8903ff22fb3956cf9c38e5fbd)(zod@4.4.3) + version: 4.31.0(patch_hash=b005d7fc3498bc499bfd30ff79be29138a48b63944761da6f1b68bec59ef9d68)(zod@4.4.3) '@chat-adapter/state-pg': specifier: ^4.31.0 version: 4.31.0(patch_hash=69262b03c278ca9d4af0bed7b4e05f9d7a5a36d3d79fad31df2a85da4d349274)(zod@4.4.3) chat: specifier: ^4.31.0 - version: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + version: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) hono: specifier: ^4.12.18 version: 4.12.25 @@ -235,7 +278,7 @@ importers: version: 4.31.0(zod@4.4.3) chat: specifier: ^4.31.0 - version: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + version: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) dotenv: specifier: ^17.3.1 version: 17.4.2 @@ -282,6 +325,10 @@ packages: resolution: {integrity: sha512-1dVw1+6ZBwdVh5ynLK9D+SyHdPWIBbiPmKRC+WhzDSddR8eySswcBiDUKaDMUaOmvJavPvibJLwWbohUSBiJww==} engines: {node: '>=20'} + '@chat-adapter/github@4.31.0': + resolution: {integrity: sha512-yMKNR5WWMoCP1jGCxOigI6NrgC7Dm60zuy6ey1BgNPjgA6n769ZI8dvf+N6wgwqFHOeqSNIYhNnT/ojB6WsjtQ==} + engines: {node: '>=20'} + '@chat-adapter/linear@4.31.0': resolution: {integrity: sha512-myEDw3LoSDaVCjLQ4nDNWK3XFTVa+asO3/78qRxEv76kl1izcYi0g0BhZvozdJgswkKeZC2AaLzvN8GEerKZPw==} engines: {node: '>=20'} @@ -391,6 +438,164 @@ packages: '@emnapi/core': ^1.7.1 '@emnapi/runtime': ^1.7.1 + '@octokit/auth-app@7.2.2': + resolution: {integrity: sha512-p6hJtEyQDCJEPN9ijjhEC/kpFHMHN4Gca9r+8S0S8EJi7NaWftaEmexjxxpT1DFBeJpN4u/5RE22ArnyypupJw==} + engines: {node: '>= 18'} + + '@octokit/auth-app@8.2.0': + resolution: {integrity: sha512-vVjdtQQwomrZ4V46B9LaCsxsySxGoHsyw6IYBov/TqJVROrlYdyNgw5q6tQbB7KZt53v1l1W53RiqTvpzL907g==} + engines: {node: '>= 20'} + + '@octokit/auth-oauth-app@8.1.4': + resolution: {integrity: sha512-71iBa5SflSXcclk/OL3lJzdt4iFs56OJdpBGEBl1wULp7C58uiswZLV6TdRaiAzHP1LT8ezpbHlKuxADb+4NkQ==} + engines: {node: '>= 18'} + + '@octokit/auth-oauth-app@9.0.3': + resolution: {integrity: sha512-+yoFQquaF8OxJSxTb7rnytBIC2ZLbLqA/yb71I4ZXT9+Slw4TziV9j/kyGhUFRRTF2+7WlnIWsePZCWHs+OGjg==} + engines: {node: '>= 20'} + + '@octokit/auth-oauth-device@7.1.5': + resolution: {integrity: sha512-lR00+k7+N6xeECj0JuXeULQ2TSBB/zjTAmNF2+vyGPDEFx1dgk1hTDmL13MjbSmzusuAmuJD8Pu39rjp9jH6yw==} + engines: {node: '>= 18'} + + '@octokit/auth-oauth-device@8.0.3': + resolution: {integrity: sha512-zh2W0mKKMh/VWZhSqlaCzY7qFyrgd9oTWmTmHaXnHNeQRCZr/CXy2jCgHo4e4dJVTiuxP5dLa0YM5p5QVhJHbw==} + engines: {node: '>= 20'} + + '@octokit/auth-oauth-user@5.1.6': + resolution: {integrity: sha512-/R8vgeoulp7rJs+wfJ2LtXEVC7pjQTIqDab7wPKwVG6+2v/lUnCOub6vaHmysQBbb45FknM3tbHW8TOVqYHxCw==} + engines: {node: '>= 18'} + + '@octokit/auth-oauth-user@6.0.2': + resolution: {integrity: sha512-qLoPPc6E6GJoz3XeDG/pnDhJpTkODTGG4kY0/Py154i/I003O9NazkrwJwRuzgCalhzyIeWQ+6MDvkUmKXjg/A==} + engines: {node: '>= 20'} + + '@octokit/auth-token@5.1.2': + resolution: {integrity: sha512-JcQDsBdg49Yky2w2ld20IHAlwr8d/d8N6NiOXbtuoPCqzbsiJgF633mVUw3x4mo0H5ypataQIX7SFu3yy44Mpw==} + engines: {node: '>= 18'} + + '@octokit/auth-token@6.0.0': + resolution: {integrity: sha512-P4YJBPdPSpWTQ1NU4XYdvHvXJJDxM6YwpS0FZHRgP7YFkdVxsWcpWGy/NVqlAA7PcPCnMacXlRm1y2PFZRWL/w==} + engines: {node: '>= 20'} + + '@octokit/core@6.1.6': + resolution: {integrity: sha512-kIU8SLQkYWGp3pVKiYzA5OSaNF5EE03P/R8zEmmrG6XwOg5oBjXyQVVIauQ0dgau4zYhpZEhJrvIYt6oM+zZZA==} + engines: {node: '>= 18'} + + '@octokit/core@7.0.6': + resolution: {integrity: sha512-DhGl4xMVFGVIyMwswXeyzdL4uXD5OGILGX5N8Y+f6W7LhC1Ze2poSNrkF/fedpVDHEEZ+PHFW0vL14I+mm8K3Q==} + engines: {node: '>= 20'} + + '@octokit/endpoint@10.1.4': + resolution: {integrity: sha512-OlYOlZIsfEVZm5HCSR8aSg02T2lbUWOsCQoPKfTXJwDzcHQBrVBGdGXb89dv2Kw2ToZaRtudp8O3ZIYoaOjKlA==} + engines: {node: '>= 18'} + + '@octokit/endpoint@11.0.3': + resolution: {integrity: sha512-FWFlNxghg4HrXkD3ifYbS/IdL/mDHjh9QcsNyhQjN8dplUoZbejsdpmuqdA76nxj2xoWPs7p8uX2SNr9rYu0Ag==} + engines: {node: '>= 20'} + + '@octokit/graphql@8.2.2': + resolution: {integrity: sha512-Yi8hcoqsrXGdt0yObxbebHXFOiUA+2v3n53epuOg1QUgOB6c4XzvisBNVXJSl8RYA5KrDuSL2yq9Qmqe5N0ryA==} + engines: {node: '>= 18'} + + '@octokit/graphql@9.0.3': + resolution: {integrity: sha512-grAEuupr/C1rALFnXTv6ZQhFuL1D8G5y8CN04RgrO4FIPMrtm+mcZzFG7dcBm+nq+1ppNixu+Jd78aeJOYxlGA==} + engines: {node: '>= 20'} + + '@octokit/oauth-authorization-url@7.1.1': + resolution: {integrity: sha512-ooXV8GBSabSWyhLUowlMIVd9l1s2nsOGQdlP2SQ4LnkEsGXzeCvbSbCPdZThXhEFzleGPwbapT0Sb+YhXRyjCA==} + engines: {node: '>= 18'} + + '@octokit/oauth-authorization-url@8.0.0': + resolution: {integrity: sha512-7QoLPRh/ssEA/HuHBHdVdSgF8xNLz/Bc5m9fZkArJE5bb6NmVkDm3anKxXPmN1zh6b5WKZPRr3697xKT/yM3qQ==} + engines: {node: '>= 20'} + + '@octokit/oauth-methods@5.1.5': + resolution: {integrity: sha512-Ev7K8bkYrYLhoOSZGVAGsLEscZQyq7XQONCBBAl2JdMg7IT3PQn/y8P0KjloPoYpI5UylqYrLeUcScaYWXwDvw==} + engines: {node: '>= 18'} + + '@octokit/oauth-methods@6.0.2': + resolution: {integrity: sha512-HiNOO3MqLxlt5Da5bZbLV8Zarnphi4y9XehrbaFMkcoJ+FL7sMxH/UlUsCVxpddVu4qvNDrBdaTVE2o4ITK8ng==} + engines: {node: '>= 20'} + + '@octokit/openapi-types@24.2.0': + resolution: {integrity: sha512-9sIH3nSUttelJSXUrmGzl7QUBFul0/mB8HRYl3fOlgHbIWG+WnYDXU3v/2zMtAvuzZ/ed00Ei6on975FhBfzrg==} + + '@octokit/openapi-types@25.1.0': + resolution: {integrity: sha512-idsIggNXUKkk0+BExUn1dQ92sfysJrje03Q0bv0e+KPLrvyqZF8MnBpFz8UNfYDwB3Ie7Z0TByjWfzxt7vseaA==} + + '@octokit/openapi-types@27.0.0': + resolution: {integrity: sha512-whrdktVs1h6gtR+09+QsNk2+FO+49j6ga1c55YZudfEG+oKJVvJLQi3zkOm5JjiUXAagWK2tI2kTGKJ2Ys7MGA==} + + '@octokit/plugin-paginate-rest@11.6.0': + resolution: {integrity: sha512-n5KPteiF7pWKgBIBJSk8qzoZWcUkza2O6A0za97pMGVrGfPdltxrfmfF5GucHYvHGZD8BdaZmmHGz5cX/3gdpw==} + engines: {node: '>= 18'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/plugin-paginate-rest@14.0.0': + resolution: {integrity: sha512-fNVRE7ufJiAA3XUrha2omTA39M6IXIc6GIZLvlbsm8QOQCYvpq/LkMNGyFlB1d8hTDzsAXa3OKtybdMAYsV/fw==} + engines: {node: '>= 20'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/plugin-request-log@5.3.1': + resolution: {integrity: sha512-n/lNeCtq+9ofhC15xzmJCNKP2BWTv8Ih2TTy+jatNCCq/gQP/V7rK3fjIfuz0pDWDALO/o/4QY4hyOF6TQQFUw==} + engines: {node: '>= 18'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/plugin-request-log@6.0.0': + resolution: {integrity: sha512-UkOzeEN3W91/eBq9sPZNQ7sUBvYCqYbrrD8gTbBuGtHEuycE4/awMXcYvx6sVYo7LypPhmQwwpUe4Yyu4QZN5Q==} + engines: {node: '>= 20'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/plugin-rest-endpoint-methods@13.5.0': + resolution: {integrity: sha512-9Pas60Iv9ejO3WlAX3maE1+38c5nqbJXV5GrncEfkndIpZrJ/WPMRd2xYDcPPEt5yzpxcjw9fWNoPhsSGzqKqw==} + engines: {node: '>= 18'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/plugin-rest-endpoint-methods@17.0.0': + resolution: {integrity: sha512-B5yCyIlOJFPqUUeiD0cnBJwWJO8lkJs5d8+ze9QDP6SvfiXSz1BF+91+0MeI1d2yxgOhU/O+CvtiZ9jSkHhFAw==} + engines: {node: '>= 20'} + peerDependencies: + '@octokit/core': '>=6' + + '@octokit/request-error@6.1.8': + resolution: {integrity: sha512-WEi/R0Jmq+IJKydWlKDmryPcmdYSVjL3ekaiEL1L9eo1sUnqMJ+grqmC9cjk7CA7+b2/T397tO5d8YLOH3qYpQ==} + engines: {node: '>= 18'} + + '@octokit/request-error@7.1.0': + resolution: {integrity: sha512-KMQIfq5sOPpkQYajXHwnhjCC0slzCNScLHs9JafXc4RAJI+9f+jNDlBNaIMTvazOPLgb4BnlhGJOTbnN0wIjPw==} + engines: {node: '>= 20'} + + '@octokit/request@10.0.10': + resolution: {integrity: sha512-KxNC2pTqqhszMNrf12ZRd4PonRgyJdsM4F/jySiddQK+DsRcfBtUvqn8t7UsyZhnRJHvX46OohDt5N3VqIWC2w==} + engines: {node: '>= 20'} + + '@octokit/request@9.2.4': + resolution: {integrity: sha512-q8ybdytBmxa6KogWlNa818r0k1wlqzNC+yNkcQDECHvQo8Vmstrg18JwqJHdJdUiHD2sjlwBgSm9kHkOKe2iyA==} + engines: {node: '>= 18'} + + '@octokit/rest@21.1.1': + resolution: {integrity: sha512-sTQV7va0IUVZcntzy1q3QqPm/r8rWtDCqpRAmb8eXXnKkjoQEtFe3Nt5GTVsHft+R6jJoHeSiVLcgcvhtue/rg==} + engines: {node: '>= 18'} + + '@octokit/rest@22.0.1': + resolution: {integrity: sha512-Jzbhzl3CEexhnivb1iQ0KJ7s5vvjMWcmRtq5aUsKmKDrRW6z3r84ngmiFKFvpZjpiU/9/S6ITPFRpn5s/3uQJw==} + engines: {node: '>= 20'} + + '@octokit/types@13.10.0': + resolution: {integrity: sha512-ifLaO34EbbPj0Xgro4G5lP5asESjwHracYJvVaPIyXMuiuXLlhic3S47cBdTb+jfODkTE5YtGCLt3Ay3+J97sA==} + + '@octokit/types@14.1.0': + resolution: {integrity: sha512-1y6DgTy8Jomcpu33N+p5w58l6xyt55Ar2I91RPiIA0xCJBXyUAhXCcmZaDWSANiha7R9a6qJJ2CRomGPZ6f46g==} + + '@octokit/types@16.0.0': + resolution: {integrity: sha512-sKq+9r1Mm4efXW1FCk7hFSeJo4QKreL/tTbR0rz/qx/r1Oa2VV83LTA/H/MuCOX7uCIJmQVRKBcbmWoySjAnSg==} + '@opentelemetry/api@1.9.1': resolution: {integrity: sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==} engines: {node: '>=8.0.0'} @@ -698,6 +903,12 @@ packages: bail@2.0.2: resolution: {integrity: sha512-0xO6mYd7JB2YesxDKplafRpsiOzPt9V02ddPCLbY1xYGPOX24NTyN50qnUxgCPcSoYMhKpAuBTjQoRZCAkUDRw==} + before-after-hook@3.0.2: + resolution: {integrity: sha512-Nik3Sc0ncrMK4UUdXQmAnRtzmNQTAAXmXIopizwZ1W1t8QmfJj+zL4OA2I7XPTPW5z5TDqv4hRo/JzouDJnX3A==} + + before-after-hook@4.0.0: + resolution: {integrity: sha512-q6tR3RPqIB1pMiTRMFcZwuG5T8vwp+vUvEG0vuI6B+Rikh5BfPp2fQ82c925FOs+b0lcFQ8CFrL+KbilfZFhOQ==} + body-parser@2.3.0: resolution: {integrity: sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==} engines: {node: '>=18'} @@ -883,10 +1094,6 @@ packages: eventemitter3@5.0.4: resolution: {integrity: sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==} - eventsource-parser@3.1.0: - resolution: {integrity: sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==} - engines: {node: '>=18.0.0'} - expect-type@1.3.0: resolution: {integrity: sha512-knvyeauYhqjOYvQ66MznSMs83wmHrCycNEN6Ao+2AeYEfxUIkuiVxdEa1qlGEPK+We3n0THiDciYSsCcgW/DoA==} engines: {node: '>=12.0.0'} @@ -898,6 +1105,9 @@ packages: extend@3.0.2: resolution: {integrity: sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==} + fast-content-type-parse@2.0.1: + resolution: {integrity: sha512-nGqtvLrj5w0naR6tDPfB4cUmYCqouzyQiz6C5y/LtcDllJdrcc6WaWW6iXyIIOErTa/XRybj28aasdn4LkVk6Q==} + fast-deep-equal@3.1.3: resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==} @@ -1015,6 +1225,9 @@ packages: jose@4.15.9: resolution: {integrity: sha512-1vUQX+IdDMVPj4k8kOxgUqlcK518yluMuGZwqlr44FS1ppZB/5GWh4rZG89erpOBOJjU/OBsnCVFfapsRz6nEA==} + json-with-bigint@3.5.8: + resolution: {integrity: sha512-eq/4KP6K34kwa7TcFdtvnftvHCD9KvHOGGICWwMFc4dOOKF5t4iYqnfLK8otCRCRv06FXOzGGyqE8h8ElMvvdw==} + jsonwebtoken@9.0.3: resolution: {integrity: sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g==} engines: {node: '>=12', npm: '>=6'} @@ -1540,6 +1753,10 @@ packages: resolution: {integrity: sha512-Bf+ILmBgretUrdJxzXM0SgXLZ3XfiaUuOj/IKQHuTXip+05Xn+uyEYdVg0kYDipTBcLrCVyUzAPz7QmArb0mmw==} engines: {node: '>=14.0.0'} + toad-cache@3.7.1: + resolution: {integrity: sha512-5DXWzE4Vz7xNHsv+xQ+MGfJYyC78Aok3tEr0MNwHoRf7vZnga1mQXZ4/Nsodld4VR6Wd+VhfmqnNrsRJyYPfrQ==} + engines: {node: '>=20'} + toidentifier@1.0.1: resolution: {integrity: sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==} engines: {node: '>=0.6'} @@ -1589,6 +1806,12 @@ packages: unist-util-visit@5.1.0: resolution: {integrity: sha512-m+vIdyeCOpdr/QeQCu2EzxX/ohgS8KbnPDgFni4dQsfSCtpz8UqDyY5GjRru8PDKuYn7Fq19j1CQ+nJSsGKOzg==} + universal-github-app-jwt@2.2.2: + resolution: {integrity: sha512-dcmbeSrOdTnsjGjUfAlqNDJrhxXizjAz94ija9Qw8YkZ1uu0d+GoZzyH+Jb9tIIqvGsadUfwg+22k5aDqqwzbw==} + + universal-user-agent@7.0.3: + resolution: {integrity: sha512-TmnEAEAsBJVZM/AADELsK76llnwcf9vMKuPz8JflO1frO8Lchitr0fNaN9d+Ap0BjKtqWqd/J17qeDnXh8CL2A==} + unpipe@1.0.0: resolution: {integrity: sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==} engines: {node: '>= 0.8'} @@ -1737,7 +1960,7 @@ snapshots: '@chat-adapter/discord@4.31.0(patch_hash=8f4fbb770159f924570fbad681297fcb9f8f7a6353a705cfcb78e39ef9e3a3ab)(zod@4.4.3)': dependencies: '@chat-adapter/shared': 4.31.0(zod@4.4.3) - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) discord-api-types: 0.37.120 discord-interactions: 4.4.0 discord.js: 14.26.4 @@ -1748,11 +1971,22 @@ snapshots: - utf-8-validate - zod + '@chat-adapter/github@4.31.0(zod@4.4.3)': + dependencies: + '@chat-adapter/shared': 4.31.0(zod@4.4.3) + '@octokit/auth-app': 8.2.0 + '@octokit/rest': 22.0.1 + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) + transitivePeerDependencies: + - ai + - supports-color + - zod + '@chat-adapter/linear@4.31.0(patch_hash=fce7a692b030cfe3d325b020a4472b9424b8976aa2f7faded6ad4c83421e9132)(graphql@17.0.0)(zod@4.4.3)': dependencies: '@chat-adapter/shared': 4.31.0(zod@4.4.3) '@linear/sdk': 76.0.0(graphql@17.0.0) - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) transitivePeerDependencies: - ai - graphql @@ -1761,18 +1995,18 @@ snapshots: '@chat-adapter/shared@4.31.0(zod@4.4.3)': dependencies: - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) transitivePeerDependencies: - ai - supports-color - zod - '@chat-adapter/slack@4.31.0(patch_hash=fab60ea727e32c0113ae183fe326244df85c5ac8903ff22fb3956cf9c38e5fbd)(zod@4.4.3)': + '@chat-adapter/slack@4.31.0(patch_hash=b005d7fc3498bc499bfd30ff79be29138a48b63944761da6f1b68bec59ef9d68)(zod@4.4.3)': dependencies: '@chat-adapter/shared': 4.31.0(zod@4.4.3) '@slack/socket-mode': 2.0.7 '@slack/web-api': 7.17.0 - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) transitivePeerDependencies: - ai - bufferutil @@ -1783,7 +2017,7 @@ snapshots: '@chat-adapter/state-memory@4.31.0(zod@4.4.3)': dependencies: - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) transitivePeerDependencies: - ai - supports-color @@ -1791,7 +2025,7 @@ snapshots: '@chat-adapter/state-pg@4.31.0(patch_hash=69262b03c278ca9d4af0bed7b4e05f9d7a5a36d3d79fad31df2a85da4d349274)(zod@4.4.3)': dependencies: - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) pg: 8.21.0 transitivePeerDependencies: - ai @@ -1806,7 +2040,7 @@ snapshots: '@microsoft/teams.apps': 2.0.13 '@microsoft/teams.cards': 2.0.13 '@microsoft/teams.graph-endpoints': 2.0.13 - chat: 4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3) + chat: 4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3) transitivePeerDependencies: - ai - debug @@ -1946,6 +2180,223 @@ snapshots: '@tybys/wasm-util': 0.10.2 optional: true + '@octokit/auth-app@7.2.2': + dependencies: + '@octokit/auth-oauth-app': 8.1.4 + '@octokit/auth-oauth-user': 5.1.6 + '@octokit/request': 9.2.4 + '@octokit/request-error': 6.1.8 + '@octokit/types': 14.1.0 + toad-cache: 3.7.1 + universal-github-app-jwt: 2.2.2 + universal-user-agent: 7.0.3 + + '@octokit/auth-app@8.2.0': + dependencies: + '@octokit/auth-oauth-app': 9.0.3 + '@octokit/auth-oauth-user': 6.0.2 + '@octokit/request': 10.0.10 + '@octokit/request-error': 7.1.0 + '@octokit/types': 16.0.0 + toad-cache: 3.7.1 + universal-github-app-jwt: 2.2.2 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-app@8.1.4': + dependencies: + '@octokit/auth-oauth-device': 7.1.5 + '@octokit/auth-oauth-user': 5.1.6 + '@octokit/request': 9.2.4 + '@octokit/types': 14.1.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-app@9.0.3': + dependencies: + '@octokit/auth-oauth-device': 8.0.3 + '@octokit/auth-oauth-user': 6.0.2 + '@octokit/request': 10.0.10 + '@octokit/types': 16.0.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-device@7.1.5': + dependencies: + '@octokit/oauth-methods': 5.1.5 + '@octokit/request': 9.2.4 + '@octokit/types': 14.1.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-device@8.0.3': + dependencies: + '@octokit/oauth-methods': 6.0.2 + '@octokit/request': 10.0.10 + '@octokit/types': 16.0.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-user@5.1.6': + dependencies: + '@octokit/auth-oauth-device': 7.1.5 + '@octokit/oauth-methods': 5.1.5 + '@octokit/request': 9.2.4 + '@octokit/types': 14.1.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-oauth-user@6.0.2': + dependencies: + '@octokit/auth-oauth-device': 8.0.3 + '@octokit/oauth-methods': 6.0.2 + '@octokit/request': 10.0.10 + '@octokit/types': 16.0.0 + universal-user-agent: 7.0.3 + + '@octokit/auth-token@5.1.2': {} + + '@octokit/auth-token@6.0.0': {} + + '@octokit/core@6.1.6': + dependencies: + '@octokit/auth-token': 5.1.2 + '@octokit/graphql': 8.2.2 + '@octokit/request': 9.2.4 + '@octokit/request-error': 6.1.8 + '@octokit/types': 14.1.0 + before-after-hook: 3.0.2 + universal-user-agent: 7.0.3 + + '@octokit/core@7.0.6': + dependencies: + '@octokit/auth-token': 6.0.0 + '@octokit/graphql': 9.0.3 + '@octokit/request': 10.0.10 + '@octokit/request-error': 7.1.0 + '@octokit/types': 16.0.0 + before-after-hook: 4.0.0 + universal-user-agent: 7.0.3 + + '@octokit/endpoint@10.1.4': + dependencies: + '@octokit/types': 14.1.0 + universal-user-agent: 7.0.3 + + '@octokit/endpoint@11.0.3': + dependencies: + '@octokit/types': 16.0.0 + universal-user-agent: 7.0.3 + + '@octokit/graphql@8.2.2': + dependencies: + '@octokit/request': 9.2.4 + '@octokit/types': 14.1.0 + universal-user-agent: 7.0.3 + + '@octokit/graphql@9.0.3': + dependencies: + '@octokit/request': 10.0.10 + '@octokit/types': 16.0.0 + universal-user-agent: 7.0.3 + + '@octokit/oauth-authorization-url@7.1.1': {} + + '@octokit/oauth-authorization-url@8.0.0': {} + + '@octokit/oauth-methods@5.1.5': + dependencies: + '@octokit/oauth-authorization-url': 7.1.1 + '@octokit/request': 9.2.4 + '@octokit/request-error': 6.1.8 + '@octokit/types': 14.1.0 + + '@octokit/oauth-methods@6.0.2': + dependencies: + '@octokit/oauth-authorization-url': 8.0.0 + '@octokit/request': 10.0.10 + '@octokit/request-error': 7.1.0 + '@octokit/types': 16.0.0 + + '@octokit/openapi-types@24.2.0': {} + + '@octokit/openapi-types@25.1.0': {} + + '@octokit/openapi-types@27.0.0': {} + + '@octokit/plugin-paginate-rest@11.6.0(@octokit/core@6.1.6)': + dependencies: + '@octokit/core': 6.1.6 + '@octokit/types': 13.10.0 + + '@octokit/plugin-paginate-rest@14.0.0(@octokit/core@7.0.6)': + dependencies: + '@octokit/core': 7.0.6 + '@octokit/types': 16.0.0 + + '@octokit/plugin-request-log@5.3.1(@octokit/core@6.1.6)': + dependencies: + '@octokit/core': 6.1.6 + + '@octokit/plugin-request-log@6.0.0(@octokit/core@7.0.6)': + dependencies: + '@octokit/core': 7.0.6 + + '@octokit/plugin-rest-endpoint-methods@13.5.0(@octokit/core@6.1.6)': + dependencies: + '@octokit/core': 6.1.6 + '@octokit/types': 13.10.0 + + '@octokit/plugin-rest-endpoint-methods@17.0.0(@octokit/core@7.0.6)': + dependencies: + '@octokit/core': 7.0.6 + '@octokit/types': 16.0.0 + + '@octokit/request-error@6.1.8': + dependencies: + '@octokit/types': 14.1.0 + + '@octokit/request-error@7.1.0': + dependencies: + '@octokit/types': 16.0.0 + + '@octokit/request@10.0.10': + dependencies: + '@octokit/endpoint': 11.0.3 + '@octokit/request-error': 7.1.0 + '@octokit/types': 16.0.0 + content-type: 2.0.0 + json-with-bigint: 3.5.8 + universal-user-agent: 7.0.3 + + '@octokit/request@9.2.4': + dependencies: + '@octokit/endpoint': 10.1.4 + '@octokit/request-error': 6.1.8 + '@octokit/types': 14.1.0 + fast-content-type-parse: 2.0.1 + universal-user-agent: 7.0.3 + + '@octokit/rest@21.1.1': + dependencies: + '@octokit/core': 6.1.6 + '@octokit/plugin-paginate-rest': 11.6.0(@octokit/core@6.1.6) + '@octokit/plugin-request-log': 5.3.1(@octokit/core@6.1.6) + '@octokit/plugin-rest-endpoint-methods': 13.5.0(@octokit/core@6.1.6) + + '@octokit/rest@22.0.1': + dependencies: + '@octokit/core': 7.0.6 + '@octokit/plugin-paginate-rest': 14.0.0(@octokit/core@7.0.6) + '@octokit/plugin-request-log': 6.0.0(@octokit/core@7.0.6) + '@octokit/plugin-rest-endpoint-methods': 17.0.0(@octokit/core@7.0.6) + + '@octokit/types@13.10.0': + dependencies: + '@octokit/openapi-types': 24.2.0 + + '@octokit/types@14.1.0': + dependencies: + '@octokit/openapi-types': 25.1.0 + + '@octokit/types@16.0.0': + dependencies: + '@octokit/openapi-types': 27.0.0 + '@opentelemetry/api@1.9.1': optional: true @@ -2244,6 +2695,10 @@ snapshots: bail@2.0.2: {} + before-after-hook@3.0.2: {} + + before-after-hook@4.0.0: {} + body-parser@2.3.0: dependencies: bytes: 3.1.2 @@ -2282,7 +2737,7 @@ snapshots: character-entities@2.0.2: {} - chat@4.31.0(patch_hash=af3488d2a7d0620bb5e8960feb423fd2a5211516e5b8a193df65bfe52a02c87c)(zod@4.4.3): + chat@4.31.0(patch_hash=378e9d2b3a9218ea973cb4dd64af92652affe86f61676540d504151cdce421fd)(zod@4.4.3): dependencies: '@workflow/serde': 4.1.0-beta.2 mdast-util-to-string: 4.0.0 @@ -2420,8 +2875,6 @@ snapshots: eventemitter3@5.0.4: {} - eventsource-parser@3.1.0: {} - expect-type@1.3.0: {} express@5.2.1: @@ -2459,6 +2912,8 @@ snapshots: extend@3.0.2: {} + fast-content-type-parse@2.0.1: {} + fast-deep-equal@3.1.3: {} fdir@6.5.0(picomatch@4.0.4): @@ -2565,6 +3020,8 @@ snapshots: jose@4.15.9: {} + json-with-bigint@3.5.8: {} + jsonwebtoken@9.0.3: dependencies: jws: 4.0.1 @@ -3266,6 +3723,8 @@ snapshots: tinyrainbow@3.1.0: {} + toad-cache@3.7.1: {} + toidentifier@1.0.1: {} trough@2.2.0: {} @@ -3317,6 +3776,10 @@ snapshots: unist-util-is: 6.0.1 unist-util-visit-parents: 6.0.2 + universal-github-app-jwt@2.2.2: {} + + universal-user-agent@7.0.3: {} + unpipe@1.0.0: {} vary@1.1.2: {} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index e2b1be999..ef3fb16b7 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -4,6 +4,7 @@ packages: - services/slackbotv2 - services/discordbot - services/teamsbot + - services/githubbot patchedDependencies: '@chat-adapter/linear@4.31.0': patches/@chat-adapter__linear@4.31.0.patch diff --git a/scripts/mirror-prod-threads-snapshot.sh b/scripts/mirror-prod-threads-snapshot.sh new file mode 100755 index 000000000..e4134e116 --- /dev/null +++ b/scripts/mirror-prod-threads-snapshot.sh @@ -0,0 +1,732 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat <<'USAGE' +Usage: + bash scripts/mirror-prod-threads-snapshot.sh [all|snapshot|import] + +Creates a bounded, local-only snapshot of production thread/session data for +Centaur Console UX work. The source connection is forced into a read-only +transaction mode with PGOPTIONS. The import target is expected to be a local +ai_v2 database and is truncated by default. + +Modes: + all Export from source and import into local target. Default. + snapshot Export CSV files only. + import Import CSV files from SNAPSHOT_DIR only. + +Required for snapshot/all: + CENTAUR_PROD_DATABASE_URL + Read-only Postgres DSN for the production ai_v2 database. + +Optional production secret lookup: + CENTAUR_PROD_KUBE_CONTEXT + CENTAUR_PROD_NAMESPACE=centaur + CENTAUR_PROD_DATABASE_URL_SECRET_NAME + CENTAUR_PROD_DATABASE_URL_SECRET_KEY=DATABASE_URL + +Optional import target: + CENTAUR_LOCAL_DB_CONTAINER=codex-centaur-console-db + CENTAUR_LOCAL_CENTAUR_DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_v2 + +Snapshot sizing: + THREAD_LIMIT=250 + MESSAGE_LIMIT_PER_THREAD=120 + EXECUTION_LIMIT_PER_THREAD=20 + EVENT_LIMIT_PER_THREAD=40 + THINKING_EVENT_LIMIT_PER_THREAD=200 + +Safety: + TRUNCATE_LOCAL_SESSION_TABLES=1 + ALLOW_NONLOCAL_TARGET=0 + +After import, run Console with: + CENTAUR_CONSOLE_THREADS_READ_ONLY=1 +USAGE +} + +die() { + echo "error: $*" >&2 + exit 1 +} + +info() { + echo "==> $*" +} + +require_command() { + command -v "$1" >/dev/null 2>&1 || die "missing required command: $1" +} + +validate_integer() { + local name="$1" + local value="$2" + [[ "$value" =~ ^[0-9]+$ ]] || die "$name must be a non-negative integer" +} + +sql_quote_path() { + local value="$1" + printf "%s" "${value//\'/\'\'}" +} + +resolve_source_database_url() { + if [[ -n "${CENTAUR_PROD_DATABASE_URL:-}" ]]; then + printf "%s" "$CENTAUR_PROD_DATABASE_URL" + return + fi + + if [[ -n "${CENTAUR_PROD_KUBE_CONTEXT:-}" ]]; then + require_command kubectl + local secret_name="${CENTAUR_PROD_DATABASE_URL_SECRET_NAME:-}" + local secret_key="${CENTAUR_PROD_DATABASE_URL_SECRET_KEY:-DATABASE_URL}" + local namespace="${CENTAUR_PROD_NAMESPACE:-centaur}" + [[ -n "$secret_name" ]] || die \ + "set CENTAUR_PROD_DATABASE_URL to a read-only DSN, or set CENTAUR_PROD_DATABASE_URL_SECRET_NAME for kube lookup" + + kubectl --context "$CENTAUR_PROD_KUBE_CONTEXT" \ + -n "$namespace" \ + get secret "$secret_name" \ + -o "jsonpath={.data.${secret_key}}" | + python3 -c 'import base64, sys; sys.stdout.write(base64.b64decode(sys.stdin.read()).decode())' + return + fi + + die "set CENTAUR_PROD_DATABASE_URL to a read-only production ai_v2 DSN" +} + +assert_import_target_is_local() { + local source_url="${1:-}" + local target_url="$2" + + require_command python3 + python3 - "$source_url" "$target_url" "${ALLOW_NONLOCAL_TARGET:-0}" <<'PY' +import sys +from urllib.parse import urlparse + +source = urlparse(sys.argv[1]) if sys.argv[1] else None +target = urlparse(sys.argv[2]) +allow_nonlocal = sys.argv[3].lower() in {"1", "true", "yes"} + +local_hosts = { + "", + "localhost", + "127.0.0.1", + "::1", + "codex-centaur-console-db", + "host.docker.internal", +} +target_host = target.hostname or "" +target_db = (target.path or "").lstrip("/") + +if not allow_nonlocal and target_host not in local_hosts: + raise SystemExit( + f"target host {target_host!r} is not local; set ALLOW_NONLOCAL_TARGET=1 to override" + ) + +if source: + source_db = (source.path or "").lstrip("/") + same_host = (source.hostname or "") == target_host + same_port = (source.port or 5432) == (target.port or 5432) + same_db = source_db == target_db + if same_host and same_port and same_db: + raise SystemExit("source and target appear to point at the same database") +PY +} + +copy_to_csv() { + local database_url="$1" + local output_file="$2" + local sql="$3" + + PGOPTIONS="-c default_transaction_read_only=on -c statement_timeout=600000" \ + psql --no-psqlrc -X "$database_url" \ + -v ON_ERROR_STOP=1 \ + -c "copy ($sql) to stdout with (format csv, header true, force_quote *);" \ + > "$output_file" +} + +create_snapshot() { + local source_url="$1" + local snapshot_dir="$2" + + require_command psql + mkdir -p "$snapshot_dir" + + local recent_sessions_sql=" + select thread_key + from sessions + order by coalesce(updated_at, created_at) desc, thread_key + limit ${THREAD_LIMIT} + " + + info "exporting ${THREAD_LIMIT} recent sessions" + copy_to_csv "$source_url" "$snapshot_dir/sessions.csv" " + with recent_sessions as (${recent_sessions_sql}) + select + s.thread_key, + s.sandbox_id, + s.harness_type, + s.harness_thread_id, + s.iron_control_principal, + s.persona_id, + s.status, + s.metadata, + s.created_at, + s.updated_at + from sessions s + join recent_sessions r using (thread_key) + order by coalesce(s.updated_at, s.created_at) desc, s.thread_key + " + + info "exporting up to ${MESSAGE_LIMIT_PER_THREAD} messages per thread" + copy_to_csv "$source_url" "$snapshot_dir/session_messages.csv" " + with recent_sessions as (${recent_sessions_sql}), + ranked as ( + select + m.message_id, + m.thread_key, + m.client_message_id, + m.role, + m.parts, + m.metadata, + m.created_at, + row_number() over ( + partition by m.thread_key + order by m.created_at desc, m.message_id desc + ) as rn + from session_messages m + join recent_sessions r using (thread_key) + ) + select message_id, thread_key, client_message_id, role, parts, metadata, created_at + from ranked + where rn <= ${MESSAGE_LIMIT_PER_THREAD} + order by thread_key, created_at, message_id + " + + info "exporting up to ${EXECUTION_LIMIT_PER_THREAD} executions per thread" + copy_to_csv "$source_url" "$snapshot_dir/session_executions.csv" " + with recent_sessions as (${recent_sessions_sql}), + ranked as ( + select + e.execution_id, + e.thread_key, + e.idempotency_key, + e.status, + e.metadata, + e.error, + e.created_at, + e.updated_at, + e.started_at, + e.completed_at, + row_number() over ( + partition by e.thread_key + order by e.created_at desc, e.execution_id desc + ) as rn + from session_executions e + join recent_sessions r using (thread_key) + ) + select + execution_id, + thread_key, + idempotency_key, + status, + metadata, + error, + created_at, + updated_at, + started_at, + completed_at + from ranked + where rn <= ${EXECUTION_LIMIT_PER_THREAD} + order by thread_key, created_at, execution_id + " + + info "exporting up to ${EVENT_LIMIT_PER_THREAD} terminal events and ${THINKING_EVENT_LIMIT_PER_THREAD} reasoning lines per thread" + copy_to_csv "$source_url" "$snapshot_dir/session_events.csv" " + with recent_sessions as (${recent_sessions_sql}), + ranked as ( + select + ev.thread_key, + ev.execution_id, + ev.event_type, + ev.payload, + ev.created_at, + row_number() over ( + partition by ev.thread_key + order by ev.event_id desc + ) as rn + from session_events ev + join recent_sessions r using (thread_key) + where ev.event_type in ( + 'session.execution_completed', + 'session.execution_failed', + 'session.execution_cancelled' + ) + ), + -- Reasoning traces live in the session.output.line firehose as + -- item/completed notifications for reasoning items. The LIKE filter keeps + -- the export from paging every stdout line; Console re-filters exactly. + ranked_thinking as ( + select + ev.thread_key, + ev.execution_id, + ev.event_type, + ev.payload, + ev.created_at, + row_number() over ( + partition by ev.thread_key + order by ev.event_id desc + ) as rn + from session_events ev + join recent_sessions r using (thread_key) + where ev.event_type = 'session.output.line' + and ev.payload::text like '%reasoning%' + ) + select thread_key, execution_id, event_type, payload, created_at + from ( + select thread_key, execution_id, event_type, payload, created_at + from ranked + where rn <= ${EVENT_LIMIT_PER_THREAD} + union all + select thread_key, execution_id, event_type, payload, created_at + from ranked_thinking + where rn <= ${THINKING_EVENT_LIMIT_PER_THREAD} + ) combined + order by thread_key, created_at + " + + info "exporting Slack users referenced by mirrored threads" + copy_to_csv "$source_url" "$snapshot_dir/slack_sync_users.csv" " + with recent_sessions as (${recent_sessions_sql}), + message_mentions as ( + select coalesce(mention.match[1], mention.match[2]) as user_id + from session_messages m + join recent_sessions r using (thread_key) + cross join lateral regexp_matches( + m.parts::text, + '<@([UW][A-Z0-9]+)(?:\\|[^>]+)?>|@([UW][A-Z0-9]+)', + 'g' + ) as mention(match) + ), + event_mentions as ( + select coalesce(mention.match[1], mention.match[2]) as user_id + from session_events ev + join recent_sessions r using (thread_key) + cross join lateral regexp_matches( + ev.payload::text, + '<@([UW][A-Z0-9]+)(?:\\|[^>]+)?>|@([UW][A-Z0-9]+)', + 'g' + ) as mention(match) + ), + metadata_users as ( + select s.metadata ->> key.name as user_id + from sessions s + join recent_sessions r using (thread_key) + cross join (values ('slack_user_id'), ('user_id'), ('actor_user_id')) as key(name) + union all + select m.metadata ->> key.name as user_id + from session_messages m + join recent_sessions r using (thread_key) + cross join (values ('slack_user_id'), ('user_id'), ('actor_user_id')) as key(name) + ), + referenced_users as ( + select distinct nullif(user_id, '') as user_id + from ( + select user_id from message_mentions + union all + select user_id from event_mentions + union all + select user_id from metadata_users + ) ids + where nullif(user_id, '') is not null + ) + select + u.user_id, + u.user_name, + u.real_name, + u.display_name, + u.is_bot, + u.is_deleted, + u.team_id, + u.raw_payload, + u.first_seen_at, + u.last_seen_at, + u.updated_at + from slack_sync_users u + join referenced_users r using (user_id) + order by u.user_id + " + + { + echo "created_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)" + echo "thread_limit=${THREAD_LIMIT}" + echo "message_limit_per_thread=${MESSAGE_LIMIT_PER_THREAD}" + echo "execution_limit_per_thread=${EXECUTION_LIMIT_PER_THREAD}" + echo "event_limit_per_thread=${EVENT_LIMIT_PER_THREAD}" + echo "thinking_event_limit_per_thread=${THINKING_EVENT_LIMIT_PER_THREAD}" + echo "slack_sync_users=referenced" + } > "$snapshot_dir/manifest.env" + + info "snapshot written to $snapshot_dir" +} + +write_import_sql() { + local import_root="$1" + local output_file="$2" + local sessions_csv messages_csv executions_csv events_csv slack_users_csv + sessions_csv="$(sql_quote_path "$import_root/sessions.csv")" + messages_csv="$(sql_quote_path "$import_root/session_messages.csv")" + executions_csv="$(sql_quote_path "$import_root/session_executions.csv")" + events_csv="$(sql_quote_path "$import_root/session_events.csv")" + slack_users_csv="$(sql_quote_path "$import_root/slack_sync_users.csv")" + + cat > "$output_file" <> "$output_file" <<'SQL' +truncate table session_events, session_messages, session_executions, sessions cascade; + +SQL + fi + + cat >> "$output_file" <<'SQL' +create table if not exists slack_sync_users ( + user_id text primary key, + user_name text not null default '', + real_name text not null default '', + display_name text not null default '', + is_bot boolean not null default false, + is_deleted boolean not null default false, + team_id text not null default '', + raw_payload jsonb not null default '{}'::jsonb, + first_seen_at timestamptz not null default now(), + last_seen_at timestamptz not null default now(), + updated_at timestamptz not null default now() +); + +create index if not exists idx_slack_sync_users_real_name + on slack_sync_users (real_name); + +insert into sessions ( + thread_key, + sandbox_id, + harness_type, + harness_thread_id, + iron_control_principal, + persona_id, + status, + metadata, + created_at, + updated_at +) +select + thread_key, + sandbox_id, + harness_type, + harness_thread_id, + iron_control_principal, + persona_id, + status, + metadata, + created_at, + updated_at +from import_sessions +on conflict (thread_key) do update set + sandbox_id = excluded.sandbox_id, + harness_type = excluded.harness_type, + harness_thread_id = excluded.harness_thread_id, + iron_control_principal = excluded.iron_control_principal, + persona_id = excluded.persona_id, + status = excluded.status, + metadata = excluded.metadata, + created_at = excluded.created_at, + updated_at = excluded.updated_at; + +insert into session_messages ( + message_id, + thread_key, + client_message_id, + role, + parts, + metadata, + created_at +) +select + message_id, + thread_key, + client_message_id, + role, + parts, + metadata, + created_at +from import_session_messages +on conflict (message_id) do update set + thread_key = excluded.thread_key, + client_message_id = excluded.client_message_id, + role = excluded.role, + parts = excluded.parts, + metadata = excluded.metadata, + created_at = excluded.created_at; + +insert into session_executions ( + execution_id, + thread_key, + idempotency_key, + status, + metadata, + error, + created_at, + updated_at, + started_at, + completed_at +) +select + execution_id, + thread_key, + idempotency_key, + status, + metadata, + error, + created_at, + updated_at, + started_at, + completed_at +from import_session_executions +on conflict (execution_id) do update set + thread_key = excluded.thread_key, + idempotency_key = excluded.idempotency_key, + status = excluded.status, + metadata = excluded.metadata, + error = excluded.error, + created_at = excluded.created_at, + updated_at = excluded.updated_at, + started_at = excluded.started_at, + completed_at = excluded.completed_at; + +insert into session_events ( + thread_key, + execution_id, + event_type, + payload, + created_at +) +select + ev.thread_key, + ev.execution_id, + ev.event_type, + ev.payload, + ev.created_at +from import_session_events ev +where ev.execution_id is null + or exists ( + select 1 + from session_executions imported_execution + where imported_execution.execution_id = ev.execution_id + ); + +insert into slack_sync_users ( + user_id, + user_name, + real_name, + display_name, + is_bot, + is_deleted, + team_id, + raw_payload, + first_seen_at, + last_seen_at, + updated_at +) +select + user_id, + coalesce(user_name, ''), + coalesce(real_name, ''), + coalesce(display_name, ''), + coalesce(is_bot, false), + coalesce(is_deleted, false), + coalesce(team_id, ''), + coalesce(raw_payload, '{}'::jsonb), + coalesce(first_seen_at, now()), + coalesce(last_seen_at, now()), + coalesce(updated_at, now()) +from import_slack_sync_users +where nullif(user_id, '') is not null +on conflict (user_id) do update set + user_name = excluded.user_name, + real_name = excluded.real_name, + display_name = excluded.display_name, + is_bot = excluded.is_bot, + is_deleted = excluded.is_deleted, + team_id = excluded.team_id, + raw_payload = excluded.raw_payload, + first_seen_at = excluded.first_seen_at, + last_seen_at = excluded.last_seen_at, + updated_at = excluded.updated_at; + +analyze sessions; +analyze session_messages; +analyze session_executions; +analyze session_events; +analyze slack_sync_users; + +commit; +SQL +} + +import_snapshot() { + local snapshot_dir="$1" + local target_url="$2" + + [[ -f "$snapshot_dir/sessions.csv" ]] || die "missing $snapshot_dir/sessions.csv" + [[ -f "$snapshot_dir/session_messages.csv" ]] || die "missing $snapshot_dir/session_messages.csv" + [[ -f "$snapshot_dir/session_executions.csv" ]] || die "missing $snapshot_dir/session_executions.csv" + [[ -f "$snapshot_dir/session_events.csv" ]] || die "missing $snapshot_dir/session_events.csv" + [[ -f "$snapshot_dir/slack_sync_users.csv" ]] || die "missing $snapshot_dir/slack_sync_users.csv" + + local local_container="${CENTAUR_LOCAL_DB_CONTAINER:-codex-centaur-console-db}" + local use_container="${USE_LOCAL_DB_CONTAINER:-auto}" + local import_root="$snapshot_dir" + local import_sql="$snapshot_dir/import.sql" + + if [[ "$use_container" == "auto" ]] && command -v docker >/dev/null 2>&1 \ + && docker inspect "$local_container" >/dev/null 2>&1; then + use_container="1" + fi + + if [[ "$use_container" == "1" || "$use_container" == "true" ]]; then + require_command docker + import_root="/tmp/centaur-thread-snapshot" + write_import_sql "$import_root" "$import_sql" + + info "copying snapshot into local database container $local_container" + docker exec "$local_container" sh -c "rm -rf '$import_root' && mkdir -p '$import_root'" + docker cp "$snapshot_dir/." "$local_container:$import_root/" + + info "importing snapshot into local target via $local_container" + docker exec -i "$local_container" psql --no-psqlrc -X "$target_url" < "$import_sql" + else + require_command psql + write_import_sql "$import_root" "$import_sql" + + info "importing snapshot into local target" + psql --no-psqlrc -X "$target_url" < "$import_sql" + fi +} + +main() { + local mode="${1:-all}" + if [[ "$mode" == "-h" || "$mode" == "--help" ]]; then + usage + exit 0 + fi + [[ "$mode" =~ ^(all|snapshot|import)$ ]] || die "unknown mode: $mode" + + THREAD_LIMIT="${THREAD_LIMIT:-250}" + MESSAGE_LIMIT_PER_THREAD="${MESSAGE_LIMIT_PER_THREAD:-120}" + EXECUTION_LIMIT_PER_THREAD="${EXECUTION_LIMIT_PER_THREAD:-20}" + EVENT_LIMIT_PER_THREAD="${EVENT_LIMIT_PER_THREAD:-40}" + THINKING_EVENT_LIMIT_PER_THREAD="${THINKING_EVENT_LIMIT_PER_THREAD:-200}" + TRUNCATE_LOCAL_SESSION_TABLES="${TRUNCATE_LOCAL_SESSION_TABLES:-1}" + + validate_integer THREAD_LIMIT "$THREAD_LIMIT" + validate_integer MESSAGE_LIMIT_PER_THREAD "$MESSAGE_LIMIT_PER_THREAD" + validate_integer EXECUTION_LIMIT_PER_THREAD "$EXECUTION_LIMIT_PER_THREAD" + validate_integer EVENT_LIMIT_PER_THREAD "$EVENT_LIMIT_PER_THREAD" + + local snapshot_dir="${SNAPSHOT_DIR:-}" + if [[ -z "$snapshot_dir" ]]; then + snapshot_dir="$(mktemp -d "${TMPDIR:-/tmp}/centaur-thread-snapshot.XXXXXX")" + fi + + local source_url="" + if [[ "$mode" == "all" || "$mode" == "snapshot" ]]; then + source_url="$(resolve_source_database_url)" + fi + + local target_url="${CENTAUR_LOCAL_CENTAUR_DATABASE_URL:-${CENTAUR_CONSOLE_CENTAUR_DATABASE_URL:-${TARGET_DATABASE_URL:-postgresql://postgres:postgres@localhost:5432/ai_v2}}}" + if [[ "$mode" == "all" || "$mode" == "import" ]]; then + assert_import_target_is_local "$source_url" "$target_url" + fi + + if [[ "$mode" == "all" || "$mode" == "snapshot" ]]; then + create_snapshot "$source_url" "$snapshot_dir" + fi + + if [[ "$mode" == "all" || "$mode" == "import" ]]; then + import_snapshot "$snapshot_dir" "$target_url" + info "import complete" + info "restart Console with CENTAUR_CONSOLE_THREADS_READ_ONLY=1 before browsing mirrored data" + fi +} + +main "$@" diff --git a/scripts/probe-agent-harness-image.sh b/scripts/probe-agent-harness-image.sh new file mode 100644 index 000000000..90f280cdf --- /dev/null +++ b/scripts/probe-agent-harness-image.sh @@ -0,0 +1,200 @@ +#!/usr/bin/env bash +set -euo pipefail +IFS=$'\n\t' + +script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +repo_root="$(cd "${script_dir}/.." && pwd)" +image="${1:-}" + +if [[ -z "$image" || "$image" == *$'\n'* ]]; then + echo "usage: $0 IMAGE" >&2 + exit 2 +fi +if ! command -v docker >/dev/null 2>&1; then + echo "missing required command: docker" >&2 + exit 1 +fi +if ! docker image inspect "$image" >/dev/null 2>&1; then + echo "agent image is not loaded locally: $image" >&2 + exit 1 +fi + +codex_version="$( + awk -F= '$1 == "ARG CODEX_VERSION" { print $2; exit }' \ + "${repo_root}/services/sandbox/Dockerfile" +)" +if [[ ! "$codex_version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "could not derive the exact CODEX_VERSION from the sandbox Dockerfile" >&2 + exit 1 +fi + +docker run --rm -i \ + --network none \ + --entrypoint /bin/bash \ + --env "EXPECTED_CODEX_VERSION=codex-cli ${codex_version}" \ + --env OPENAI_API_KEY= \ + --env CODEX_API_KEY= \ + --env OPENROUTER_API_KEY= \ + --env META_AI_API_KEY= \ + "$image" -seu <<'CONTAINER_SCRIPT' +set -euo pipefail +IFS=$'\n\t' + +export HOME=/tmp/centaur-harness-probe +export CODEX_HOME="$HOME/.codex" +mkdir -p "$CODEX_HOME" +cp /home/agent/harness/codex/config.toml "$CODEX_HOME/config.toml" + +python3 - <<'PY' +from __future__ import annotations + +import json +import os +import select +import subprocess +import time +import tomllib +from pathlib import Path + + +BAKED_HARNESS = Path("/home/agent/harness") + + +with (BAKED_HARNESS / "codex/config.toml").open("rb") as handle: + baked_codex = tomllib.load(handle) +with (BAKED_HARNESS / "claude/settings.json").open(encoding="utf-8") as handle: + baked_claude = json.load(handle) + +assert baked_codex["model_providers"]["openrouter"] == { + "name": "OpenRouter", + "base_url": "https://openrouter.ai/api/v1", + "env_key": "OPENROUTER_API_KEY", + "wire_api": "responses", + "requires_openai_auth": False, +} +assert baked_codex["model_providers"]["responses"] == { + "name": "azure", + "base_url": "https://api.ai.meta.com/v1", + "env_key": "META_AI_API_KEY", + "wire_api": "responses", + "requires_openai_auth": False, +} +assert baked_codex["projects"]["/"]["trust_level"] == "trusted" +assert baked_claude["permissions"]["defaultMode"] == "bypassPermissions" +assert isinstance(baked_claude["model"], str) and baked_claude["model"] + +version = subprocess.run( + ["codex", "--version"], + check=True, + capture_output=True, + text=True, +).stdout.strip() +assert version == os.environ["EXPECTED_CODEX_VERSION"], ( + f"unexpected packaged Codex version: {version!r}" +) + +features_output = subprocess.run( + ["codex", "features", "list"], + check=True, + capture_output=True, + text=True, +).stdout +features = { + fields[0]: fields[-1] + for line in features_output.splitlines() + if len(fields := line.split()) >= 2 +} +assert features.get("multi_agent") == "false" +assert features.get("multi_agent_v2") == "false" + + +def send(process: subprocess.Popen[str], request: dict) -> None: + assert process.stdin is not None + process.stdin.write(json.dumps(request, separators=(",", ":")) + "\n") + process.stdin.flush() + + +def read_response(process: subprocess.Popen[str], request_id: int) -> dict: + assert process.stdout is not None + deadline = time.monotonic() + 10 + while True: + remaining = deadline - time.monotonic() + if remaining <= 0: + raise AssertionError(f"timed out waiting for app-server response {request_id}") + readable, _, _ = select.select([process.stdout], [], [], remaining) + if not readable: + raise AssertionError(f"timed out waiting for app-server response {request_id}") + line = process.stdout.readline() + if not line: + raise AssertionError( + f"app-server exited before response {request_id}: {process.poll()}" + ) + message = json.loads(line) + if message.get("id") != request_id: + continue + if "error" in message: + raise AssertionError( + f"app-server rejected response {request_id}: {message['error']!r}" + ) + return message + + +def probe_provider(provider: str, model: str) -> None: + process = subprocess.Popen( + ["codex", "app-server", "--listen", "stdio://"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + try: + send( + process, + { + "id": 1, + "method": "initialize", + "params": { + "clientInfo": { + "name": "centaur-image-probe", + "title": None, + "version": "0", + }, + "capabilities": None, + }, + }, + ) + read_response(process, 1) + send( + process, + { + "id": 2, + "method": "thread/start", + "params": { + "approvalPolicy": "never", + "sandbox": "danger-full-access", + "model": model, + "modelProvider": provider, + }, + }, + ) + response = read_response(process, 2) + thread_id = response.get("result", {}).get("thread", {}).get("id") + assert isinstance(thread_id, str) and thread_id + finally: + if process.poll() is None: + process.terminate() + try: + process.wait(timeout=5) + except subprocess.TimeoutExpired: + process.kill() + process.wait(timeout=5) + + +# No turn/start request is sent. Docker also disables the container network, so +# these checks exercise packaged provider discovery without a model call. +probe_provider("openrouter", "openrouter/auto") +probe_provider("responses", "meta-llama/llama-4-maverick") + +print("packaged agent harness probe passed") +PY +CONTAINER_SCRIPT diff --git a/services/AGENTS.md b/services/AGENTS.md new file mode 100644 index 000000000..3415f7e76 --- /dev/null +++ b/services/AGENTS.md @@ -0,0 +1,58 @@ +# Service Development Guide + +These instructions apply to all tracked services. Also follow the repository +root guide and the nearest service-local `AGENTS.md`. + +## Shared rules + +- Keep service ownership explicit. Chat services own transport and rendering; + `api-rs` owns durable orchestration; the sandbox owns harness adaptation; the + workflow host executes Python handlers but does not own durable state. +- Treat HTTP routes, NDJSON/SSE shapes, thread keys, database rows, environment + variables, health checks, metrics, and shutdown behavior as contracts. +- When a runtime setting, port, route, probe, secret reference, or network path + changes, inspect the matching files under `contrib/chart/`. +- Use structured logs with stable event names and correlation fields. Never log + authorization headers, cookies, tokens, secret values, raw credential + payloads, or unnecessarily large webhook bodies. +- Preserve fail-closed policy gates. Authenticate or verify signatures before + accepting untrusted input, and keep allowlists and attachment restrictions + conservative. +- Add focused regression coverage next to the behavior changed. For a + cross-service contract, test both sides or provide an integration test that + crosses the boundary. +- For credentialed tools, prove the full path: declared tool metadata -> + principal/role authorization -> control-plane and proxy sync -> injected + outbound request. Confirm the sandbox contains placeholders, not real secret + values. + +## Chat ingress services + +Chat integrations should verify the platform event, derive a stable thread +identity, persist the message, start or append to the durable session, and +render replayable events. Do not move sandbox lifecycle, harness translation, +workflow durability, or credential resolution into an ingress service. + +Keep these failure boundaries distinct: + +- webhook acknowledgement versus background execution completion; +- message persistence versus execution start; +- execution completion versus final platform delivery; +- retryable transport failures versus permanent validation failures; +- deduplication versus per-session serialization. + +Tests should cover signature/auth rejection, self-message loops, duplicate +deliveries, session API failures, replay/reconnect behavior, and terminal render +outcomes where applicable. + +All TypeScript services belong to the root pnpm workspace. Install dependencies +once from the repository root with `pnpm install --frozen-lockfile`; their +scripts invoke Bun for runtime, tests, and type checking. Do not create nested +lockfiles. If Chat SDK behavior is unclear, inspect `~/github/vercel/chat` and +the repository's registered patches, not `node_modules`. + +## Validation + +After unit checks, use the root local-stack flow when behavior crosses process, +database, proxy, sandbox, or platform-emulation boundaries. Do not use a remote +deployment as a development test environment. diff --git a/services/api-rs/AGENTS.md b/services/api-rs/AGENTS.md new file mode 100644 index 000000000..4d6508692 --- /dev/null +++ b/services/api-rs/AGENTS.md @@ -0,0 +1,95 @@ +# api-rs Guide + +## Role + +`api-rs` is the Rust control plane. It owns durable sessions and events, +sandbox assignment and recovery, execution serialization, workflow state, +service authentication, and control-plane telemetry. Postgres is the source of +truth; process-local maps and attach streams are recoverable caches. + +Important crate boundaries: + +- `centaur-api-server`: HTTP routes, middleware, startup, health, and metrics. +- `centaur-session-core`: shared session types and backend-neutral contracts. +- `centaur-session-runtime`: orchestration, execution, recovery, and lifecycle. +- `centaur-session-sqlx`: persistence and embedded SQLx migrations. +- `centaur-sandbox-*`: backend-neutral sandbox contract and implementations. +- `centaur-workflows` and `absurd-sdk`: durable workflow scheduling and state. +- `centaur-iron-control`, `centaur-iron-proxy`, and `centaur-perms`: credential + control-plane integration and authorization resources. +- `centaur-telemetry`: shared tracing and metrics support. + +Read the relevant RFC under `rfcs/` before changing a core protocol. + +## Invariants + +- The session flow remains create/reuse -> append messages -> execute -> replay + events. Persist state transitions before reporting them to clients. +- `input_lines` are opaque, single-line NDJSON strings at the API boundary. + Add trace/session context without teaching the control plane every harness's + input format; harness-specific translation belongs in the runtime adapter. +- Execution idempotency, per-session serialization, cancellation, leases, and + terminal events must remain correct across retries and process restarts. +- New durable state belongs in Postgres, with repository methods and recovery + tests. Do not introduce a process-local source of truth. +- Keep ingress/platform behavior out of the API. Keep Kubernetes-specific code + behind sandbox backend interfaces. +- Authorization must be checked at the resource boundary. A valid token alone + is not proof that the caller may read another session, tool, or file. +- Logs and durable events must not contain bearer tokens, secret values, or raw + credential material. + +## Database changes + +Migrations live in `crates/centaur-session-sqlx/migrations` and are embedded in +the binary and tests. Add the next numbered SQL file; never edit or reorder an +applied migration. Update SQLx repository code and add database-backed coverage +for upgrade, read/write, and recovery behavior. + +Database-backed tests skip when their URL is absent. Point these variables at a +disposable Postgres as required by the packages you run: + +- `SESSION_RUNTIME_TEST_DATABASE_URL`: session SQLx, runtime, and warm-pool + tests; the SQLx RLS integration tests also accept it as a fallback. +- `SESSION_SQLX_TEST_DATABASE_URL`: SQLx RLS integration tests specifically. +- `ABSURD_TEST_DATABASE_URL`: `absurd-sdk` database tests. + +Do not report full database coverage from `cargo test --workspace` unless the +relevant variables were set and the database-backed tests actually ran. + +## Validation + +From `services/api-rs`: + +```bash +cargo fmt --all --check +cargo clippy --workspace --all-targets -- -D warnings +cargo test --workspace +``` + +During iteration, prefer a focused package/test first, for example: + +```bash +cargo test -p centaur-session-runtime +cargo test -p centaur-session-sqlx +cargo test -p centaur-workflows +``` + +Sandbox backend invariants have a local Kind suite. Prepare the cluster and +images with the `kind-e2e-*` recipes, then run all integration test binaries +(the older `e2e-kind` wrapper names a removed test target): + +```bash +just kind-e2e-up +just kind-e2e-build-images +KIND_E2E_FORCE_IMAGE_LOAD=1 just kind-e2e-load-images +SANDBOX_E2E_IMPLS=all \ +SANDBOX_E2E_K8S_CONTEXT=kind-centaur-api-rs-e2e \ +SANDBOX_E2E_K8S_NAMESPACE=centaur-sandbox-e2e \ +cargo test -p centaur-sandbox-e2e --tests -- --ignored --nocapture +``` + +For an API contract or runtime change, also build the API image, deploy to the local +stack, drive a real session through create/append/execute/events, and verify the +durable rows and terminal event. Use explicit contexts for any Kind command so +an ambient Kubernetes context cannot redirect a destructive operation. diff --git a/services/api-rs/Cargo.lock b/services/api-rs/Cargo.lock index a5034c645..7b9e5e04f 100644 --- a/services/api-rs/Cargo.lock +++ b/services/api-rs/Cargo.lock @@ -186,6 +186,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5ec2f1fc3ec205783a5da9a7e6c1509cc69dedf09a1949e412c1e18469326d00" dependencies = [ "aws-lc-sys", + "untrusted 0.7.1", "zeroize", ] @@ -814,6 +815,7 @@ dependencies = [ "futures-util", "hex", "hmac 0.12.1", + "jsonwebtoken", "kube", "reqwest", "rustls 0.23.40", @@ -934,8 +936,11 @@ version = "0.1.0" dependencies = [ "async-trait", "centaur-sandbox-core", + "centaur-session-core", "centaur-session-sqlx", "centaur-telemetry", + "serde_json", + "sqlx", "thiserror", "tokio", "tracing", @@ -981,9 +986,11 @@ dependencies = [ "centaur-telemetry", "dashmap", "futures-util", + "reqwest", "serde", "serde_json", "sha2 0.10.9", + "sqlx", "thiserror", "time", "tokio", @@ -1030,6 +1037,7 @@ name = "centaur-workflows" version = "0.1.0" dependencies = [ "absurd-sdk", + "base64", "centaur-sandbox-core", "centaur-session-core", "centaur-session-runtime", @@ -1039,9 +1047,11 @@ dependencies = [ "chrono-tz", "cron", "futures-util", + "hmac 0.12.1", "reqwest", "serde", "serde_json", + "sha2 0.10.9", "sqlx", "thiserror", "time", @@ -2529,6 +2539,22 @@ dependencies = [ "thiserror", ] +[[package]] +name = "jsonwebtoken" +version = "10.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eba32bfb4ffdeaca3e34431072faf01745c9b26d25504aa7a6cf5684334fc4fc" +dependencies = [ + "aws-lc-rs", + "base64", + "getrandom 0.2.17", + "js-sys", + "serde", + "serde_json", + "signature", + "zeroize", +] + [[package]] name = "k8s-openapi" version = "0.27.1" @@ -3635,6 +3661,7 @@ dependencies = [ "rustls-platform-verifier", "serde", "serde_json", + "serde_urlencoded", "sync_wrapper", "tokio", "tokio-rustls 0.26.4", @@ -3669,7 +3696,7 @@ dependencies = [ "cfg-if", "getrandom 0.2.17", "libc", - "untrusted", + "untrusted 0.9.0", "windows-sys 0.52.0", ] @@ -3805,7 +3832,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8b6275d1ee7a1cd780b64aca7726599a1dbc893b1e64144529e55c3c2f745765" dependencies = [ "ring", - "untrusted", + "untrusted 0.9.0", ] [[package]] @@ -3817,7 +3844,7 @@ dependencies = [ "aws-lc-rs", "ring", "rustls-pki-types", - "untrusted", + "untrusted 0.9.0", ] [[package]] @@ -3894,7 +3921,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "da046153aa2352493d6cb7da4b6e5c0c057d8a1d0a9aa8560baffdd945acd414" dependencies = [ "ring", - "untrusted", + "untrusted 0.9.0", ] [[package]] @@ -5045,6 +5072,12 @@ version = "0.2.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "673aac59facbab8a9007c7f6108d11f63b603f7cabff99fabf650fea5c32b861" +[[package]] +name = "untrusted" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a156c684c91ea7d62626509bce3cb4e1d9ed5c4d978f7b4352658f96a4c26b4a" + [[package]] name = "untrusted" version = "0.9.0" @@ -5837,6 +5870,20 @@ name = "zeroize" version = "1.8.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b97154e67e32c85465826e8bcc1c59429aaaf107c1e4a9e53c8d8ccd5eff88d0" +dependencies = [ + "zeroize_derive", +] + +[[package]] +name = "zeroize_derive" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] [[package]] name = "zerotrie" diff --git a/services/api-rs/Cargo.toml b/services/api-rs/Cargo.toml index ff383d4a2..07b789c09 100644 --- a/services/api-rs/Cargo.toml +++ b/services/api-rs/Cargo.toml @@ -67,13 +67,14 @@ futures-util = { version = "0.3", features = ["sink"] } hmac = "0.12" hex = "0.4" jiff = "0.2" +jsonwebtoken = { version = "10.4.0", default-features = false, features = ["aws_lc_rs"] } k8s-openapi = { version = "0.27.1", features = ["latest"] } kube = "3.1.0" serde = { version = "1", features = ["derive"] } serde_json = "1" serde_yaml = "0.9" sha2 = "0.10" -reqwest = { version = "0.13.4", default-features = false, features = ["json", "rustls", "stream"] } +reqwest = { version = "0.13.4", default-features = false, features = ["form", "json", "rustls", "stream"] } ratatui = "0.29" rustls = "0.23" opentelemetry = "0.32.0" diff --git a/services/api-rs/Dockerfile b/services/api-rs/Dockerfile index 9e2950116..b61ca2de2 100644 --- a/services/api-rs/Dockerfile +++ b/services/api-rs/Dockerfile @@ -16,7 +16,6 @@ COPY services/api-rs/ ./ ARG RUST_BUILD_PROFILE=release RUN --mount=type=cache,target=/usr/local/cargo/registry,sharing=locked \ --mount=type=cache,target=/usr/local/cargo/git,sharing=locked \ - --mount=type=cache,target=/build/target,sharing=locked \ case "$RUST_BUILD_PROFILE" in \ release) cargo build --release -p centaur-api-server && cp target/release/centaur-api-server /usr/local/bin/centaur-api-server ;; \ debug|dev) cargo build -p centaur-api-server && cp target/debug/centaur-api-server /usr/local/bin/centaur-api-server ;; \ @@ -29,9 +28,10 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ apt-get update && apt-get install -y --no-install-recommends ca-certificates curl python3 python3-pip COPY --from=builder /usr/local/bin/centaur-api-server /usr/local/bin/centaur-api-server WORKDIR /app +COPY centaur_sdk/ /app/centaur_sdk/ COPY services/workflow-python/ /app/workflow-python/ RUN --mount=type=cache,target=/root/.cache/pip,sharing=locked \ - pip3 install --break-system-packages --no-compile /app/workflow-python \ + pip3 install --break-system-packages --no-compile /app/centaur_sdk /app/workflow-python \ && python3 -c "import boto3, botocore" \ && rm -rf /usr/share/doc /usr/share/man /usr/share/info # api-rs discovers tool secret metadata from pyproject.toml at startup so it can diff --git a/services/api-rs/crates/absurd-sdk/src/lib.rs b/services/api-rs/crates/absurd-sdk/src/lib.rs index 9a2de812e..7b67e3be1 100644 --- a/services/api-rs/crates/absurd-sdk/src/lib.rs +++ b/services/api-rs/crates/absurd-sdk/src/lib.rs @@ -402,6 +402,26 @@ impl Worker { let _ = self.shutdown.send(true); self.join.await? } + + /// Stop intake immediately, then wait at most `timeout` for already + /// claimed handlers. On timeout abort the worker loop; dropping its + /// JoinSet aborts every remaining handler so caller cleanup guards run and + /// the task lease can be recovered by another worker. + pub async fn close_with_timeout(self, timeout: Duration) -> Result { + let _ = self.shutdown.send(true); + let mut join = self.join; + match tokio::time::timeout(timeout, &mut join).await { + Ok(result) => { + result??; + Ok(true) + } + Err(_) => { + join.abort(); + let _ = join.await; + Ok(false) + } + } + } } impl Client { @@ -870,6 +890,21 @@ impl Client { } }; + // A close signal can arrive while the database claim query is in + // flight. Do not launch that freshly claimed batch on a draining + // worker; relinquish it for another replica and exit intake. + if *shutdown.borrow() { + for task in tasks { + if let Err(err) = self + .defer_claimed_run(&task.run_id, options.poll_interval) + .await + { + on_error(err); + } + } + break; + } + if tasks.is_empty() { tokio::select! { changed = shutdown.changed() => { @@ -1212,6 +1247,31 @@ impl TaskContext { self.inner.headers.clone() } + /// Spawn a child task on this context's queue through the same registered + /// client as the parent worker. This is the authenticated in-process path + /// for workflow hosts; no control-plane HTTP credential crosses into the + /// child process. + pub async fn spawn_child( + &self, + task_name: &str, + params: P, + mut options: SpawnOptions, + ) -> Result { + if let Some(queue) = options.queue.as_deref() { + if validate_queue_name(queue)? != self.inner.queue_name { + return Err(Error::QueueMismatch { + task_name: task_name.to_owned(), + registered_queue: self.inner.queue_name.clone(), + requested_queue: queue.to_owned(), + }); + } + } + // Keep the parent queue explicit so `Client::resolve_spawn` also + // verifies that a registered task has not been routed elsewhere. + options.queue = Some(self.inner.queue_name.clone()); + self.inner.client.spawn(task_name, params, options).await + } + pub async fn step(&self, name: &str, f: F) -> Result where T: Serialize + DeserializeOwned + Send + 'static, @@ -2036,7 +2096,7 @@ fn panic_message(payload: &(dyn std::any::Any + Send)) -> String { mod tests { use super::*; use std::{ - sync::atomic::{AtomicUsize, Ordering}, + sync::atomic::{AtomicBool, AtomicUsize, Ordering}, sync::OnceLock, time::{SystemTime, UNIX_EPOCH}, }; @@ -2125,6 +2185,64 @@ mod tests { assert!(!payload.contains_key("queue")); } + #[tokio::test] + async fn spawn_child_rejects_a_task_registered_on_another_queue() -> Result<()> { + let parent_queue = unique_queue("parent_queue"); + let child_queue = unique_queue("child_queue"); + let pool = PgPoolOptions::new() + .connect_lazy("postgres://postgres:postgres@localhost/absurd_test")?; + let client = Client::from_pool_with_options( + pool, + ClientOptions { + queue_name: parent_queue.clone(), + ..ClientOptions::default() + }, + )?; + client.register_task_with::( + TaskRegistrationOptions { + name: "other-queue-task".to_owned(), + queue: Some(child_queue.clone()), + default_max_attempts: None, + default_cancellation: None, + }, + |_params, _ctx| async { Ok(json!({"ok": true})) }, + )?; + let ctx = TaskContext { + inner: Arc::new(TaskContextInner { + client, + queue_name: parent_queue.clone(), + task_id: "00000000-0000-0000-0000-000000000001".to_owned(), + run_id: "00000000-0000-0000-0000-000000000002".to_owned(), + task_name: "parent-task".to_owned(), + attempt: 1, + claim_timeout: Duration::from_secs(30), + headers: Map::new(), + wake_event: Mutex::new(None), + event_payload: Mutex::new(None), + checkpoint_cache: Mutex::new(HashMap::new()), + step_name_counter: Mutex::new(HashMap::new()), + on_lease_extended: Arc::new(|_| {}), + }), + }; + + let error = ctx + .spawn_child("other-queue-task", json!({}), SpawnOptions::default()) + .await + .expect_err("a child task registered on another queue must be rejected"); + assert!(matches!( + error, + Error::QueueMismatch { + task_name, + registered_queue, + requested_queue, + } if task_name == "other-queue-task" + && registered_queue == child_queue + && requested_queue == parent_queue + )); + + Ok(()) + } + #[test] fn task_result_snapshot_uses_parity_json_shape() { let snapshot = TaskResultSnapshot::Completed { @@ -2440,4 +2558,77 @@ mod tests { drop(worker); Ok(()) } + + #[tokio::test] + async fn integration_worker_close_timeout_aborts_active_handlers_when_database_url_is_set( + ) -> Result<()> { + let Some(pool) = optional_test_pool().await? else { + return Ok(()); + }; + + struct DropSignal(Arc); + impl Drop for DropSignal { + fn drop(&mut self) { + self.0.store(true, Ordering::SeqCst); + } + } + + let queue = unique_queue("rust_close_timeout"); + let app = Client::from_pool_with_options( + pool, + ClientOptions { + queue_name: queue, + ..ClientOptions::default() + }, + )?; + app.create_queue(None, Default::default()).await?; + let started = Arc::new(Notify::new()); + let dropped = Arc::new(AtomicBool::new(false)); + app.register_task("never", { + let started = started.clone(); + let dropped = dropped.clone(); + move |_params: Value, _ctx| { + let started = started.clone(); + let dropped = dropped.clone(); + async move { + let _drop_signal = DropSignal(dropped); + started.notify_one(); + std::future::pending::>().await + } + } + })?; + app.register_task("after-close", |_params: Value, _ctx| async move { + Ok(json!({"unexpected": true})) + })?; + app.spawn("never", json!({}), Default::default()).await?; + let worker = app.start_worker(WorkerOptions { + worker_id: Some("rust-close-timeout-worker".to_owned()), + poll_interval: Duration::from_millis(10), + fatal_on_lease_timeout: false, + ..WorkerOptions::default() + }); + tokio::time::timeout(Duration::from_secs(2), started.notified()) + .await + .map_err(|_| Error::Timeout("never task did not start".to_owned()))?; + + assert!(!worker.close_with_timeout(Duration::from_millis(50)).await?); + tokio::time::timeout(Duration::from_secs(1), async { + while !dropped.load(Ordering::SeqCst) { + tokio::task::yield_now().await; + } + }) + .await + .map_err(|_| Error::Timeout("active handler was not aborted".to_owned()))?; + + let after = app + .spawn("after-close", json!({}), Default::default()) + .await?; + tokio::time::sleep(Duration::from_millis(100)).await; + assert_eq!( + app.fetch_task_result(&after.task_id, None).await?, + Some(TaskResultSnapshot::Pending), + "closed worker must not claim new work" + ); + Ok(()) + } } diff --git a/services/api-rs/crates/centaur-api-integration-test/src/main.rs b/services/api-rs/crates/centaur-api-integration-test/src/main.rs index 61a6e638c..e07a16890 100644 --- a/services/api-rs/crates/centaur-api-integration-test/src/main.rs +++ b/services/api-rs/crates/centaur-api-integration-test/src/main.rs @@ -17,6 +17,26 @@ const DEFAULT_API_URL: &str = "http://127.0.0.1:18080"; const SOURCE_PATH: &str = "services/api-rs/crates/centaur-api-integration-test/src/main.rs"; const TEST_MODEL: &str = "gpt-api-integration-test"; +fn workflow_api_key() -> Result { + env::var("WORKFLOW_API_KEY") + .context("WORKFLOW_API_KEY is required for workflow API integration tests") +} + +fn session_api_key() -> Result { + env::var("SLACKBOT_API_KEY") + .context("SLACKBOT_API_KEY is required for session API integration tests") +} + +fn control_api_key() -> Result { + env::var("CENTAUR_CONTROL_API_KEY") + .context("CENTAUR_CONTROL_API_KEY is required for control API integration tests") +} + +fn feedback_api_key() -> Result { + env::var("SLACK_FEEDBACK_API_KEY") + .context("SLACK_FEEDBACK_API_KEY is required for feedback API integration tests") +} + #[tokio::main] async fn main() -> Result<()> { let base_url = env::var("CENTAUR_API_URL") @@ -75,6 +95,16 @@ async fn main() -> Result<()> { test_metrics(&http, &base_url).await, ); + // The authorized drain is intentionally last: it irreversibly fences new + // executions for the lifetime of this control-plane process. + let line = line!() + 1; + record_result( + &mut results, + "Control authorization drains sandboxes after other API checks", + line, + test_control_drain(&http, &base_url).await, + ); + write_report(&results)?; if results.iter().all(|result| result.passed) { @@ -249,6 +279,7 @@ async fn test_harness_wire_values(http: &HttpClient, base_url: &str) -> Result<( "harness_wire_value": wire_value, }, }), + Some(&session_api_key()?), ) .await .with_context(|| format!("create {wire_value} session"))?; @@ -273,6 +304,7 @@ async fn test_harness_wire_values(http: &HttpClient, base_url: &str) -> Result<( let invalid_thread_key = test_thread_key("invalid-harness")?; let invalid_response = http .post(session_url(base_url, &invalid_thread_key)) + .bearer_auth(session_api_key()?) .json(&json!({ "harness_type": "claude-code", "metadata": {"source": "centaur-api-integration-test"}, @@ -290,6 +322,46 @@ async fn test_harness_wire_values(http: &HttpClient, base_url: &str) -> Result<( } async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { + let feedback_thread = format!( + "feedback-improvement:api-integration:{}", + Uuid::new_v4().simple() + ); + let anonymous_feedback = http + .post(session_url(base_url, &feedback_thread)) + .header("X-Centaur-Feedback-Key", feedback_api_key()?) + .json(&json!({"harness_type": "codex"})) + .send() + .await + .context("request feedback session without principal JWT")?; + if anonymous_feedback.status() != StatusCode::UNAUTHORIZED { + let status = anonymous_feedback.status(); + let body = anonymous_feedback.text().await.unwrap_or_default(); + bail!("feedback session without principal JWT returned {status}, expected 401: {body}"); + } + + let anonymous_drain = http + .post(format!("{base_url}/api/sandboxes/drain")) + .send() + .await + .context("request sandbox drain without authorization")?; + if anonymous_drain.status() != StatusCode::UNAUTHORIZED { + let status = anonymous_drain.status(); + let body = anonymous_drain.text().await.unwrap_or_default(); + bail!("anonymous sandbox drain returned {status}, expected 401: {body}"); + } + + let bot_drain = http + .post(format!("{base_url}/api/sandboxes/drain")) + .bearer_auth(session_api_key()?) + .send() + .await + .context("request sandbox drain with bot authorization")?; + if bot_drain.status() != StatusCode::UNAUTHORIZED { + let status = bot_drain.status(); + let body = bot_drain.text().await.unwrap_or_default(); + bail!("bot-authorized sandbox drain returned {status}, expected 401: {body}"); + } + let thread_key = test_thread_key("turn")?; let harness_wire_value = serde_json::to_value(HarnessType::Codex) .context("serialize executable harness type")? @@ -307,10 +379,23 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { }, "on_harness_conflict": "restart", }), + Some(&session_api_key()?), ) .await .context("create executable session")?; + let anonymous_release = http + .post(format!("{}/release", session_url(base_url, &thread_key))) + .json(&json!({"release_id": "anonymous", "cancel_inflight": false})) + .send() + .await + .context("request session release without authorization")?; + if anonymous_release.status() != StatusCode::UNAUTHORIZED { + let status = anonymous_release.status(); + let body = anonymous_release.text().await.unwrap_or_default(); + bail!("anonymous session release returned {status}, expected 401: {body}"); + } + let append = post_json_ok( http, format!("{}/messages", session_url(base_url, &thread_key)), @@ -330,6 +415,7 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { }, ], }), + Some(&session_api_key()?), ) .await .context("append user message")?; @@ -372,6 +458,7 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { "idle_timeout_ms": 5_000, "max_duration_ms": 15_000, }), + Some(&session_api_key()?), ) .await .context("execute session")?; @@ -397,6 +484,7 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { "idle_timeout_ms": 5_000, "max_duration_ms": 15_000, }), + Some(&session_api_key()?), ) .await .context("replay idempotent execute")?; @@ -416,6 +504,7 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { "{}/events?after_event_id=0", session_url(base_url, &thread_key) )) + .bearer_auth(session_api_key()?) .send() .await .context("open session event stream")?; @@ -469,6 +558,24 @@ async fn test_session_turn(http: &HttpClient, base_url: &str) -> Result<()> { Ok(()) } +async fn test_control_drain(http: &HttpClient, base_url: &str) -> Result<()> { + let response = http + .post(format!("{base_url}/api/sandboxes/drain")) + .bearer_auth(control_api_key()?) + .send() + .await + .context("request sandbox drain with control authorization")?; + let status = response.status(); + let body = response + .json::() + .await + .context("parse control-authorized sandbox drain response")?; + if status != StatusCode::OK || body.get("ok").and_then(Value::as_bool) != Some(true) { + bail!("control-authorized sandbox drain returned {status}: {body}"); + } + Ok(()) +} + async fn test_metrics(http: &HttpClient, base_url: &str) -> Result<()> { let response = http .get(format!("{base_url}/metrics")) @@ -492,6 +599,46 @@ async fn test_metrics(http: &HttpClient, base_url: &str) -> Result<()> { } async fn test_workflows_api(http: &HttpClient, base_url: &str) -> Result<()> { + let anonymous_malformed_workflow = http + .post(format!("{base_url}/api/workflows/runs")) + .header("content-type", "application/json") + .body("{not-json") + .send() + .await + .context("request malformed workflow without authorization")?; + if anonymous_malformed_workflow.status() != StatusCode::UNAUTHORIZED { + let status = anonymous_malformed_workflow.status(); + let body = anonymous_malformed_workflow + .text() + .await + .unwrap_or_default(); + bail!("anonymous malformed workflow returned {status}, expected 401: {body}"); + } + + let anonymous_admin_batch = http + .post(format!("{base_url}/api/admin/slack/dm-sync/batch")) + .header("content-type", "application/json") + .body(format!("{{{}", "x".repeat(1024 * 1024))) + .send() + .await + .context("request oversized malformed admin batch without authorization")?; + if anonymous_admin_batch.status() != StatusCode::UNAUTHORIZED { + let status = anonymous_admin_batch.status(); + let body = anonymous_admin_batch.text().await.unwrap_or_default(); + bail!("anonymous malformed admin batch returned {status}, expected 401: {body}"); + } + + let anonymous = http + .get(format!("{base_url}/api/workflows/schedules")) + .send() + .await + .context("request workflow schedules without authorization")?; + if anonymous.status() != StatusCode::UNAUTHORIZED { + let status = anonymous.status(); + let body = anonymous.text().await.unwrap_or_default(); + bail!("anonymous workflow schedules request returned {status}, expected 401: {body}"); + } + let workflow_dir = integration_workflow_dir()?; fs::create_dir_all(&workflow_dir) .with_context(|| format!("create workflow dir {}", workflow_dir.display()))?; @@ -515,6 +662,7 @@ async fn test_workflows_api(http: &HttpClient, base_url: &str) -> Result<()> { json!({ "case": "added-workflow-run", "sleep_ms": 0, + "thread_key": "api-integration-test:workflow-filter", }), ) .await @@ -533,6 +681,33 @@ async fn test_workflows_api(http: &HttpClient, base_url: &str) -> Result<()> { bail!("completed workflow output did not echo input: {completed_run}"); } + let filtered = http + .get(format!( + "{base_url}/api/workflows/runs?workflow_name={workflow_name}&thread_key=api-integration-test%3Aworkflow-filter" + )) + .bearer_auth(workflow_api_key()?) + .send() + .await + .context("list workflow runs with resource filters")?; + if !filtered.status().is_success() { + let status = filtered.status(); + let body = filtered.text().await.unwrap_or_default(); + bail!("filtered workflow run list returned {status}: {body}"); + } + let filtered = filtered + .json::() + .await + .context("parse filtered workflow run list")?; + let filtered_runs = filtered + .get("runs") + .and_then(Value::as_array) + .context("filtered workflow run list missing runs")?; + if filtered_runs.len() != 1 + || filtered_runs[0].get("run_id").and_then(Value::as_str) != Some(completed_run_id.as_str()) + { + bail!("workflow run filters returned unexpected rows: {filtered}"); + } + let removed_run_id = create_workflow_run( http, base_url, @@ -631,6 +806,7 @@ async fn create_workflow_run( "harness_type": HarnessType::Codex, "max_attempts": 1, }), + Some(&workflow_api_key()?), ) .await?; if response.get("ok").and_then(Value::as_bool) != Some(true) { @@ -656,7 +832,12 @@ async fn wait_for_workflow_run_status( let mut last_run = Value::Null; while Instant::now() < deadline { - let body = get_json_ok(http, format!("{base_url}/api/workflows/runs/{run_id}")).await?; + let body = get_json_ok( + http, + format!("{base_url}/api/workflows/runs/{run_id}"), + Some(&workflow_api_key()?), + ) + .await?; let run = body .get("run") .cloned() @@ -694,7 +875,12 @@ async fn wait_for_workflow_schedule( let mut last_body = Value::Null; while Instant::now() < deadline { - let body = get_json_ok(http, format!("{base_url}/api/workflows/schedules")).await?; + let body = get_json_ok( + http, + format!("{base_url}/api/workflows/schedules"), + Some(&workflow_api_key()?), + ) + .await?; let present = body .get("schedules") .and_then(Value::as_array) @@ -718,9 +904,16 @@ fn parse_json(data: &str) -> Result { serde_json::from_str(data).with_context(|| format!("parse event payload as JSON: {data}")) } -async fn get_json_ok(http: &HttpClient, url: impl AsRef) -> Result { - let response = http - .get(url.as_ref()) +async fn get_json_ok( + http: &HttpClient, + url: impl AsRef, + bearer_token: Option<&str>, +) -> Result { + let mut request = http.get(url.as_ref()); + if let Some(token) = bearer_token { + request = request.bearer_auth(token); + } + let response = request .send() .await .with_context(|| format!("GET {}", url.as_ref()))?; @@ -735,10 +928,17 @@ async fn get_json_ok(http: &HttpClient, url: impl AsRef) -> Result { .with_context(|| format!("parse GET {} response", url.as_ref())) } -async fn post_json_ok(http: &HttpClient, url: impl AsRef, body: Value) -> Result { - let response = http - .post(url.as_ref()) - .json(&body) +async fn post_json_ok( + http: &HttpClient, + url: impl AsRef, + body: Value, + bearer_token: Option<&str>, +) -> Result { + let mut request = http.post(url.as_ref()).json(&body); + if let Some(token) = bearer_token { + request = request.bearer_auth(token); + } + let response = request .send() .await .with_context(|| format!("POST {}", url.as_ref()))?; diff --git a/services/api-rs/crates/centaur-api-server/Cargo.toml b/services/api-rs/crates/centaur-api-server/Cargo.toml index 82d015e98..62b07a426 100644 --- a/services/api-rs/crates/centaur-api-server/Cargo.toml +++ b/services/api-rs/crates/centaur-api-server/Cargo.toml @@ -27,6 +27,7 @@ eventsource-stream.workspace = true futures-util.workspace = true hmac.workspace = true hex.workspace = true +jsonwebtoken.workspace = true kube.workspace = true reqwest.workspace = true rustls.workspace = true diff --git a/services/api-rs/crates/centaur-api-server/src/activity_summary.rs b/services/api-rs/crates/centaur-api-server/src/activity_summary.rs new file mode 100644 index 000000000..6f63dff11 --- /dev/null +++ b/services/api-rs/crates/centaur-api-server/src/activity_summary.rs @@ -0,0 +1,1176 @@ +use std::{ + collections::{HashMap, VecDeque}, + time::{Duration, Instant}, +}; + +use centaur_session_core::{MessageRole, SessionEvent, ThreadKey, ThreadKeyError}; +use centaur_session_runtime::SESSION_OUTPUT_LINE_EVENT; +use centaur_session_sqlx::{PgSessionStore, SessionEventNotification, SessionStoreError}; +use reqwest::StatusCode; +use serde_json::{Value, json}; +use thiserror::Error; +use tokio::time::sleep; +use tracing::{debug, info, warn}; + +pub(crate) const SESSION_ACTIVITY_SUMMARY_EVENT: &str = "session.activity_summary"; + +const SYSTEM_PROMPT: &str = "\ +You write live status text for a software agent. Use only the supplied event facts. \ +Write one first-person present-tense sentence of at most 40 characters, including \ +spaces, as if you are the agent. The hard limit is 45 characters: anything longer is \ +thrown away, so when in doubt cut words and use the shortest name for things. \ +Describe the current step or latest finding, not the overall session goal: say what \ +you are doing or learned right now, like \"I'm computing TPS from blocks\", \ +\"I found the chain config\", or \"I'm blocked on metrics access\". Take the newest \ +facts labeled commentary, plan, or tool as the current step; earlier facts are only \ +context. Name one specific thing from the facts (a chain, PR, partner, tool, or \ +topic); avoid generic words like details, info, items, update, or summary, and avoid \ +repeating the session goal word for word. Each status must say something new \ +compared to the previous status sentence; if you cannot, output exactly SKIP. If the \ +facts only show setup, help output, dependency installs, builds, command output, \ +logs, tests, or other mechanics, output exactly SKIP. Do not mention commands, \ +paths, IDs, or flags. Do not refer to \"the agent\". No markdown, no quotes, no \ +event IDs, and no speculation."; + +#[derive(Clone)] +pub(crate) struct ActivitySummaryConfig { + pub(crate) base_url: String, + pub(crate) api_key: String, + pub(crate) max_facts: usize, + pub(crate) max_output_tokens: u16, + pub(crate) min_interval: Duration, + pub(crate) model: String, + pub(crate) timeout: Duration, +} + +pub(crate) struct ActivitySummaryWorker { + client: ActivitySummaryClient, + config: ActivitySummaryConfig, + states: HashMap, + store: PgSessionStore, +} + +impl ActivitySummaryWorker { + pub(crate) fn new( + store: PgSessionStore, + config: ActivitySummaryConfig, + ) -> Result { + Ok(Self { + client: ActivitySummaryClient::new(&config)?, + config, + states: HashMap::new(), + store, + }) + } + + pub(crate) async fn run(mut self) { + info!( + model = %self.config.model, + min_interval_ms = self.config.min_interval.as_millis(), + "session activity summary worker started" + ); + loop { + let mut listener = match self.store.listen_session_events().await { + Ok(listener) => listener, + Err(error) => { + warn!(%error, "failed to listen for session activity events"); + sleep(Duration::from_secs(5)).await; + continue; + } + }; + + loop { + match listener.recv().await { + Ok(notification) => { + if let Err(error) = self.process_notification(notification).await { + warn!(%error, "failed to process session activity event"); + } + } + Err(error) => { + warn!(%error, "session activity event listener failed; reconnecting"); + sleep(Duration::from_secs(1)).await; + break; + } + } + } + } + } + + async fn process_notification( + &mut self, + notification: SessionEventNotification, + ) -> Result<(), ActivitySummaryError> { + let thread_key = ThreadKey::parse(notification.thread_key)?; + let events = self + .store + .list_events_after( + &thread_key, + notification.event_id.saturating_sub(1), + None, + 8, + ) + .await?; + let Some(event) = events + .into_iter() + .find(|event| event.event_id == notification.event_id) + else { + return Ok(()); + }; + self.process_event(event).await + } + + async fn process_event(&mut self, event: SessionEvent) -> Result<(), ActivitySummaryError> { + if event.event_type == SESSION_ACTIVITY_SUMMARY_EVENT { + return Ok(()); + } + let Some(execution_id) = event.execution_id.as_deref() else { + return Ok(()); + }; + if is_terminal_session_event(&event.event_type) { + self.states.remove(execution_id); + return Ok(()); + } + if event.event_type != SESSION_OUTPUT_LINE_EVENT { + return Ok(()); + } + + let Some(fact) = activity_fact_from_output_event(&event) else { + return Ok(()); + }; + let goal = if self.states.contains_key(execution_id) { + None + } else { + self.activity_goal_context(&event.thread_key).await? + }; + let now = Instant::now(); + let publish = { + let state = self + .states + .entry(execution_id.to_owned()) + .or_insert_with(|| ExecutionActivity::new(self.config.max_facts, goal)); + state.push(fact); + state.prepare_publish(now, self.config.min_interval) + }; + + let Some(prompt) = publish else { + return Ok(()); + }; + + let summary = match self.client.summarize(&prompt).await { + Ok(summary) => summary, + Err(error) => { + warn!(%error, "failed to generate session activity summary"); + return Ok(()); + } + }; + let Some(summary) = sanitize_summary(&summary) else { + debug!("discarded empty session activity summary"); + return Ok(()); + }; + if self + .states + .get(execution_id) + .and_then(|state| state.last_summary.as_deref()) + .is_some_and(|last| summaries_are_similar(last, &summary)) + { + debug!(summary, "discarded redundant session activity summary"); + return Ok(()); + } + + self.store + .append_event( + &event.thread_key, + Some(execution_id), + SESSION_ACTIVITY_SUMMARY_EVENT, + json!({ + "execution_id": execution_id, + "model": self.config.model.as_str(), + "source_event_id": event.event_id, + "summary": summary, + }), + ) + .await?; + + if let Some(state) = self.states.get_mut(execution_id) { + state.last_published_signature = Some(state.signature()); + state.last_summary = Some(summary); + } + Ok(()) + } + + async fn activity_goal_context( + &self, + thread_key: &ThreadKey, + ) -> Result, ActivitySummaryError> { + if let Some(title) = self.store.get_session_title(thread_key).await? + && let Some(title) = clean_goal_text(&title) + { + return Ok(Some(title)); + } + + let messages = self.store.list_messages(thread_key).await?; + let goal = messages + .iter() + .find(|message| message.role == MessageRole::User) + .and_then(|message| message_parts_text(&message.parts)); + Ok(goal.and_then(|goal| clean_goal_text(&goal))) + } +} + +#[derive(Debug)] +struct ExecutionActivity { + facts: VecDeque, + goal: Option, + last_attempt_at: Option, + last_published_signature: Option, + last_summary: Option, + max_facts: usize, +} + +impl ExecutionActivity { + fn new(max_facts: usize, goal: Option) -> Self { + Self { + facts: VecDeque::with_capacity(max_facts), + goal, + last_attempt_at: None, + last_published_signature: None, + last_summary: None, + max_facts, + } + } + + fn push(&mut self, fact: ActivityFact) { + if !fact.is_publishable() { + return; + } + if self + .facts + .iter() + .any(|existing| existing.kind == fact.kind && existing.text == fact.text) + { + return; + } + self.facts.push_back(fact); + while self.facts.len() > self.max_facts { + self.facts.pop_front(); + } + } + + fn prepare_publish(&mut self, now: Instant, min_interval: Duration) -> Option { + if self.facts.is_empty() { + return None; + } + if !self.facts.iter().any(ActivityFact::is_publishable) { + return None; + } + if self + .last_attempt_at + .is_some_and(|last| now.saturating_duration_since(last) < min_interval) + { + return None; + } + let signature = self.signature(); + if self + .last_published_signature + .as_ref() + .is_some_and(|last| last == &signature) + { + return None; + } + self.last_attempt_at = Some(now); + Some(self.prompt()) + } + + fn prompt(&self) -> String { + let mut lines = Vec::new(); + if let Some(summary) = self.last_summary.as_deref() { + lines.push(format!("Previous status sentence: {summary}")); + } + if let Some(goal) = self.goal.as_deref() { + lines.push(format!("Session goal: {goal}")); + } + lines.push("Recent activity facts, oldest to newest:".to_owned()); + for fact in self.facts.iter().filter(|fact| fact.is_publishable()) { + lines.push(format!("- {}: {}", fact.kind, fact.text)); + } + lines.join("\n") + } + + fn signature(&self) -> String { + let goal = self.goal.as_deref().unwrap_or_default(); + std::iter::once(format!("goal={goal}")) + .chain( + self.facts + .iter() + .filter(|fact| fact.is_publishable()) + .map(|fact| format!("{}={}", fact.kind, fact.text)), + ) + .collect::>() + .join("\n") + } +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum ActivitySignal { + High, + Low, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +struct ActivityFact { + kind: &'static str, + signal: ActivitySignal, + text: String, +} + +impl ActivityFact { + fn high(kind: &'static str, text: impl Into) -> Self { + Self { + kind, + signal: ActivitySignal::High, + text: text.into(), + } + } + + fn low(kind: &'static str, text: impl Into) -> Self { + Self { + kind, + signal: ActivitySignal::Low, + text: text.into(), + } + } + + fn is_publishable(&self) -> bool { + self.signal == ActivitySignal::High + } +} + +fn message_parts_text(parts: &[Value]) -> Option { + let text = parts + .iter() + .filter_map(message_part_text) + .collect::>() + .join(" "); + (!text.trim().is_empty()).then_some(text) +} + +fn message_part_text(part: &Value) -> Option { + if let Some(text) = part.as_str() { + return Some(text.trim().to_owned()).filter(|text| !text.is_empty()); + } + string_at(part, &["text"]) + .or_else(|| string_at(part, &["content"])) + .or_else(|| string_at(part, &["title"])) +} + +fn clean_goal_text(value: &str) -> Option { + let text = one_line(value, 160); + let lower = text.to_ascii_lowercase(); + if lower.is_empty() + || matches!( + lower.as_str(), + "continue" | "go on" | "ok" | "okay" | "yes" | "yep" | "sure" + ) + { + return None; + } + Some(text) +} + +fn activity_fact_from_output_event(event: &SessionEvent) -> Option { + let line = event.payload.as_str()?; + let value = serde_json::from_str::(line).ok()?; + activity_fact_from_value(&value) +} + +fn activity_fact_from_value(value: &Value) -> Option { + let event_type = event_type(value)?; + let normalized = event_type.replace('/', "."); + match normalized.as_str() { + "turn.plan.updated" => plan_fact(value), + "item.plan.delta" => string_field(value, &["delta", "text"]) + .map(|text| ActivityFact::high("plan", format!("planning {}", one_line(&text, 180)))), + "item.reasoning.summaryTextDelta" | "item.reasoning.textDelta" => { + string_field(value, &["delta", "text"]) + .map(|text| ActivityFact::high("thinking", one_line(&text, 220))) + } + "item.commandExecution.outputDelta" => None, + "item.mcpToolCall.progress" => Some(ActivityFact::high("tool", progress_fact_text(value))), + "item.started" | "item.updated" | "item.completed" => item_fact(value, &normalized), + "assistant" => assistant_tool_fact(value), + "tool" | "user" => tool_result_fact(value), + _ => None, + } +} + +fn event_type(value: &Value) -> Option { + string_at(value, &["method"]).or_else(|| string_at(value, &["type"])) +} + +fn plan_fact(value: &Value) -> Option { + let plan = value + .get("plan") + .or_else(|| value.get("params").and_then(|params| params.get("plan")))?; + let items = plan.as_array()?; + let current = items + .iter() + .find(|item| { + let status = string_at(item, &["status"]) + .unwrap_or_default() + .to_ascii_lowercase(); + matches!( + status.as_str(), + "inprogress" | "in_progress" | "running" | "pending" | "" + ) + }) + .or_else(|| items.last())?; + let step = string_at(current, &["step"]) + .or_else(|| string_at(current, &["title"])) + .or_else(|| string_at(current, &["text"]))?; + Some(ActivityFact::high( + "plan", + format!("working on {}", one_line(&strip_plan_marker(&step), 180)), + )) +} + +fn item_fact(value: &Value, normalized_event_type: &str) -> Option { + let item = protocol_item(value)?; + let item_type = string_at(item, &["type"]).unwrap_or_default(); + let completed = normalized_event_type == "item.completed"; + match item_type.as_str() { + "commandExecution" | "command_execution" => { + let command = string_at(item, &["command"]).unwrap_or_else(|| "command".to_owned()); + command_fact(&command, completed) + } + "fileChange" | "file_change" => Some(ActivityFact::high( + "files", + file_change_text(item, completed), + )), + "reasoning" => reasoning_item_fact(item, completed), + "mcpToolCall" | "mcp_tool_call" | "dynamicToolCall" | "dynamic_tool_call" => { + let name = tool_name(item); + let action = if completed { "finished using" } else { "using" }; + Some(ActivityFact::high("tool", format!("{action} {name}"))) + } + "agentMessage" | "agent_message" => agent_message_fact(item, completed), + "plan" => string_at(item, &["text"]).map(|text| { + ActivityFact::high("plan", format!("updated plan {}", one_line(&text, 180))) + }), + _ => None, + } +} + +fn command_fact(command: &str, completed: bool) -> Option { + let command = unwrap_shell_command(command); + if is_low_signal_command(&command) { + return Some(ActivityFact::low( + "command", + low_signal_command_label(&command), + )); + } + let tool = command_tool_name(&command)?; + let action = if completed { "finished using" } else { "using" }; + Some(ActivityFact::high("tool", format!("{action} {tool}"))) +} + +fn agent_message_fact(item: &Value, completed: bool) -> Option { + if !completed { + return None; + } + let phase = string_at(item, &["phase"]).unwrap_or_default(); + if phase != "commentary" { + return None; + } + let text = string_at(item, &["text"])?; + if is_low_signal_commentary(&text) { + return None; + } + Some(ActivityFact::high("commentary", one_line(&text, 220))) +} + +fn protocol_item(value: &Value) -> Option<&Value> { + value + .get("item") + .or_else(|| value.get("params").and_then(|params| params.get("item"))) +} + +fn reasoning_item_fact(item: &Value, completed: bool) -> Option { + let text = string_at(item, &["text"]) + .or_else(|| array_text(item.get("summary"))) + .or_else(|| array_text(item.get("content")))?; + Some(ActivityFact::high( + "thinking", + if completed { + format!("finished thinking about {}", one_line(&text, 180)) + } else { + one_line(&text, 220) + }, + )) +} + +fn file_change_text(item: &Value, completed: bool) -> String { + let action = if completed { + "finished editing" + } else { + "editing" + }; + let paths = item + .get("changes") + .and_then(Value::as_array) + .map(|changes| { + changes + .iter() + .filter_map(|change| string_at(change, &["path"])) + .collect::>() + }) + .unwrap_or_default(); + if paths.is_empty() { + return format!("{action} files"); + } + let unique = paths + .into_iter() + .fold(Vec::::new(), |mut out, path| { + if !out.contains(&path) { + out.push(path); + } + out + }); + format!("{action} {}", one_line(&unique.join(", "), 180)) +} + +fn progress_fact_text(value: &Value) -> String { + let name = string_at(value, &["name"]) + .or_else(|| string_at(value, &["toolName"])) + .or_else(|| string_at(value, &["params", "name"])) + .or_else(|| string_at(value, &["params", "toolName"])) + .unwrap_or_else(|| "tool".to_owned()); + format!("waiting on {name}") +} + +fn assistant_tool_fact(value: &Value) -> Option { + let content = value.get("content").and_then(Value::as_array)?; + let tool = content + .iter() + .find(|item| string_at(item, &["type"]).as_deref() == Some("tool_use"))?; + Some(ActivityFact::high( + "tool", + format!("using {}", tool_name(tool)), + )) +} + +fn tool_result_fact(value: &Value) -> Option { + let content = value.get("content").and_then(Value::as_array)?; + if content.iter().any(|item| { + string_at(item, &["type"]).as_deref() == Some("tool_result") + || string_at(item, &["tool_use_id"]).is_some() + }) { + return Some(ActivityFact::low("tool", "reading tool results")); + } + None +} + +fn tool_name(item: &Value) -> String { + string_at(item, &["name"]) + .or_else(|| string_at(item, &["toolName"])) + .or_else(|| string_at(item, &["tool_name"])) + .or_else(|| string_at(item, &["serverLabel"])) + .or_else(|| string_at(item, &["server_label"])) + .unwrap_or_else(|| "tool".to_owned()) +} + +fn command_tool_name(command: &str) -> Option { + let first = command + .split_whitespace() + .next()? + .trim_matches(|ch| ch == '"' || ch == '\''); + let name = first.rsplit('/').next().unwrap_or(first); + if name.is_empty() || is_shell_or_package_command(name) { + return None; + } + Some(name.to_owned()) +} + +fn is_low_signal_command(command: &str) -> bool { + let lower = command.to_ascii_lowercase(); + let first = lower.split_whitespace().next().unwrap_or_default(); + lower.is_empty() + || lower == "command" + || lower.contains(" --help") + || lower.ends_with(" --help") + || lower.contains(" -h") + || lower.contains("centaur-tools list") + || lower.contains("centaur-tools refresh") + || lower.contains("uv sync") + || lower.contains("uv pip install") + || lower.contains("pip install") + || lower.contains("pnpm install") + || lower.contains("npm install") + || lower.contains("cargo build") + || lower.contains("cargo check") + || lower.contains("cargo test") + || lower.contains("cargo fmt") + || lower.contains("ruff ") + || lower.contains("pytest") + || lower.contains("helm template") + || lower.contains("helm lint") + || matches!( + first, + "rg" | "grep" + | "sed" + | "awk" + | "cat" + | "ls" + | "find" + | "git" + | "kubectl" + | "jq" + | "curl" + | "python" + | "python3" + | "node" + | "sh" + | "bash" + ) +} + +fn low_signal_command_label(command: &str) -> String { + let lower = command.to_ascii_lowercase(); + if lower.contains(" --help") || lower.ends_with(" --help") || lower.contains(" -h") { + "checking tool help".to_owned() + } else if lower.contains("install") || lower.contains("build") { + "setup work".to_owned() + } else { + "mechanical command".to_owned() + } +} + +fn is_shell_or_package_command(name: &str) -> bool { + matches!( + name, + "bash" + | "sh" + | "zsh" + | "python" + | "python3" + | "node" + | "bun" + | "uv" + | "pip" + | "pnpm" + | "npm" + | "cargo" + | "git" + | "kubectl" + | "rg" + | "grep" + | "sed" + | "awk" + | "cat" + | "ls" + | "find" + | "jq" + | "curl" + ) +} + +fn is_low_signal_commentary(text: &str) -> bool { + let lower = text.trim().to_ascii_lowercase(); + lower.is_empty() + || lower == "i'll take a look." + || lower == "i\u{2019}ll take a look." + || lower == "i'll check." + || lower == "i\u{2019}ll check." + || lower == "i'm working on it." + || lower == "i\u{2019}m working on it." +} + +fn array_text(value: Option<&Value>) -> Option { + let texts = value? + .as_array()? + .iter() + .filter_map(|item| { + if let Some(text) = item.as_str() { + return Some(text.to_owned()); + } + string_at(item, &["text"]) + }) + .filter(|text| !text.trim().is_empty()) + .collect::>(); + (!texts.is_empty()).then(|| texts.join(" ")) +} + +fn string_field(value: &Value, keys: &[&str]) -> Option { + keys.iter().find_map(|key| string_at(value, &[*key])) +} + +fn string_at(value: &Value, path: &[&str]) -> Option { + let mut current = value; + for key in path { + current = current.get(*key)?; + } + current + .as_str() + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(ToOwned::to_owned) +} + +fn strip_plan_marker(value: &str) -> String { + let mut text = value.trim(); + if let Some(rest) = text.strip_prefix("- ") { + text = rest; + } else if let Some(rest) = text.strip_prefix("* ") { + text = rest; + } + for marker in ["[ ] ", "[x] ", "[X] "] { + if let Some(rest) = text.strip_prefix(marker) { + text = rest; + } + } + text.trim().to_owned() +} + +fn unwrap_shell_command(command: &str) -> String { + let trimmed = command.trim(); + let Some(rest) = trimmed.strip_prefix("/bin/bash -lc ") else { + return trimmed.to_owned(); + }; + rest.trim() + .trim_matches(|ch| ch == '"' || ch == '\'') + .trim() + .to_owned() +} + +fn one_line(value: &str, max_chars: usize) -> String { + let normalized = value.split_whitespace().collect::>().join(" "); + if normalized.chars().count() <= max_chars { + return normalized; + } + let mut out = normalized + .chars() + .take(max_chars.saturating_sub(3)) + .collect::(); + out.push_str("..."); + out +} + +fn sanitize_summary(summary: &str) -> Option { + let summary = summary + .trim() + .trim_matches('"') + .trim_matches('\'') + .trim() + .trim_end_matches('.') + .to_owned(); + if summary.eq_ignore_ascii_case("skip") || summary.chars().count() > 45 { + return None; + } + if is_generic_summary(&summary) { + return None; + } + (!summary.is_empty()).then_some(summary) +} + +fn is_generic_summary(summary: &str) -> bool { + let normalized = normalize_summary(summary); + normalized.is_empty() + || normalized.contains("gathering details") + || normalized.contains("gathering info") + || (normalized.contains("gathering") && normalized.contains("info")) + || normalized.contains("listing available") + || normalized.contains("available items") + || normalized.contains("preparing your update") + || normalized.contains("preparing your summary") + || (normalized.contains("preparing your") && normalized.contains("summary")) + || normalized.contains("checking the request") + || normalized.contains("working on it") + || normalized.contains("making progress") + || normalized.contains("handling the task") +} + +fn summaries_are_similar(previous: &str, candidate: &str) -> bool { + let previous = summary_keywords(previous); + let candidate = summary_keywords(candidate); + if previous.is_empty() || candidate.is_empty() { + return false; + } + let shared = candidate + .iter() + .filter(|word| previous.contains(*word)) + .count(); + let smaller = previous.len().min(candidate.len()); + shared * 4 >= smaller * 3 +} + +fn summary_keywords(summary: &str) -> Vec { + normalize_summary(summary) + .split_whitespace() + .filter(|word| { + !matches!( + *word, + "i" | "m" + | "im" + | "i'm" + | "am" + | "the" + | "a" + | "an" + | "for" + | "to" + | "on" + | "your" + | "my" + | "this" + | "that" + ) + }) + .map(ToOwned::to_owned) + .collect() +} + +fn normalize_summary(summary: &str) -> String { + summary + .to_ascii_lowercase() + .chars() + .map(|ch| if ch.is_ascii_alphanumeric() { ch } else { ' ' }) + .collect::() + .split_whitespace() + .collect::>() + .join(" ") +} + +fn is_terminal_session_event(event_type: &str) -> bool { + matches!( + event_type, + "session.execution_completed" + | "session.execution_failed" + | "session.execution_cancelled" + | "session.stream_error" + | "session.stdout_pump_failed" + ) +} + +#[derive(Clone)] +struct ActivitySummaryClient { + api_key: String, + client: reqwest::Client, + max_output_tokens: u16, + model: String, + responses_url: String, +} + +impl ActivitySummaryClient { + fn new(config: &ActivitySummaryConfig) -> Result { + let client = reqwest::Client::builder() + .timeout(config.timeout) + .build() + .map_err(ActivitySummaryError::Http)?; + let responses_url = format!("{}/responses", config.base_url.trim_end_matches('/')); + Ok(Self { + api_key: config.api_key.clone(), + client, + max_output_tokens: config.max_output_tokens, + model: config.model.clone(), + responses_url, + }) + } + + async fn summarize(&self, prompt: &str) -> Result { + let response = self + .client + .post(&self.responses_url) + .bearer_auth(&self.api_key) + .json(&json!({ + "model": self.model.as_str(), + "instructions": SYSTEM_PROMPT, + "input": prompt, + "max_output_tokens": self.max_output_tokens, + "store": false, + })) + .send() + .await?; + let status = response.status(); + let body = response.text().await?; + if !status.is_success() { + return Err(ActivitySummaryError::OpenAiStatus { + body: redact_openai_error_body(&body), + status, + }); + } + let value = serde_json::from_str::(&body)?; + if let Some(reason) = string_at(&value, &["incomplete_details", "reason"]) { + return Err(ActivitySummaryError::Incomplete { reason }); + } + extract_response_text(&value).ok_or(ActivitySummaryError::MissingOutputText) + } +} + +fn extract_response_text(value: &Value) -> Option { + if let Some(text) = string_at(value, &["output_text"]) { + return Some(text); + } + let output = value.get("output")?.as_array()?; + let mut parts = Vec::new(); + for item in output { + let Some(content) = item.get("content").and_then(Value::as_array) else { + continue; + }; + for content_item in content { + if let Some(text) = string_at(content_item, &["text"]) { + parts.push(text); + } + } + } + (!parts.is_empty()).then(|| parts.join(" ")) +} + +fn redact_openai_error_body(body: &str) -> String { + let body = one_line(body, 300); + let marker = "Incorrect API key provided:"; + let Some(marker_index) = body.find(marker) else { + return body; + }; + let value_start = marker_index + marker.len(); + let value_end = body[value_start..] + .find('.') + .map(|offset| value_start + offset) + .unwrap_or(body.len()); + format!( + "{} [redacted]{}", + body[..value_start].trim_end(), + &body[value_end..] + ) +} + +#[derive(Debug, Error)] +pub(crate) enum ActivitySummaryError { + #[error("activity summary HTTP error: {0}")] + Http(#[from] reqwest::Error), + #[error("activity summary OpenAI request failed with {status}: {body}")] + OpenAiStatus { status: StatusCode, body: String }, + #[error("activity summary OpenAI response incomplete: {reason}")] + Incomplete { reason: String }, + #[error("activity summary OpenAI response did not include output text")] + MissingOutputText, + #[error("activity summary JSON error: {0}")] + Json(#[from] serde_json::Error), + #[error("activity summary session store error: {0}")] + Store(#[from] SessionStoreError), + #[error("activity summary thread key error: {0}")] + ThreadKey(#[from] ThreadKeyError), +} + +#[cfg(test)] +mod tests { + use centaur_session_core::ThreadKey; + use time::OffsetDateTime; + + use super::*; + + fn event(line: Value) -> SessionEvent { + SessionEvent { + event_id: 7, + thread_key: ThreadKey::parse("test:thread").unwrap(), + execution_id: Some("exec-1".to_owned()), + event_type: SESSION_OUTPUT_LINE_EVENT.to_owned(), + payload: Value::String(line.to_string()), + created_at: OffsetDateTime::now_utc(), + } + } + + #[test] + fn projects_plan_update_into_activity_fact() { + let fact = activity_fact_from_output_event(&event(json!({ + "type": "turn.plan.updated", + "plan": [ + {"step": "Inspect App Server events", "status": "completed"}, + {"step": "Add activity summary worker", "status": "in_progress"} + ] + }))) + .unwrap(); + + assert_eq!( + fact, + ActivityFact::high("plan", "working on Add activity summary worker") + ); + } + + #[test] + fn drops_low_signal_command_events() { + let fact = activity_fact_from_output_event(&event(json!({ + "method": "item/started", + "params": { + "item": { + "id": "cmd-1", + "type": "commandExecution", + "command": "/bin/bash -lc 'centaur-tools list'" + } + } + }))) + .unwrap(); + + assert_eq!(fact, ActivityFact::low("command", "mechanical command")); + } + + #[test] + fn projects_tool_command_by_tool_name() { + let fact = activity_fact_from_output_event(&event(json!({ + "method": "item/started", + "params": { + "item": { + "id": "cmd-1", + "type": "commandExecution", + "command": "/bin/bash -lc 'websearch search --query usdG yield'" + } + } + }))) + .unwrap(); + + assert_eq!(fact, ActivityFact::high("tool", "using websearch")); + } + + #[test] + fn captures_completed_agent_commentary_as_activity() { + let fact = activity_fact_from_output_event(&event(json!({ + "method": "item/completed", + "params": { + "item": { + "id": "msg-1", + "phase": "commentary", + "text": "I'll trace the USDG vault yield source.", + "type": "agentMessage" + } + } + }))) + .unwrap(); + + assert_eq!( + fact, + ActivityFact::high("commentary", "I'll trace the USDG vault yield source.") + ); + } + + #[test] + fn system_prompt_requires_conversational_step_status() { + assert!(SYSTEM_PROMPT.contains("first-person")); + assert!(SYSTEM_PROMPT.contains("at most 40 characters")); + assert!(SYSTEM_PROMPT.contains("hard limit is 45 characters")); + assert!(SYSTEM_PROMPT.contains("current step or latest finding")); + assert!(SYSTEM_PROMPT.contains("not the overall session goal")); + assert!(SYSTEM_PROMPT.contains("Name one specific thing")); + assert!(SYSTEM_PROMPT.contains("output exactly SKIP")); + assert!(SYSTEM_PROMPT.contains("Do not mention commands")); + assert!(SYSTEM_PROMPT.contains("Do not refer to \"the agent\"")); + } + + #[test] + fn extracts_output_text_from_responses_body() { + let text = extract_response_text(&json!({ + "output": [ + { + "type": "message", + "content": [ + {"type": "output_text", "text": "I'm inspecting events."} + ] + } + ] + })) + .unwrap(); + + assert_eq!(text, "I'm inspecting events."); + } + + #[test] + fn detects_incomplete_responses_body() { + let reason = string_at( + &json!({ + "status": "incomplete", + "incomplete_details": {"reason": "max_output_tokens"}, + "output": [ + {"type": "reasoning", "content": [], "summary": []} + ] + }), + &["incomplete_details", "reason"], + ) + .unwrap(); + + assert_eq!(reason, "max_output_tokens"); + } + + #[test] + fn redacts_openai_invalid_key_errors() { + let redacted = redact_openai_error_body( + r#"{"error":{"message":"Incorrect API key provided: sk-svc-secret. You can find your API key at https://platform.openai.com/account/api-keys."}}"#, + ); + + assert!(redacted.contains("Incorrect API key provided: [redacted]")); + assert!(!redacted.contains("sk-svc-secret")); + } + + #[test] + fn throttles_unchanged_activity() { + let mut state = ExecutionActivity::new(4, Some("Investigate USDG vault yield".to_owned())); + let now = Instant::now(); + state.push(ActivityFact::high("tool", "using websearch")); + assert!(state.prepare_publish(now, Duration::from_secs(8)).is_some()); + state.last_published_signature = Some(state.signature()); + assert!( + state + .prepare_publish(now + Duration::from_secs(9), Duration::from_secs(8)) + .is_none() + ); + } + + #[test] + fn skips_low_signal_only_activity() { + let mut state = ExecutionActivity::new(4, Some("Investigate USDG vault yield".to_owned())); + let now = Instant::now(); + state.push(ActivityFact::low("command", "checking tool help")); + + assert!(state.prepare_publish(now, Duration::from_secs(8)).is_none()); + } + + #[test] + fn prompt_includes_session_goal() { + let mut state = ExecutionActivity::new(4, Some("Investigate USDG vault yield".to_owned())); + state.push(ActivityFact::high("tool", "using websearch")); + + let prompt = state.prompt(); + + assert!(prompt.contains("Session goal: Investigate USDG vault yield")); + assert!(prompt.contains("- tool: using websearch")); + } + + #[test] + fn sanitizes_useless_summaries() { + assert_eq!(sanitize_summary("SKIP"), None); + assert_eq!( + sanitize_summary("I'm gathering details for the USDG info."), + None + ); + assert_eq!( + sanitize_summary("I'm preparing your USDG vault update summary"), + None + ); + assert_eq!( + sanitize_summary("I'm checking USDG yield sources."), + Some("I'm checking USDG yield sources".to_owned()) + ); + assert_eq!( + sanitize_summary("I'm checking a summary that is far too long for Slack status text"), + None + ); + } + + #[test] + fn detects_redundant_summary_phrasing() { + assert!(summaries_are_similar( + "I'm checking USDG yield sources", + "I'm checking USDG yield source" + )); + assert!(!summaries_are_similar( + "I'm checking USDG yield sources", + "I'm comparing vault contract events" + )); + } +} diff --git a/services/api-rs/crates/centaur-api-server/src/api_jwt.rs b/services/api-rs/crates/centaur-api-server/src/api_jwt.rs new file mode 100644 index 000000000..4e9830deb --- /dev/null +++ b/services/api-rs/crates/centaur-api-server/src/api_jwt.rs @@ -0,0 +1,204 @@ +use std::{env, sync::OnceLock}; + +use axum::http::{HeaderMap, header}; +use base64::{Engine as _, engine::general_purpose}; +use jsonwebtoken::{Algorithm, DecodingKey, Validation, decode}; +use serde::de::DeserializeOwned; +use serde_json::Value; + +use crate::ApiError; + +const DEFAULT_API_JWT_AUDIENCE: &str = "centaur-api"; +const DEFAULT_API_JWT_ISSUER: &str = "centaur-console"; +const JWT_CLOCK_SKEW_SECONDS: i64 = 30; + +pub(crate) fn bearer_token(headers: &HeaderMap) -> Result<&str, ApiError> { + let value = headers + .get(header::AUTHORIZATION) + .and_then(|value| value.to_str().ok()) + .ok_or_else(|| ApiError::Unauthorized("missing bearer token".to_owned()))?; + value + .split_once(' ') + .filter(|(scheme, _)| scheme.eq_ignore_ascii_case("Bearer")) + .map(|(_, token)| token.trim()) + .filter(|token| !token.is_empty()) + .ok_or_else(|| ApiError::Unauthorized("missing bearer token".to_owned())) +} + +pub(crate) fn bearer_jwt_from_headers(headers: &HeaderMap) -> Option<&str> { + let token = bearer_token(headers).ok()?; + if token.matches('.').count() == 2 { + Some(token) + } else { + None + } +} + +pub(crate) fn decode_jwt_payload(token: &str) -> Result { + let mut parts = token.split('.'); + let _header = parts.next(); + let payload = parts + .next() + .ok_or_else(|| "JWT payload is missing".to_owned())?; + if parts.next().is_none() || parts.next().is_some() { + return Err("JWT must have three segments".to_owned()); + } + let decoded = general_purpose::URL_SAFE_NO_PAD + .decode(payload) + .or_else(|_| general_purpose::URL_SAFE.decode(payload)) + .map_err(|_| "JWT payload is not valid base64url".to_owned())?; + serde_json::from_slice(&decoded).map_err(|_| "JWT payload is not valid JSON".to_owned()) +} + +pub(crate) fn verify_console_jwt(token: &str) -> Result +where + T: DeserializeOwned, +{ + let secret = jwt_signing_secret().ok_or_else(|| { + ApiError::Internal("CENTAUR_JWT_SIGNING_SECRET is not configured".to_owned()) + })?; + let audience = non_empty_env("CENTAUR_API_JWT_AUDIENCE") + .unwrap_or_else(|| DEFAULT_API_JWT_AUDIENCE.to_owned()); + let issuer = non_empty_env("CENTAUR_API_JWT_ISSUER") + .unwrap_or_else(|| DEFAULT_API_JWT_ISSUER.to_owned()); + verify_hs256_jwt(token, secret.as_bytes(), &audience, &issuer) +} + +pub(crate) fn verify_hs256_jwt( + token: &str, + secret: &[u8], + expected_audience: &str, + expected_issuer: &str, +) -> Result +where + T: DeserializeOwned, +{ + let mut validation = Validation::new(Algorithm::HS256); + validation.leeway = JWT_CLOCK_SKEW_SECONDS as u64; + validation.validate_nbf = true; + validation.set_audience(&[expected_audience]); + validation.set_issuer(&[expected_issuer]); + validation.set_required_spec_claims(&["exp", "iss", "sub", "aud"]); + let token_data = decode::(token, &DecodingKey::from_secret(secret), &validation) + .map_err(|_| ApiError::Unauthorized("invalid JWT".to_owned()))?; + let payload = + decode_jwt_payload(token).map_err(|_| ApiError::Unauthorized("invalid JWT".to_owned()))?; + validate_standard_claims(&payload)?; + Ok(token_data.claims) +} + +fn validate_standard_claims(claims: &Value) -> Result<(), ApiError> { + let now = time::OffsetDateTime::now_utc().unix_timestamp(); + let iat = claims + .get("iat") + .and_then(Value::as_i64) + .ok_or_else(|| ApiError::Unauthorized("JWT issued-at is required".to_owned()))?; + if iat > now + JWT_CLOCK_SKEW_SECONDS { + return Err(ApiError::Unauthorized( + "JWT issued-at is in the future".to_owned(), + )); + } + if claims + .get("sub") + .and_then(Value::as_str) + .unwrap_or_default() + .trim() + .is_empty() + { + return Err(ApiError::Unauthorized("JWT subject is required".to_owned())); + } + Ok(()) +} + +// Deployment JWT configuration is static, so it is resolved once per process. +// Tests mutate env per-case, so cfg!(test) reads live. +fn static_env(cell: &'static OnceLock>, name: &str) -> Option { + if cfg!(test) { + return env::var(name).ok(); + } + cell.get_or_init(|| env::var(name).ok()).clone() +} + +pub(crate) fn jwt_signing_secret() -> Option { + static CELL: OnceLock> = OnceLock::new(); + static_env(&CELL, "CENTAUR_JWT_SIGNING_SECRET") +} + +fn non_empty_env(name: &str) -> Option { + env::var(name) + .ok() + .map(|value| value.trim().to_owned()) + .filter(|value| !value.is_empty()) +} + +#[cfg(test)] +mod tests { + use super::*; + use axum::http::HeaderValue; + use jsonwebtoken::{EncodingKey, Header, encode}; + use serde_json::json; + + fn test_jwt(secret: &[u8], claims: Value) -> String { + encode( + &Header::new(Algorithm::HS256), + &claims, + &EncodingKey::from_secret(secret), + ) + .unwrap() + } + + #[test] + fn bearer_token_scheme_is_case_insensitive() { + for value in ["Bearer token-1", "bearer token-1", "BEARER token-1"] { + let mut headers = HeaderMap::new(); + headers.insert(header::AUTHORIZATION, value.parse().unwrap()); + assert_eq!(bearer_token(&headers).unwrap(), "token-1"); + } + + for value in ["Bearer ", "token-1", "Basic token-1"] { + let mut headers = HeaderMap::new(); + headers.insert(header::AUTHORIZATION, value.parse().unwrap()); + assert!(matches!( + bearer_token(&headers).unwrap_err(), + ApiError::Unauthorized(_) + )); + } + } + + #[test] + fn bearer_jwt_from_headers_requires_jwt_shape() { + let mut headers = HeaderMap::new(); + headers.insert( + header::AUTHORIZATION, + HeaderValue::from_static("Bearer not-a-jwt"), + ); + assert!(bearer_jwt_from_headers(&headers).is_none()); + + headers.insert( + header::AUTHORIZATION, + HeaderValue::from_static("Bearer header.payload.signature"), + ); + assert_eq!( + bearer_jwt_from_headers(&headers), + Some("header.payload.signature") + ); + } + + #[test] + fn verify_console_jwt_rejects_missing_issued_at() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "principal_123", + "aud": "centaur-api", + "exp": 4_102_444_800i64, + }), + ); + assert!(matches!( + verify_hs256_jwt::(&token, b"secret", "centaur-api", "centaur-console") + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } +} diff --git a/services/api-rs/crates/centaur-api-server/src/args.rs b/services/api-rs/crates/centaur-api-server/src/args.rs index 186967787..30c2e99b8 100644 --- a/services/api-rs/crates/centaur-api-server/src/args.rs +++ b/services/api-rs/crates/centaur-api-server/src/args.rs @@ -11,7 +11,10 @@ use std::{ #[cfg(unix)] use std::os::unix::fs::PermissionsExt; -use centaur_api_server::SandboxRuntime; +use centaur_api_server::{ + DiscoveredToolProxyFragment, SandboxRuntime, ToolDiscoveryConfig, discover_persona_registry, + discover_tool_proxy_fragment, +}; use centaur_iron_control::{ IdentityInput, IronControlClient, IronControlError, RegisterError, RoleSpec, SessionRegistrar, register_role, @@ -27,20 +30,56 @@ use centaur_sandbox_core::{Mount, MountKind, SandboxSpec}; use centaur_sandbox_local::LocalSandboxBackend; use centaur_sandbox_manager::{SandboxReaperConfig, WarmPoolConfig}; use centaur_session_core::HarnessType; -use centaur_session_runtime::{PersonaRegistry, SandboxWorkloadMode, SessionSandboxCleanupConfig}; +use centaur_session_runtime::{ + PersonaRegistry, SandboxCapacityConfig, SandboxWorkloadMode, SessionSandboxCleanupConfig, +}; use centaur_workflows::WorkflowHostSandboxRuntime; use clap::{Args as ClapArgs, Parser, ValueEnum}; -use tracing::{error, info, warn}; - -use crate::{ - ServerError, - tool_discovery::{ - DiscoveredToolProxyFragment, ToolDiscoveryConfig, discover_persona_registry, - discover_tool_proxy_fragment, - }, -}; +use serde_json::json; +use sha2::{Digest, Sha256}; +use tracing::{info, warn}; + +use crate::{ServerError, activity_summary::ActivitySummaryConfig}; const SANDBOX_REPOS_MOUNT_PATH: &str = "/home/agent/github"; +const GITHUB_TOKEN_ENV: &str = "GITHUB_TOKEN"; +const SLACK_BOT_TOKEN_ENV: &str = "SLACK_BOT_TOKEN"; +const SERVICE_API_KEY_ENVS: &[&str] = &[ + "CENTAUR_CONTROL_API_KEY", + "SLACKBOT_API_KEY", + "GITHUBBOT_API_KEY", + "LINEARBOT_API_KEY", + "DISCORDBOT_API_KEY", + "TEAMSBOT_API_KEY", + "WORKFLOW_API_KEY", + "SLACK_FEEDBACK_API_KEY", + "CENTAUR_JWT_SIGNING_SECRET", +]; +const MIN_SERVICE_API_KEY_BYTES: usize = 32; + +pub(crate) fn validate_service_api_key_separation() -> Result<(), ServerError> { + let mut owners = BTreeMap::::new(); + for env_name in SERVICE_API_KEY_ENVS { + let Ok(value) = env::var(env_name) else { + continue; + }; + let value = value.trim(); + if value.is_empty() { + continue; + } + if value.len() < MIN_SERVICE_API_KEY_BYTES { + return Err(ServerError::UnsupportedConfig(format!( + "{env_name} must contain at least {MIN_SERVICE_API_KEY_BYTES} bytes" + ))); + } + if let Some(existing) = owners.insert(value.to_owned(), env_name) { + return Err(ServerError::UnsupportedConfig(format!( + "{existing} and {env_name} must contain distinct service credentials" + ))); + } + } + Ok(()) +} /// OTLP env always forwarded from the api-rs process into codex sandboxes, /// mirroring the Python control plane's `_SANDBOX_PASSTHROUGH_ENV_KEYS`. The @@ -60,6 +99,8 @@ pub(crate) struct Args { pub(crate) server: ServerArgs, #[command(flatten)] sandbox: SandboxArgs, + #[command(flatten)] + activity_summary: ActivitySummaryArgs, } impl Args { @@ -73,12 +114,6 @@ impl Args { self.sandbox.iron_control_runtime().await } - pub(crate) fn iron_control_tool_reconciler( - &self, - ) -> Result, ServerError> { - self.sandbox.iron_control_tool_reconciler() - } - pub(crate) fn persona_registry(&self) -> Result { self.sandbox.persona_registry() } @@ -87,6 +122,10 @@ impl Args { self.sandbox.warm_pool_config() } + pub(crate) fn sandbox_capacity_config(&self) -> Option { + self.sandbox.sandbox_capacity_config() + } + pub(crate) fn sandbox_reaper_config(&self) -> SandboxReaperConfig { self.sandbox.sandbox_reaper_config() } @@ -103,6 +142,19 @@ impl Args { .workflow_host_sandbox_runtime(bootstrap_iron_control_principal) .await } + + pub(crate) fn activity_summary_config(&self) -> Option { + self.activity_summary.config() + } + + pub(crate) fn shutdown_execution_drain_timeout(&self) -> Duration { + Duration::from_secs(self.server.shutdown_execution_drain_timeout_secs) + } + + pub(crate) fn execution_adoption_interval(&self) -> Option { + (self.server.execution_adoption_interval_secs > 0) + .then(|| Duration::from_secs(self.server.execution_adoption_interval_secs)) + } } pub(crate) struct IronControlRuntime { @@ -111,14 +163,80 @@ pub(crate) struct IronControlRuntime { pub(crate) workflow_host_principal: String, } -pub(crate) struct IronControlToolReconciler { - client: IronControlClient, - namespace: String, - source_policy: SourcePolicy, - base_infra_fragment: ProxyFragment, - tool_dirs: Vec, - tool_git_sources: Vec, - interval: Duration, +#[derive(Debug, ClapArgs)] +struct ActivitySummaryArgs { + /// Enable API-side model summaries of durable Codex App Server activity. + #[arg( + long = "session-activity-summary-enabled", + env = "SESSION_ACTIVITY_SUMMARY_ENABLED", + default_value_t = false, + action = clap::ArgAction::Set + )] + enabled: bool, + #[arg( + long = "session-activity-summary-model", + env = "SESSION_ACTIVITY_SUMMARY_MODEL", + default_value = "gpt-5.4-nano" + )] + model: String, + #[arg( + long = "session-activity-summary-openai-base-url", + env = "SESSION_ACTIVITY_SUMMARY_OPENAI_BASE_URL", + default_value = "https://api.openai.com/v1" + )] + openai_base_url: String, + #[arg( + long = "session-activity-summary-min-interval-secs", + env = "SESSION_ACTIVITY_SUMMARY_MIN_INTERVAL_SECS", + default_value_t = 20, + value_parser = clap::value_parser!(u64).range(1..) + )] + min_interval_secs: u64, + #[arg( + long = "session-activity-summary-timeout-secs", + env = "SESSION_ACTIVITY_SUMMARY_TIMEOUT_SECS", + default_value_t = 5, + value_parser = clap::value_parser!(u64).range(1..) + )] + timeout_secs: u64, + #[arg( + long = "session-activity-summary-max-facts", + env = "SESSION_ACTIVITY_SUMMARY_MAX_FACTS", + default_value_t = 12, + value_parser = clap::value_parser!(u64).range(1..) + )] + max_facts: u64, + #[arg( + long = "session-activity-summary-max-output-tokens", + env = "SESSION_ACTIVITY_SUMMARY_MAX_OUTPUT_TOKENS", + default_value_t = 128, + value_parser = clap::value_parser!(u64).range(1..) + )] + max_output_tokens: u64, +} + +impl ActivitySummaryArgs { + fn config(&self) -> Option { + if !self.enabled { + return None; + } + let Some(api_key) = clean_optional_value(env::var("OPENAI_API_KEY").ok().as_deref()) else { + warn!( + "session activity summaries are enabled but no OpenAI credential is configured; \ + set OPENAI_API_KEY in the api-rs environment" + ); + return None; + }; + Some(ActivitySummaryConfig { + base_url: self.openai_base_url.clone(), + api_key, + max_facts: usize::try_from(self.max_facts).unwrap_or(usize::MAX), + max_output_tokens: u16::try_from(self.max_output_tokens).unwrap_or(u16::MAX), + min_interval: Duration::from_secs(self.min_interval_secs), + model: self.model.clone(), + timeout: Duration::from_secs(self.timeout_secs), + }) + } } #[derive(Clone, Debug, Eq, PartialEq)] @@ -130,79 +248,6 @@ struct ToolGitSource { repo_cache_path: Option, } -impl IronControlToolReconciler { - pub(crate) async fn run(self) { - let mut interval = tokio::time::interval(self.interval); - // The startup path already registered once; wait a full period so this - // task only handles post-start git/volume updates. - interval.tick().await; - loop { - interval.tick().await; - if let Err(error) = self.reconcile_once().await { - error!(%error, "failed to reconcile iron-control tool secrets"); - } - } - } - - async fn reconcile_once(&self) -> Result<(), ServerError> { - let tool_dirs = self.tool_dirs()?; - let tool_fragment = self.discover_tool_proxy_fragment()?; - let mut infra = self.base_infra_fragment.clone(); - if let Some(tool_fragment) = &tool_fragment { - merge_fragment(&mut infra, tool_fragment.fragment.clone()); - } - let role_id = register_role( - &self.client, - &self.namespace, - &RoleSpec::infra(), - &infra, - &self.source_policy, - ) - .await?; - info!( - role_id, - tool_dirs = ?tool_dirs, - tool_count = tool_fragment - .as_ref() - .map_or(0, |fragment| fragment.tool_count), - secret_count = tool_fragment - .as_ref() - .map_or(0, |fragment| fragment.secret_count), - "reconciled iron-control tool secrets" - ); - Ok(()) - } - - fn discover_tool_proxy_fragment( - &self, - ) -> Result, ServerError> { - let tool_dirs = self.tool_dirs()?; - let discovered = discover_tool_proxy_fragment(&tool_dirs)?; - if discovered.secret_count == 0 { - return Ok(None); - } - Ok(Some(discovered)) - } - - fn tool_dirs(&self) -> Result, ServerError> { - if !self.tool_git_sources.is_empty() { - let mut dirs = Vec::with_capacity(self.tool_git_sources.len()); - for source in &self.tool_git_sources { - source.sync()?; - let tools_dir = source.tools_dir(); - // Skip sources without a tools tree (chart-defaulted subdirs - // make this a normal case for non-tool overlay repos). - if !tools_dir.is_dir() { - continue; - } - dirs.push(tools_dir); - } - return Ok(dirs); - } - Ok(self.tool_dirs.clone()) - } -} - impl ToolGitSource { fn from_config(tools: &ToolsConfig) -> Vec { let mut sources = vec![Self::from_source( @@ -210,6 +255,7 @@ impl ToolGitSource { repo: tools.repo.clone(), git_ref: tools.git_ref.clone(), source_subdir: tools.source_subdir.clone(), + visibility: tools.visibility.clone(), }, tools.repo_cache_path.clone(), )]; @@ -452,6 +498,31 @@ pub(crate) struct ServerArgs { pub(crate) bind_addr: SocketAddr, #[arg(long, env = "RUN_MIGRATIONS", default_value_t = false)] pub(crate) run_migrations: bool, + /// How long shutdown waits for in-flight executions to finish before + /// releasing their stdout-owner leases for adoption by a peer. Keep + /// below the pod's terminationGracePeriodSeconds (35s in the chart) so + /// the release happens before SIGKILL. 0 releases immediately. + #[arg( + long = "shutdown-execution-drain-timeout-secs", + env = "SHUTDOWN_EXECUTION_DRAIN_TIMEOUT_SECS", + default_value_t = 20, + value_parser = clap::value_parser!(u64).range(0..=600) + )] + shutdown_execution_drain_timeout_secs: u64, + /// How often to re-run the orphaned-execution adoption scan after the + /// startup pass. Executions orphaned while the process is already + /// running (e.g. a rolling deploy terminating the previous pod mid-turn + /// after this pod's startup scan) are only recovered by these re-scans, + /// so the interval bounds how long a handed-off turn stays frozen. A + /// steady-state tick is a single SELECT (executions with a live + /// stdout-owner lease are skipped before any session or sandbox reads). + /// 0 disables re-scans and keeps the startup-only behavior. + #[arg( + long = "session-execution-adoption-interval-secs", + env = "SESSION_EXECUTION_ADOPTION_INTERVAL_SECS", + default_value_t = 15 + )] + execution_adoption_interval_secs: u64, } #[derive(Debug, ClapArgs)] @@ -474,12 +545,12 @@ struct SandboxArgs { workload: SandboxWorkloadKind, /// The default harness for warm sandboxes. Per-session sandboxes always /// run their session's harness (pinned via container args); this only - /// decides what the warm pool boots ahead of time. Defaults to claudecode + /// decides what the warm pool boots ahead of time. Defaults to codex /// to match the sandbox image's CMD. #[arg( long = "session-sandbox-harness", env = "SESSION_SANDBOX_HARNESS", - default_value = "claudecode" + default_value = "codex" )] default_harness: HarnessType, #[arg(long = "centaur-default-persona", env = "CENTAUR_DEFAULT_PERSONA")] @@ -529,21 +600,29 @@ struct SandboxArgs { value_parser = clap::value_parser!(u64).range(1..) )] warm_pool_replenish_interval_secs: u64, - /// Stop sandboxes that have been idle-paused longer than this. 0 disables - /// the idle sweep. + /// Hard cap on observed running-like sandboxes. 0 disables capacity + /// admission. + #[arg( + long = "session-sandbox-running-limit", + env = "SESSION_SANDBOX_RUNNING_LIMIT", + default_value_t = 0 + )] + sandbox_running_limit: usize, + /// Do not evict assigned idle sandboxes that were active within this + /// window. Warm sandboxes can still be discarded first. #[arg( - long = "session-sandbox-idle-stop-ttl-secs", - env = "SESSION_SANDBOX_IDLE_STOP_TTL_SECS", - default_value_t = 3600 + long = "session-sandbox-hot-idle-grace-secs", + env = "SESSION_SANDBOX_HOT_IDLE_GRACE_SECS", + default_value_t = 300 )] - sandbox_idle_stop_ttl_secs: u64, + sandbox_hot_idle_grace_secs: u64, /// Stop any sandbox older than this regardless of status; sessions replace /// reaped sandboxes on their next message. 0 disables the max-lifetime /// sweep. #[arg( long = "session-sandbox-max-lifetime-secs", env = "SESSION_SANDBOX_MAX_LIFETIME_SECS", - default_value_t = 86_400 + default_value_t = 259_200 )] sandbox_max_lifetime_secs: u64, #[arg( @@ -597,6 +676,16 @@ struct SandboxArgs { extra_env_json: Option, #[arg(long = "centaur-overlay-image", env = "CENTAUR_OVERLAY_IMAGE")] overlay_image: Option, + /// Release-scoped digest of every repo/image/config input copied into a + /// sandbox at boot. The chart computes this from the complete overlay + /// source manifest (including skills-only sources and repo-cache ref + /// overrides), so a content rollout changes the warm workload key even + /// when the tools-only compatibility view is unchanged. + #[arg( + long = "centaur-sandbox-content-revision", + env = "CENTAUR_SANDBOX_CONTENT_REVISION" + )] + sandbox_content_revision: Option, #[arg( long = "centaur-overlay-image-pull-policy", env = "CENTAUR_OVERLAY_IMAGE_PULL_POLICY" @@ -647,12 +736,6 @@ struct SandboxArgs { kubernetes_workflow_dirs: Option, #[command(flatten)] tools_source: ToolsArgs, - #[arg( - long = "tool-proxy-reconcile-interval-secs", - env = "TOOL_PROXY_RECONCILE_INTERVAL_SECS", - default_value_t = 60 - )] - tool_proxy_reconcile_interval_secs: u64, } impl SandboxArgs { @@ -665,8 +748,7 @@ impl SandboxArgs { let namespace = self.iron_control.namespace.clone(); let role_ids = if self.iron_control_sync_infra_secrets { let policy = self.iron_proxy.source_policy(); - let tool_fragment = self.discover_tool_proxy_fragment()?; - let roles = self.iron_proxy.roles_to_register(tool_fragment.as_ref())?; + let roles = self.iron_proxy.roles_to_register()?; let mut role_ids = Vec::with_capacity(roles.len()); for (spec, fragment) in &roles { role_ids.push( @@ -720,45 +802,16 @@ impl SandboxArgs { })) } - /// Background registration for git/volume-backed tool updates. Startup - /// registration keeps the stable infra role current; re-upserting that role - /// here adds newly discovered tool secrets to principals that hold the role - /// without restarting api-rs or sandboxes. Session registration only seeds - /// this role onto brand-new principals, so operator revocations stay sticky. - fn iron_control_tool_reconciler( - &self, - ) -> Result, ServerError> { - if !self.iron_control_sync_infra_secrets { - return Ok(None); - } - let Some(client) = self.iron_control.client() else { - return Ok(None); - }; - if self.tool_proxy_reconcile_interval_secs == 0 { - return Ok(None); - } - Ok(Some(IronControlToolReconciler { - client, - namespace: self.iron_control.namespace.clone(), - source_policy: self.iron_proxy.source_policy(), - base_infra_fragment: self.iron_proxy.infra_fragment()?, - tool_dirs: self.tools.resolve_tool_dirs()?, - tool_git_sources: self - .tools_source - .to_config() - .as_ref() - .map(ToolGitSource::from_config) - .unwrap_or_default(), - interval: Duration::from_secs(self.tool_proxy_reconcile_interval_secs), - })) - } - fn persona_registry(&self) -> Result { let default_persona_id = clean_optional_value(self.default_persona.as_deref()); - Ok(discover_persona_registry( - &self.tools.resolve_tool_dirs()?, - default_persona_id, - )?) + let public_source_roots = self + .tools + .resolve_public_tool_dirs() + .into_iter() + .map(|path| path.display().to_string()); + let registry = + discover_persona_registry(&self.tools.resolve_tool_dirs()?, default_persona_id)?; + Ok(registry.with_public_source_roots(public_source_roots)) } async fn runtime(&self) -> Result { @@ -934,11 +987,34 @@ impl SandboxArgs { match self.workload { SandboxWorkloadKind::Mock => Ok(SandboxWorkloadMode::mock_app_server(image)), SandboxWorkloadKind::CodexAppServer => { - let mut workload = SandboxWorkloadMode::codex_app_server( - image, - self.codex_app_server_env_template()?, - self.default_harness.clone(), - ); + let mut env = self.codex_app_server_env_template()?; + env.push(( + "CENTAUR_SANDBOX_BOOTSTRAP_FINGERPRINT".to_owned(), + sandbox_bootstrap_fingerprint( + self.tools_source.to_config().as_ref(), + clean_optional_value(self.overlay_image.as_deref()) + .map(|image| OverlayImageConfig { + image, + image_pull_policy: clean_optional_value( + self.overlay_image_pull_policy.as_deref(), + ), + source_path: clean_optional_value(Some( + self.overlay_image_source_path.as_str(), + )) + .unwrap_or_else(|| "/overlay".to_owned()), + mount_path: clean_optional_value(Some( + self.sandbox_overlay_dir.as_str(), + )) + .unwrap_or_else(|| "/home/agent/overlay/org".to_owned()), + }) + .as_ref(), + self.agent_image_pull_policy.as_deref(), + &self.image_pull_secrets, + clean_optional_value(self.sandbox_content_revision.as_deref()).as_deref(), + )?, + )); + let mut workload = + SandboxWorkloadMode::codex_app_server(image, env, self.default_harness.clone()); if let Some(repos_path) = clean_optional_value(self.repos_path.as_deref()) { workload = workload.mount( Mount::new(self.repos_mount_kind(repos_path), SANDBOX_REPOS_MOUNT_PATH) @@ -978,9 +1054,10 @@ impl SandboxArgs { // Inject the infra/harness placeholder credentials so env-based // consumers send the proxy_value iron-proxy replaces with the real - // secret: codex's OPENAI_API_KEY (api_key mode → codex logs in and + // secret: codex's OPENAI_API_KEY (api_key mode -> codex logs in and // hits api.openai.com instead of falling back to the ChatGPT - // auth.json), git's GITHUB_TOKEN, and the rest of the infra set. + // auth.json), git/gh's GITHUB_TOKEN, the slack tool's + // SLACK_BOT_TOKEN, and the rest of the infra set. for (name, value) in self.iron_proxy.sandbox_placeholder_env()? { if !envs.iter().any(|(existing, _)| existing == &name) { envs.push((name, value)); @@ -1002,6 +1079,12 @@ impl SandboxArgs { "OPENROUTER_API_KEY".to_owned(), )); } + if !envs + .iter() + .any(|(existing, _)| existing == "META_AI_API_KEY") + { + envs.push(("META_AI_API_KEY".to_owned(), "META_AI_API_KEY".to_owned())); + } // When Bedrock is enabled, codex's `amazon-bedrock` provider signs with // these placeholder AWS credentials and iron-proxy re-signs (SigV4) with // the real IAM keys. `aws_auth` is not a `secrets` transform, so the @@ -1098,11 +1181,10 @@ impl SandboxArgs { .collect() } - /// Per-sandbox OTLP egress NetworkPolicy target, derived from the OTLP - /// endpoint the codex sandbox env will carry. Only in-cluster service DNS - /// endpoints (`..svc[...]`) map to a namespace - /// selector; anything else gets no rule and a warning, because a silently - /// missing rule means harness usage/cost spans never reach the collector. + /// Per-sandbox proxy OTLP egress NetworkPolicy target, derived from the + /// OTLP endpoint the codex sandbox env will carry. Only in-cluster service + /// DNS endpoints (`..svc[...]`) map to a namespace + /// selector. fn sandbox_otlp_egress_target(&self) -> Result, ServerError> { if !matches!(self.workload, SandboxWorkloadKind::CodexAppServer) { return Ok(None); @@ -1127,7 +1209,7 @@ impl SandboxArgs { namespace = %target.namespace, port = target.port, endpoint = %endpoint, - "sandbox OTLP egress enabled" + "sandbox proxy OTLP egress enabled" ); Ok(Some(target)) } @@ -1135,7 +1217,7 @@ impl SandboxArgs { warn!( endpoint = %endpoint, "sandbox OTLP endpoint is not an in-cluster service DNS name; \ - no sandbox egress NetworkPolicy rule will be created for it" + no proxy egress NetworkPolicy rule will be created for it" ); Ok(None) } @@ -1243,6 +1325,15 @@ impl SandboxArgs { target_size: self.warm_pool_size, replenish_interval: Duration::from_secs(self.warm_pool_replenish_interval_secs), bootstrap_iron_control_principal: None, + max_running_sandboxes: (self.sandbox_running_limit > 0) + .then_some(self.sandbox_running_limit), + }) + } + + fn sandbox_capacity_config(&self) -> Option { + (self.sandbox_running_limit > 0).then(|| SandboxCapacityConfig { + max_running: self.sandbox_running_limit, + hot_idle_grace: Duration::from_secs(self.sandbox_hot_idle_grace_secs), }) } @@ -1250,7 +1341,6 @@ impl SandboxArgs { let ttl = |secs: u64| (secs > 0).then(|| Duration::from_secs(secs)); SandboxReaperConfig { interval: Duration::from_secs(self.sandbox_reap_interval_secs), - idle_ttl: ttl(self.sandbox_idle_stop_ttl_secs), max_lifetime: ttl(self.sandbox_max_lifetime_secs), } } @@ -1264,6 +1354,61 @@ impl SandboxArgs { } } +fn sandbox_bootstrap_fingerprint( + tools: Option<&ToolsConfig>, + overlay: Option<&OverlayImageConfig>, + agent_image_pull_policy: Option<&str>, + image_pull_secrets: &[String], + content_revision: Option<&str>, +) -> Result { + let tools = tools.map(|tools| { + json!({ + "repo": tools.repo, + "git_ref": tools.git_ref, + "source_subdir": tools.source_subdir, + "visibility": tools.visibility, + "image": tools.image, + "image_pull_policy": tools.image_pull_policy, + "github_token": tools.github_token.as_ref().map(|token| json!({ + "secret_name": token.secret_name, + "secret_key": token.secret_key, + })), + "repo_cache_path": tools.repo_cache_path, + "repo_cache_pvc": tools.repo_cache_pvc, + "repo_cache_sub_path": tools.repo_cache_sub_path, + "auto_reload": tools.auto_reload, + "extra_sources": tools.extra_sources.iter().map(|source| json!({ + "repo": source.repo, + "git_ref": source.git_ref, + "source_subdir": source.source_subdir, + "visibility": source.visibility, + })).collect::>(), + }) + }); + let overlay = overlay.map(|overlay| { + json!({ + "image": overlay.image, + "image_pull_policy": overlay.image_pull_policy, + "source_path": overlay.source_path, + "mount_path": overlay.mount_path, + }) + }); + let material = serde_json::to_vec(&json!({ + "version": 2, + "tools": tools, + "overlay_image": overlay, + "agent_image_pull_policy": agent_image_pull_policy, + "image_pull_secrets": image_pull_secrets, + "content_revision": content_revision, + })) + .map_err(|error| { + ServerError::UnsupportedConfig(format!( + "failed to fingerprint sandbox bootstrap configuration: {error}" + )) + })?; + Ok(format!("{:x}", Sha256::digest(material))) +} + const IRON_CONTROL_REGISTER_MAX_ATTEMPTS: u32 = 5; const IRON_CONTROL_REGISTER_INITIAL_BACKOFF: Duration = Duration::from_millis(250); @@ -1314,6 +1459,8 @@ fn should_retry_iron_control_register(error: &RegisterError) -> bool { struct ToolDiscoveryArgs { #[arg(long = "tool-dirs", env = "TOOL_DIRS")] tool_dirs: Option, + #[arg(long = "public-tool-dirs", env = "KUBERNETES_PUBLIC_TOOL_DIRS")] + public_tool_dirs: Option, #[arg(long = "tools-path", env = "TOOLS_PATH")] tools_path: Option, #[arg(long = "tools-overlay-path", env = "TOOLS_OVERLAY_PATH")] @@ -1328,6 +1475,7 @@ impl ToolDiscoveryArgs { fn resolve_tool_dirs(&self) -> Result, ServerError> { Ok(ToolDiscoveryConfig { tool_dirs: self.tool_dirs.clone(), + public_tool_dirs: self.public_tool_dirs.clone(), tools_path: self.tools_path.clone(), tools_overlay_path: self.tools_overlay_path.clone(), plugins_dir: self.plugins_dir.clone(), @@ -1335,6 +1483,18 @@ impl ToolDiscoveryArgs { } .resolve_tool_dirs()?) } + + fn resolve_public_tool_dirs(&self) -> Vec { + ToolDiscoveryConfig { + tool_dirs: self.tool_dirs.clone(), + public_tool_dirs: self.public_tool_dirs.clone(), + tools_path: self.tools_path.clone(), + tools_overlay_path: self.tools_overlay_path.clone(), + plugins_dir: self.plugins_dir.clone(), + tools_config: self.tools_config.clone(), + } + .resolve_public_tool_dirs() + } } impl TryFrom<&SandboxArgs> for AgentSandboxConfig { @@ -1375,9 +1535,8 @@ impl TryFrom<&SandboxArgs> for AgentSandboxConfig { .unwrap_or_else(|| "/home/agent/overlay/org".to_owned()), }); } - // Direct harness OTLP export (codex usage/cost spans) needs a hole in - // the per-sandbox egress NetworkPolicy; derived from the sandbox's own - // OTLP endpoint env so there is a single source of truth. + // The chart label policy handles sandbox OTLP egress; keep the + // per-sandbox proxy's own in-cluster OTLP egress explicit. config.otlp_egress = args.sandbox_otlp_egress_target()?; // iron-control is the only proxy mode: a per-sandbox proxy syncs its // secrets from the control plane, so configuring iron-proxy without @@ -1460,6 +1619,21 @@ struct ToolsArgs { env = "KUBERNETES_TOOLS_REPO_CACHE_PVC" )] repo_cache_pvc: Option, + #[arg( + id = "tools_visibility", + long = "kubernetes-tools-visibility", + env = "KUBERNETES_TOOLS_VISIBILITY", + default_value = "private" + )] + visibility: Option, + #[arg( + id = "tools_auto_reload", + long = "kubernetes-tools-auto-reload", + env = "KUBERNETES_TOOLS_AUTO_RELOAD", + default_value_t = true, + action = clap::ArgAction::Set + )] + auto_reload: bool, #[arg( id = "tools_extra_sources", long = "kubernetes-tools-extra-sources", @@ -1501,6 +1675,7 @@ impl ToolsArgs { let mut config = ToolsConfig::new(repo, image); config.image_pull_policy = self.image_pull_policy.clone(); config.git_ref = clean_optional_value(self.git_ref.as_deref()); + config.visibility = repository_visibility(self.visibility.as_deref()); if let Some(subdir) = clean_optional_value(Some(self.source_subdir.as_str())) { config.source_subdir = subdir; } @@ -1513,6 +1688,7 @@ impl ToolsArgs { } config.repo_cache_path = clean_optional_value(self.repo_cache_path.as_deref()); config.repo_cache_pvc = clean_optional_value(self.repo_cache_pvc.as_deref()); + config.auto_reload = self.auto_reload; config.extra_sources = self.extra_sources(); Some(config) } @@ -1525,6 +1701,8 @@ struct ToolSourceArg { git_ref: Option, #[serde(default)] subdir: Option, + #[serde(default)] + visibility: Option, } impl ToolSourceArg { @@ -1540,10 +1718,18 @@ impl ToolSourceArg { .as_deref() .and_then(|value| clean_optional_value(Some(value))) .unwrap_or_else(|| "tools".to_owned()), + visibility: repository_visibility(self.visibility.as_deref()), }) } } +fn repository_visibility(value: Option<&str>) -> String { + match value.map(str::trim).map(str::to_ascii_lowercase).as_deref() { + Some("public") => "public".to_owned(), + _ => "private".to_owned(), + } +} + #[derive(Debug, ClapArgs)] struct IronProxyArgs { #[arg( @@ -1564,6 +1750,12 @@ struct IronProxyArgs { env = "KUBERNETES_IRON_PROXY_IMAGE_PULL_POLICY" )] image_pull_policy: Option, + #[arg( + long = "kubernetes-iron-proxy-upstream-deny-cidrs", + env = "KUBERNETES_IRON_PROXY_UPSTREAM_DENY_CIDRS", + value_delimiter = ',' + )] + upstream_deny_cidrs: Vec, #[command(flatten)] ca: IronProxyCaArgs, #[command(flatten)] @@ -1600,6 +1792,12 @@ impl IronProxyArgs { let mut config = IronProxyConfig::new(self.image.clone(), ca_cert_secret_name, ca_key_secret_name); config.image_pull_policy = self.image_pull_policy.clone(); + config.upstream_deny_cidrs = self + .upstream_deny_cidrs + .iter() + .filter_map(|cidr| non_empty(Some(cidr.as_str()))) + .map(ToOwned::to_owned) + .collect(); self.source.apply_to_config(&mut config); config.fragments = harness_fragments; config.env_from_secret_names = self.env_from_secret_names(); @@ -1618,22 +1816,15 @@ impl IronProxyArgs { } /// The role to register in iron-control. The shared `infra` role contains - /// infra, harness, and discovered tool secrets, and every session principal - /// is granted that single role (see [`SessionRegistrar`]). - fn roles_to_register( - &self, - tool_fragment: Option<&DiscoveredToolProxyFragment>, - ) -> Result, ServerError> { - let mut infra = self.infra_fragment()?; - if let Some(tool_fragment) = tool_fragment { - merge_fragment(&mut infra, tool_fragment.fragment.clone()); - } + /// infra and harness secrets, and every session principal is granted that + /// role (see [`SessionRegistrar`]). + fn roles_to_register(&self) -> Result, ServerError> { + let infra = self.infra_fragment()?; Ok(vec![(RoleSpec::infra(), infra)]) } /// The full infra fragment: the shared infra secrets plus every available - /// harness auth fragment (also infra), selected by auth mode. Discovered - /// tool secrets are folded into the same infra role at registration time. + /// harness auth fragment (also infra), selected by auth mode. fn infra_fragment(&self) -> Result { let mut infra = infra_fragment()?; for fragment in self.harness.fragments()? { @@ -1644,14 +1835,17 @@ impl IronProxyArgs { /// Placeholder env (`PLACEHOLDER=PLACEHOLDER`) for the infra/harness /// secrets, whose consumers read credentials straight from the environment - /// (codex's `OPENAI_API_KEY`, git's `GITHUB_TOKEN`, …). Discovered tool + /// (for example codex's `OPENAI_API_KEY`). Discovered tool /// secrets contribute nothing here: tools read credentials through the SDK, /// whose `StubBackend` already returns the key name iron-proxy matches on, /// and the cloudwatch tool embeds its own throwaway SigV4 credentials. fn sandbox_placeholder_env(&self) -> Result, ServerError> { - Ok(centaur_iron_proxy::placeholder_env(&[ - self.infra_fragment()? - ])) + let mut env = centaur_iron_proxy::placeholder_env(&[self.infra_fragment()?]); + env.entry(GITHUB_TOKEN_ENV.to_owned()) + .or_insert_with(|| GITHUB_TOKEN_ENV.to_owned()); + env.entry(SLACK_BOT_TOKEN_ENV.to_owned()) + .or_insert_with(|| SLACK_BOT_TOKEN_ENV.to_owned()); + Ok(env) } fn env_from_secret_names(&self) -> Vec { @@ -1712,6 +1906,12 @@ struct IronProxySourceArgs { default_value = "10m" )] secret_ttl: String, + #[arg( + long = "kubernetes-firewall-manager-secret-env-prefix", + env = "FIREWALL_MANAGER_SECRET_ENV_PREFIX", + default_value = "" + )] + secret_env_prefix: String, #[arg( long = "kubernetes-op-connect-host", env = "KUBERNETES_OP_CONNECT_HOST" @@ -1735,6 +1935,7 @@ impl IronProxySourceArgs { kind: self.source, op_vault: self.op_vault.clone(), ttl: self.secret_ttl.clone(), + env_prefix: self.secret_env_prefix.clone(), } } @@ -1766,7 +1967,7 @@ struct IronProxyHarnessArgs { #[arg( long = "kubernetes-iron-proxy-harness-engine", env = "KUBERNETES_IRON_PROXY_HARNESS_ENGINE", - default_value = "claudecode" + default_value = "codex" )] engine: HarnessType, #[arg( @@ -1822,6 +2023,9 @@ impl IronProxyHarnessArgs { if let Some(fragment) = harness_auth_fragment("openrouter", "api_key")? { fragments.push(fragment); } + if let Some(fragment) = harness_auth_fragment("meta-ai", "api_key")? { + fragments.push(fragment); + } // Bedrock is opt-in (not the default codex provider): only register its // SigV4 re-signing fragment when the operator has set CODEX_BEDROCK_REGION, // since the fragment expects AWS keys in the secrets backend. @@ -1913,8 +2117,8 @@ fn parse_host_port(value: &str) -> Option { } /// Map an OTLP endpoint URL onto a NetworkPolicy egress target. Only -/// in-cluster service DNS hosts (`..svc[.]`) -/// are mapped; the namespace label is the policy's `kubernetes.io/metadata.name` +/// in-cluster service DNS hosts (`..svc[...]`) are mapped; +/// the namespace label is the policy's `kubernetes.io/metadata.name` /// selector. Ports default by scheme when absent. fn parse_otlp_egress_target(endpoint: &str) -> Option { let trimmed = endpoint.trim(); @@ -2015,6 +2219,162 @@ mod tests { } } + #[test] + fn service_api_keys_must_be_distinct_across_trust_lanes() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ( + "CENTAUR_CONTROL_API_KEY", + "shared-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "SLACKBOT_API_KEY", + "shared-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ("GITHUBBOT_API_KEY", ""), + ("LINEARBOT_API_KEY", ""), + ("DISCORDBOT_API_KEY", ""), + ("TEAMSBOT_API_KEY", ""), + ( + "WORKFLOW_API_KEY", + "workflow-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "SLACK_FEEDBACK_API_KEY", + "feedback-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "CENTAUR_JWT_SIGNING_SECRET", + "jwt-signing-key-xxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ]); + + let error = validate_service_api_key_separation() + .expect_err("equal control and Slackbot keys must fail startup"); + assert!(error.to_string().contains("must contain distinct")); + assert!(!error.to_string().contains("shared-key")); + } + + #[test] + fn configured_service_api_keys_must_be_at_least_32_bytes() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ( + "CENTAUR_CONTROL_API_KEY", + "control-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ("WORKFLOW_API_KEY", "short"), + ( + "CENTAUR_JWT_SIGNING_SECRET", + "jwt-signing-key-xxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ]); + + let error = validate_service_api_key_separation().expect_err("short workflow key"); + assert!( + error + .to_string() + .contains("WORKFLOW_API_KEY must contain at least 32 bytes") + ); + assert!(!error.to_string().contains("short")); + } + + #[test] + fn jwt_signing_key_must_be_distinct_from_service_credentials() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ( + "CENTAUR_CONTROL_API_KEY", + "shared-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "CENTAUR_JWT_SIGNING_SECRET", + "shared-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ("SLACKBOT_API_KEY", ""), + ("GITHUBBOT_API_KEY", ""), + ("LINEARBOT_API_KEY", ""), + ("DISCORDBOT_API_KEY", ""), + ("TEAMSBOT_API_KEY", ""), + ("WORKFLOW_API_KEY", ""), + ("SLACK_FEEDBACK_API_KEY", ""), + ]); + + let error = validate_service_api_key_separation() + .expect_err("equal signing and control keys must fail startup"); + assert!(error.to_string().contains("must contain distinct")); + assert!(!error.to_string().contains("shared-key")); + } + + #[test] + fn jwt_signing_key_must_be_at_least_32_bytes() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_CONTROL_API_KEY", ""), + ("SLACKBOT_API_KEY", ""), + ("GITHUBBOT_API_KEY", ""), + ("LINEARBOT_API_KEY", ""), + ("DISCORDBOT_API_KEY", ""), + ("TEAMSBOT_API_KEY", ""), + ("WORKFLOW_API_KEY", ""), + ("SLACK_FEEDBACK_API_KEY", ""), + ("CENTAUR_JWT_SIGNING_SECRET", "short"), + ]); + + let error = validate_service_api_key_separation().expect_err("short signing key"); + assert!( + error + .to_string() + .contains("CENTAUR_JWT_SIGNING_SECRET must contain at least 32 bytes") + ); + assert!(!error.to_string().contains("short")); + } + + #[test] + fn distinct_service_api_keys_pass_startup_validation() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ( + "CENTAUR_CONTROL_API_KEY", + "control-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "SLACKBOT_API_KEY", + "slackbot-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "GITHUBBOT_API_KEY", + "githubbot-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "LINEARBOT_API_KEY", + "linearbot-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "DISCORDBOT_API_KEY", + "discordbot-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "TEAMSBOT_API_KEY", + "teamsbot-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "WORKFLOW_API_KEY", + "workflow-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "SLACK_FEEDBACK_API_KEY", + "feedback-key-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ( + "CENTAUR_JWT_SIGNING_SECRET", + "jwt-signing-key-xxxxxxxxxxxxxxxxxxxxxxxxxxx", + ), + ]); + + validate_service_api_key_separation().expect("distinct service keys"); + } + #[test] fn iron_control_registration_retry_policy_is_transient_only() { let status_error = |status| { @@ -2037,6 +2397,55 @@ mod tests { )); } + #[test] + fn activity_summary_uses_direct_openai_key_by_default() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("OPENAI_API_KEY", "sk-test"), + ("FIREWALL_MANAGER_SECRET_SOURCE", "env"), + ("KUBERNETES_OP_CONNECT_HOST", ""), + ("OP_CONNECT_TOKEN", ""), + ("OP_VAULT", ""), + ]); + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + "--session-activity-summary-enabled", + "true", + ]) + .unwrap(); + + let config = args.activity_summary_config().unwrap(); + assert_eq!(config.api_key, "sk-test"); + } + + #[test] + fn activity_summary_uses_mounted_openai_key_even_with_onepassword_connect_source() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("OPENAI_API_KEY", "sk-mounted"), + ("FIREWALL_MANAGER_SECRET_SOURCE", "onepassword-connect"), + ( + "KUBERNETES_OP_CONNECT_HOST", + "http://onepassword-connect:8080", + ), + ("OP_CONNECT_TOKEN", "op-token"), + ("OP_VAULT", "centaur-agent"), + ]); + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + "--session-activity-summary-enabled", + "true", + ]) + .unwrap(); + + let config = args.activity_summary_config().unwrap(); + assert_eq!(config.api_key, "sk-mounted"); + } + #[test] fn parses_session_sandbox_flags() { let args = Args::try_parse_from([ @@ -2067,6 +2476,77 @@ mod tests { assert_eq!(args.sandbox.k8s_context.as_deref(), Some("kind-test")); } + #[test] + fn execution_adoption_rescans_every_fifteen_seconds_by_default() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + ]) + .unwrap(); + + assert_eq!( + args.execution_adoption_interval(), + Some(Duration::from_secs(15)) + ); + } + + #[test] + fn execution_adoption_interval_zero_disables_rescans() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + "--session-execution-adoption-interval-secs", + "0", + ]) + .unwrap(); + + assert_eq!(args.execution_adoption_interval(), None); + } + + #[test] + fn shutdown_drain_defaults_to_twenty_seconds() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + ]) + .unwrap(); + + assert_eq!( + args.shutdown_execution_drain_timeout(), + Duration::from_secs(20) + ); + } + + #[test] + fn shutdown_drain_timeout_is_configurable() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + "--shutdown-execution-drain-timeout-secs", + "0", + ]) + .unwrap(); + + assert_eq!(args.shutdown_execution_drain_timeout(), Duration::ZERO); + } + + #[test] + fn sandbox_reaper_defaults_delete_after_max_lifetime() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + ]) + .unwrap(); + + let config = args.sandbox_reaper_config(); + assert_eq!(config.max_lifetime, Some(Duration::from_secs(259_200))); + } + #[test] fn accepts_kubernetes_aliases_for_sandbox_flags() { let args = Args::try_parse_from([ @@ -2119,7 +2599,7 @@ mod tests { } #[test] - fn agent_k8s_config_reads_overlay_image_from_flags() { + fn agent_k8s_config_reads_transitional_overlay_image_flags() { let args = Args::try_parse_from([ "centaur-api-server", "--database-url", @@ -2129,27 +2609,68 @@ mod tests { "--kubernetes-sandbox-iron-proxy-mode", "disabled", "--centaur-overlay-image", - "ghcr.io/tiplink/fineas-centaur-overlay:sha-test", + "ghcr.io/tiplink/overlay:sha-test", "--centaur-overlay-image-pull-policy", "Always", "--centaur-overlay-image-source-path", - "/overlay", - "--centaur-overlay-dir", - "/app/overlay/org", + "/org-overlay", "--centaur-sandbox-overlay-dir", - "/home/agent/overlay/org", + "/home/agent/overlay/tiplink", ]) .unwrap(); let config = AgentSandboxConfig::try_from(&args.sandbox).unwrap(); let overlay = config.overlay_image.expect("overlay image should be set"); - assert_eq!( - overlay.image, - "ghcr.io/tiplink/fineas-centaur-overlay:sha-test" - ); + assert_eq!(overlay.image, "ghcr.io/tiplink/overlay:sha-test"); assert_eq!(overlay.image_pull_policy.as_deref(), Some("Always")); - assert_eq!(overlay.source_path, "/overlay"); - assert_eq!(overlay.mount_path, "/home/agent/overlay/org"); + assert_eq!(overlay.source_path, "/org-overlay"); + assert_eq!(overlay.mount_path, "/home/agent/overlay/tiplink"); + } + + #[test] + fn sandbox_bootstrap_fingerprint_changes_with_repo_ref_and_overlay_image() { + let mut tools = ToolsConfig::new("TipLink/centaur", "centaur-agent:reviewed"); + tools.git_ref = Some("1111111111111111111111111111111111111111".to_owned()); + let overlay = OverlayImageConfig::new("fineas-overlay:sha-1111111"); + let first = sandbox_bootstrap_fingerprint( + Some(&tools), + Some(&overlay), + Some("IfNotPresent"), + &["ghcr-pull".to_owned()], + Some("release-one"), + ) + .expect("first fingerprint"); + + tools.git_ref = Some("2222222222222222222222222222222222222222".to_owned()); + let repo_changed = sandbox_bootstrap_fingerprint( + Some(&tools), + Some(&overlay), + Some("IfNotPresent"), + &["ghcr-pull".to_owned()], + Some("release-one"), + ) + .expect("repo fingerprint"); + assert_ne!(first, repo_changed); + + let overlay_changed = sandbox_bootstrap_fingerprint( + Some(&tools), + Some(&OverlayImageConfig::new("fineas-overlay:sha-2222222")), + Some("IfNotPresent"), + &["ghcr-pull".to_owned()], + Some("release-one"), + ) + .expect("overlay fingerprint"); + assert_ne!(repo_changed, overlay_changed); + + let content_changed = sandbox_bootstrap_fingerprint( + Some(&tools), + Some(&OverlayImageConfig::new("fineas-overlay:sha-2222222")), + Some("IfNotPresent"), + &["ghcr-pull".to_owned()], + Some("release-two"), + ) + .expect("content revision fingerprint"); + assert_ne!(overlay_changed, content_changed); } #[test] @@ -2170,6 +2691,8 @@ mod tests { "centaur-agent:test", "--kubernetes-tools-repo-cache-path", "/var/lib/centaur/repos", + "--kubernetes-tools-visibility", + "public", "--kubernetes-tools-github-token-secret", "centaur-repo-cache-github-token", ]) @@ -2179,16 +2702,42 @@ mod tests { assert_eq!(tools.repo, "paradigmxyz/centaur"); assert_eq!(tools.git_ref.as_deref(), Some("main")); assert_eq!(tools.source_subdir, "tools"); + assert_eq!(tools.visibility, "public"); assert_eq!(tools.image, "centaur-agent:test"); assert_eq!( tools.repo_cache_path.as_deref(), Some("/var/lib/centaur/repos") ); + assert!(tools.auto_reload); let token = tools.github_token.expect("token should be Some"); assert_eq!(token.secret_name, "centaur-repo-cache-github-token"); assert_eq!(token.secret_key, "token"); } + #[test] + fn tools_config_reads_auto_reload_flag() { + let args = Args::try_parse_from([ + "centaur-api-server", + "--database-url", + "postgres://postgres:postgres@localhost/centaur", + "--session-sandbox-backend", + "agent-k8s", + "--kubernetes-sandbox-iron-proxy-mode", + "disabled", + "--kubernetes-tools-repo", + "paradigmxyz/centaur", + "--kubernetes-tools-runner-image", + "centaur-agent:test", + "--kubernetes-tools-auto-reload", + "false", + ]) + .unwrap(); + + let config = AgentSandboxConfig::try_from(&args.sandbox).unwrap(); + let tools = config.tools.expect("tools should be Some"); + assert!(!tools.auto_reload); + } + #[test] fn agent_k8s_workflow_dirs_fan_out_across_extra_sources() { let args = Args::try_parse_from([ @@ -2368,6 +2917,20 @@ mod tests { .map(|env| env.value.as_str()), Some("true") ); + assert_eq!( + spec.env + .iter() + .find(|env| env.name == GITHUB_TOKEN_ENV) + .map(|env| env.value.as_str()), + Some(GITHUB_TOKEN_ENV) + ); + assert_eq!( + spec.env + .iter() + .find(|env| env.name == SLACK_BOT_TOKEN_ENV) + .map(|env| env.value.as_str()), + Some(SLACK_BOT_TOKEN_ENV) + ); } #[test] @@ -2402,10 +2965,22 @@ mod tests { env.iter() .any(|(name, value)| name == "OPENAI_API_KEY" && value == "OPENAI_API_KEY") ); + assert!( + env.iter() + .any(|(name, value)| name == GITHUB_TOKEN_ENV && value == GITHUB_TOKEN_ENV) + ); + assert!( + env.iter() + .any(|(name, value)| name == SLACK_BOT_TOKEN_ENV && value == SLACK_BOT_TOKEN_ENV) + ); assert!( env.iter() .any(|(name, value)| name == "OPENROUTER_API_KEY" && value == "OPENROUTER_API_KEY") ); + assert!( + env.iter() + .any(|(name, value)| name == "META_AI_API_KEY" && value == "META_AI_API_KEY") + ); } #[test] @@ -2629,82 +3204,48 @@ mod tests { } #[test] - fn iron_control_registers_discovered_tool_secrets_on_infra_role() { - use centaur_iron_proxy::{Secret, SecretReplace, Transform, TransformConfig}; - + fn iron_control_infra_secret_sync_can_be_disabled() { let args = Args::try_parse_from([ "centaur-api-server", "--database-url", "postgres://postgres:postgres@localhost/centaur", - "--kubernetes-iron-proxy-harness-auth-mode", - "api_key", + "--iron-control-url", + "http://console.local", + "--iron-control-api-key", + "iak_test", + "--iron-control-sync-infra-secrets", + "false", ]) .unwrap(); - let tool_fragment = DiscoveredToolProxyFragment { - fragment: ProxyFragment { - transforms: vec![Transform { - name: "secrets".to_owned(), - config: TransformConfig { - secrets: vec![Secret { - id: Some("TOOL_API_KEY".to_owned()), - replace: Some(SecretReplace { - proxy_value: Some("TOOL_API_KEY".to_owned()), - ..Default::default() - }), - rules: vec![serde_yaml::from_str("{host: api.tool.test}").unwrap()], - ..Default::default() - }], - ..Default::default() - }, - ..Default::default() - }], - ..Default::default() - }, - tool_count: 1, - secret_count: 1, - }; - let roles = args - .sandbox - .iron_proxy - .roles_to_register(Some(&tool_fragment)) - .unwrap(); - - assert_eq!(roles.len(), 1); - assert_eq!(roles[0].0.foreign_id, "infra"); - assert!(roles[0].1.transforms.iter().any(|transform| { - transform.config.secrets.iter().any(|secret| { - secret.id.as_deref() == Some("TOOL_API_KEY") - && secret - .replace - .as_ref() - .and_then(|replace| replace.proxy_value.as_deref()) - == Some("TOOL_API_KEY") - }) - })); + assert!(!args.sandbox.iron_control_sync_infra_secrets); } #[test] - fn iron_control_infra_secret_sync_can_be_disabled() { + fn iron_proxy_upstream_deny_cidrs_are_parsed() { let args = Args::try_parse_from([ "centaur-api-server", "--database-url", "postgres://postgres:postgres@localhost/centaur", - "--iron-control-url", - "http://console.local", - "--iron-control-api-key", - "iak_test", - "--iron-control-sync-infra-secrets", - "false", + "--kubernetes-sandbox-iron-proxy-mode", + "enabled", + "--kubernetes-firewall-ca-secret-name", + "centaur-firewall-ca", + "--kubernetes-firewall-ca-key-secret-name", + "centaur-firewall-ca-key", + "--kubernetes-iron-proxy-upstream-deny-cidrs", + "127.0.0.0/8,10.42.0.0/16,10.43.0.0/16", ]) .unwrap(); - assert!(!args.sandbox.iron_control_sync_infra_secrets); - assert!( - args.sandbox - .iron_control_tool_reconciler() - .unwrap() - .is_none() + let config = args.sandbox.iron_proxy.to_config().unwrap().unwrap(); + assert_eq!( + config.upstream_deny_cidrs, + vec![ + "127.0.0.0/8".to_owned(), + "10.42.0.0/16".to_owned(), + "10.43.0.0/16".to_owned(), + ] ); } @@ -2729,7 +3270,7 @@ mod tests { panic!("expected codex app server workload"); }; - assert_eq!(harness, HarnessType::ClaudeCode); + assert_eq!(harness, HarnessType::Codex); assert!(mounts.iter().any(|mount| { mount.target_path == SANDBOX_REPOS_MOUNT_PATH && mount.read_only diff --git a/services/api-rs/crates/centaur-api-server/src/client.rs b/services/api-rs/crates/centaur-api-server/src/client.rs index c9b1d2e9b..c0d23bd10 100644 --- a/services/api-rs/crates/centaur-api-server/src/client.rs +++ b/services/api-rs/crates/centaur-api-server/src/client.rs @@ -3,7 +3,7 @@ use std::pin::Pin; use centaur_session_core::{Session, ThreadKey}; use eventsource_stream::Eventsource; use futures_util::{Stream, StreamExt}; -use reqwest::{Client as HttpClient, StatusCode}; +use reqwest::{Client as HttpClient, RequestBuilder, StatusCode}; use thiserror::Error; use crate::types::{ @@ -11,10 +11,11 @@ use crate::types::{ ExecuteSessionResponse, }; -#[derive(Clone, Debug)] +#[derive(Clone)] pub struct CentaurClient { client: HttpClient, base_url: String, + bearer_token: Option, } impl CentaurClient { @@ -26,9 +27,17 @@ impl CentaurClient { Self { client, base_url: base_url.into().trim_end_matches('/').to_owned(), + bearer_token: None, } } + pub fn with_bearer_token(mut self, token: impl Into) -> Self { + let token = token.into(); + let token = token.trim(); + self.bearer_token = (!token.is_empty()).then(|| token.to_owned()); + self + } + pub async fn create_session( &self, thread_key: &ThreadKey, @@ -71,7 +80,7 @@ impl CentaurClient { "{}/events?after_event_id={after_event_id}", self.session_url(thread_key) ); - let response = self.client.get(&events_url).send().await?; + let response = self.authorize(self.client.get(&events_url)).send().await?; let response = ensure_response_success(response).await?; let stream = response .bytes_stream() @@ -85,7 +94,10 @@ impl CentaurClient { T: serde::Serialize + ?Sized, R: serde::de::DeserializeOwned, { - let response = self.client.post(url).json(payload).send().await?; + let response = self + .authorize(self.client.post(url).json(payload)) + .send() + .await?; let response = ensure_response_success(response).await?; Ok(response.json().await?) } @@ -97,6 +109,13 @@ impl CentaurClient { urlencoding::encode(thread_key.as_str()) ) } + + fn authorize(&self, request: RequestBuilder) -> RequestBuilder { + match self.bearer_token.as_deref() { + Some(token) => request.bearer_auth(token), + None => request, + } + } } pub type SseEventStream = Pin> + Send>>; diff --git a/services/api-rs/crates/centaur-api-server/src/error.rs b/services/api-rs/crates/centaur-api-server/src/error.rs index a144442a3..0bbeb6369 100644 --- a/services/api-rs/crates/centaur-api-server/src/error.rs +++ b/services/api-rs/crates/centaur-api-server/src/error.rs @@ -17,6 +17,8 @@ pub enum ApiError { #[error("{0}")] Unauthorized(String), #[error("{0}")] + Forbidden(String), + #[error("{0}")] NotFound(String), #[error("{0}")] MethodNotAllowed(String), @@ -49,11 +51,13 @@ impl IntoResponse for ApiError { let status = match &self { Self::BadRequest(_) => StatusCode::BAD_REQUEST, Self::Unauthorized(_) => StatusCode::UNAUTHORIZED, + Self::Forbidden(_) => StatusCode::FORBIDDEN, Self::NotFound(_) => StatusCode::NOT_FOUND, Self::MethodNotAllowed(_) => StatusCode::METHOD_NOT_ALLOWED, Self::PayloadTooLarge(_) => StatusCode::PAYLOAD_TOO_LARGE, Self::ServiceUnavailable(_) => StatusCode::SERVICE_UNAVAILABLE, Self::Runtime(SessionRuntimeError::BadRequest(_)) => StatusCode::BAD_REQUEST, + Self::Runtime(SessionRuntimeError::ShuttingDown) => StatusCode::SERVICE_UNAVAILABLE, Self::Runtime(SessionRuntimeError::Store(SessionStoreError::NotFound { .. })) => { StatusCode::NOT_FOUND } diff --git a/services/api-rs/crates/centaur-api-server/src/lib.rs b/services/api-rs/crates/centaur-api-server/src/lib.rs index 0b0b3819b..52d8cebe8 100644 --- a/services/api-rs/crates/centaur-api-server/src/lib.rs +++ b/services/api-rs/crates/centaur-api-server/src/lib.rs @@ -1,6 +1,10 @@ +mod api_jwt; pub mod client; mod error; +mod mcp; mod routes; +mod slack_proxy; +mod tool_discovery; pub mod types; pub use centaur_session_runtime::{SandboxRuntime, SessionRuntime}; @@ -9,6 +13,10 @@ pub use routes::{ AppState, build_router_with_app_state, build_router_with_runtime, build_router_with_session_and_workflow_runtime, build_router_with_session_runtime, }; +pub use tool_discovery::{ + DiscoveredToolProxyFragment, ToolDiscoveryConfig, ToolDiscoveryError, + discover_persona_registry, discover_tool_proxy_fragment, +}; #[cfg(test)] mod tests { @@ -28,6 +36,8 @@ mod tests { }; use centaur_session_runtime::SandboxRuntime; use centaur_session_sqlx::PgSessionStore; + use jsonwebtoken::{Algorithm, EncodingKey, Header, encode}; + use serde_json::{Value, json}; use sqlx::PgPool; use tower::ServiceExt; @@ -105,6 +115,64 @@ mod tests { assert_eq!(response.status(), StatusCode::OK); } + #[tokio::test] + async fn healthz_decodes_slack_client_bearer_jwt_when_present() { + let app = build_router_with_app_state(AppState::unready()); + let token = encode( + &Header::new(Algorithm::HS256), + &json!({ + "iss": "centaur-console", + "sub": "principal_123", + "aud": "centaur-api", + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C987654321"], + "history_channels": ["C111111111"] + } + }), + &EncodingKey::from_secret(b"test-secret"), + ) + .unwrap(); + + let response = app + .oneshot( + Request::builder() + .uri("/healthz") + .header(header::AUTHORIZATION, format!("Bearer {token}")) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(response.status(), StatusCode::OK); + + let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); + let body: Value = serde_json::from_slice(&body).unwrap(); + assert_eq!(body.get("ok").and_then(Value::as_bool), Some(true)); + assert_eq!( + body.pointer("/slack_client_jwt/claims/sub") + .and_then(Value::as_str), + Some("principal_123") + ); + assert_eq!( + body.pointer("/slack_client_jwt/claims/slack/upload_channels/0") + .and_then(Value::as_str), + Some("C123456789") + ); + assert_eq!( + body.pointer("/slack_client_jwt/claims/slack/download_channels/0") + .and_then(Value::as_str), + Some("C987654321") + ); + assert_eq!( + body.pointer("/slack_client_jwt/claims/slack/history_channels/0") + .and_then(Value::as_str), + Some("C111111111") + ); + } + #[tokio::test] async fn readyz_reports_starting_until_runtime_is_ready() { let state = AppState::unready(); @@ -146,7 +214,7 @@ mod tests { } #[tokio::test] - async fn runtime_routes_report_unavailable_until_runtime_is_ready() { + async fn protected_runtime_routes_require_auth_before_runtime_is_ready() { for request in [ Request::builder() .method(Method::GET) @@ -213,18 +281,61 @@ mod tests { .header(header::CONTENT_TYPE, "application/json") .body(Body::from(r#"{"event_name":"test.event","payload":{}}"#)) .unwrap(), - Request::builder() - .method(Method::POST) - .uri("/api/webhooks/test") - .body(Body::empty()) - .unwrap(), ] { let app = build_router_with_app_state(AppState::unready()); let response = app.oneshot(request).await.unwrap(); - assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } } + #[tokio::test] + async fn unauthenticated_webhooks_report_unavailable_until_runtime_is_ready() { + let app = build_router_with_app_state(AppState::unready()); + let response = app + .oneshot( + Request::builder() + .method(Method::POST) + .uri("/api/webhooks/test") + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::SERVICE_UNAVAILABLE); + } + + #[tokio::test] + async fn mcp_requires_bearer_before_runtime_is_ready() { + let app = build_router_with_app_state(AppState::unready()); + + let response = app + .oneshot( + Request::builder() + .method(Method::POST) + .uri("/mcp") + .header(header::HOST, "centaur.local") + .header(header::CONTENT_TYPE, "application/json") + .body(Body::from( + r#"{"jsonrpc":"2.0","id":1,"method":"initialize","params":{}}"#, + )) + .unwrap(), + ) + .await + .unwrap(); + + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); + let challenge = response + .headers() + .get(header::WWW_AUTHENTICATE) + .and_then(|value| value.to_str().ok()) + .unwrap(); + assert!(challenge.contains("Bearer")); + assert!(challenge.contains( + "resource_metadata=\"http://centaur.local/.well-known/oauth-protected-resource/mcp\"" + )); + } + #[tokio::test] async fn append_messages_does_not_apply_a_session_body_limit() { let pool = @@ -248,7 +359,7 @@ mod tests { .unwrap(); assert_ne!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); - assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } #[tokio::test] @@ -274,11 +385,11 @@ mod tests { .unwrap(); assert_ne!(response.status(), StatusCode::PAYLOAD_TOO_LARGE); - assert_eq!(response.status(), StatusCode::UNPROCESSABLE_ENTITY); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } #[tokio::test] - async fn session_context_exposes_slack_channel_and_thread_ts() { + async fn session_context_rejects_anonymous_slack_access() { let pool = PgPool::connect_lazy("postgres://postgres:postgres@localhost/centaur_test").unwrap(); let app = build_router_with_runtime( @@ -296,16 +407,11 @@ mod tests { .await .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - let body: serde_json::Value = serde_json::from_slice(&body).unwrap(); - assert_eq!(body["thread_key"], "slack:C123:123.456"); - assert_eq!(body["slack"]["channel_id"], "C123"); - assert_eq!(body["slack"]["thread_ts"], "123.456"); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } #[tokio::test] - async fn session_context_omits_slack_for_non_slack_thread_key() { + async fn session_context_rejects_anonymous_non_slack_access() { let pool = PgPool::connect_lazy("postgres://postgres:postgres@localhost/centaur_test").unwrap(); let app = build_router_with_runtime( @@ -323,11 +429,7 @@ mod tests { .await .unwrap(); - assert_eq!(response.status(), StatusCode::OK); - let body = to_bytes(response.into_body(), usize::MAX).await.unwrap(); - let body: serde_json::Value = serde_json::from_slice(&body).unwrap(); - assert_eq!(body["thread_key"], "cli:test"); - assert!(body.get("slack").is_none()); + assert_eq!(response.status(), StatusCode::UNAUTHORIZED); } #[derive(Default)] diff --git a/services/api-rs/crates/centaur-api-server/src/main.rs b/services/api-rs/crates/centaur-api-server/src/main.rs index be8c1e61c..888327a38 100644 --- a/services/api-rs/crates/centaur-api-server/src/main.rs +++ b/services/api-rs/crates/centaur-api-server/src/main.rs @@ -1,5 +1,5 @@ +mod activity_summary; mod args; -mod tool_discovery; use centaur_api_server::{AppState, build_router_with_app_state}; use centaur_session_runtime::SessionRuntime; @@ -11,7 +11,7 @@ use thiserror::Error; use tokio::net::TcpListener; use tracing::info; -use args::Args; +use args::{Args, validate_service_api_key_separation}; #[tokio::main] async fn main() -> Result<(), ServerError> { @@ -19,6 +19,7 @@ async fn main() -> Result<(), ServerError> { let telemetry = init_telemetry(TelemetryConfig::from_env())?; let args = Args::parse(); + validate_service_api_key_separation()?; let listener = TcpListener::bind(args.server.bind_addr).await?; info!( bind_addr = %args.server.bind_addr, @@ -27,9 +28,25 @@ async fn main() -> Result<(), ServerError> { let app_state = AppState::unready(); let app = build_router_with_app_state(app_state.clone()); + let shutdown_state = app_state.clone(); + let drain_timeout = args.shutdown_execution_drain_timeout(); let mut server = tokio::spawn(async move { axum::serve(listener, app) - .with_graceful_shutdown(shutdown_signal()) + .with_graceful_shutdown(async move { + shutdown_signal().await; + info!("shutdown signal received; handing off in-flight executions"); + // Hand off before axum starts draining connections: open SSE + // streams can keep the server future alive until SIGKILL, and + // the lease release must not be lost to that. + if let Some(workflows) = shutdown_state.workflow_runtime() + && let Err(error) = workflows.close_workers().await + { + tracing::warn!(%error, "failed to close workflow workers during shutdown"); + } + if let Some(runtime) = shutdown_state.session_runtime() { + runtime.handoff_owned_executions(drain_timeout).await; + } + }) .await }); @@ -58,9 +75,14 @@ async fn initialize_runtime(args: Args, app_state: AppState) -> Result<(), Serve if args.server.run_migrations { store.run_migrations().await?; } + if let Some(config) = args.activity_summary_config() { + let worker = activity_summary::ActivitySummaryWorker::new(store.clone(), config)?; + tokio::spawn(worker.run()); + } let pool = store.pool().clone(); let sandbox_runtime = args.sandbox_runtime().await?; - let mut runtime = SessionRuntime::new(store.clone(), sandbox_runtime); + let mut runtime = SessionRuntime::new(store.clone(), sandbox_runtime) + .with_openai_session_title_generator_from_env(); let mut warm_pool_bootstrap_principal = None; let mut workflow_host_principal = None; if let Some(iron_control) = args.iron_control_runtime().await? { @@ -69,11 +91,11 @@ async fn initialize_runtime(args: Args, app_state: AppState) -> Result<(), Serve workflow_host_principal = Some(iron_control.workflow_host_principal); runtime = runtime.with_iron_control(iron_control.registrar); } - if let Some(reconciler) = args.iron_control_tool_reconciler()? { - info!("iron-control tool secret reconciliation enabled"); - tokio::spawn(reconciler.run()); - } runtime = runtime.with_personas(args.persona_registry()?); + let sandbox_capacity_config = args.sandbox_capacity_config(); + if let Some(config) = sandbox_capacity_config { + runtime = runtime.with_sandbox_capacity(config); + } if let Some(mut config) = args.warm_pool_config() { config.bootstrap_iron_control_principal = warm_pool_bootstrap_principal.clone(); runtime = runtime.with_warm_pool(config); @@ -82,7 +104,8 @@ async fn initialize_runtime(args: Args, app_state: AppState) -> Result<(), Serve runtime = runtime.with_sandbox_cleanup(args.sandbox_cleanup_config()); let workflow_host_sandbox = args .workflow_host_sandbox_runtime(workflow_host_principal.as_deref()) - .await?; + .await? + .map(|sandbox| sandbox.with_runtime(runtime.sandbox_runtime_handle())); let workflows = Some( WorkflowRuntime::new_with_workflow_host_sandbox( store, @@ -92,13 +115,25 @@ async fn initialize_runtime(args: Args, app_state: AppState) -> Result<(), Serve .await?, ); - // Adopt executions orphaned by the previous process (deploy/crash): - // recover finished turns from recorded sandbox output, re-attach still - // running sandboxes, and fail the rest so their threads unwedge. - let adoption_runtime = runtime.clone(); - tokio::spawn(async move { - adoption_runtime.adopt_orphaned_executions().await; - }); + // Adopt executions orphaned by another control plane process + // (deploy/crash): recover finished turns from recorded sandbox output, + // re-attach still running sandboxes, and fail the rest so their threads + // unwedge. The scan re-runs periodically because executions can be + // orphaned after startup — e.g. a rolling deploy terminates the previous + // pod mid-turn after this pod's startup scan already ran. + match args.execution_adoption_interval() { + Some(interval) => { + // Dropping a Tokio JoinHandle intentionally detaches this + // process-lifetime reconciliation loop. + drop(runtime.spawn_orphan_adoption(interval)); + } + None => { + let adoption_runtime = runtime.clone(); + tokio::spawn(async move { + adoption_runtime.adopt_orphaned_executions().await; + }); + } + } app_state.mark_ready(runtime, workflows, Some(pool)); info!("centaur api-rs runtime initialized"); @@ -109,8 +144,32 @@ fn init_crypto_provider() { let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); } +/// Resolves on SIGINT (Ctrl-C) or, on Unix, SIGTERM — the signal Kubernetes +/// sends on pod termination. The binary runs as PID 1 in its container, and +/// PID 1 ignores signals without installed handlers: without the SIGTERM arm +/// every rollout burned the full termination grace period and ended in +/// SIGKILL, never reaching the graceful shutdown path. async fn shutdown_signal() { - let _ = tokio::signal::ctrl_c().await; + #[cfg(unix)] + { + use tokio::signal::unix::{SignalKind, signal}; + let mut sigterm = match signal(SignalKind::terminate()) { + Ok(sigterm) => sigterm, + Err(error) => { + tracing::warn!(%error, "failed to install SIGTERM handler; using ctrl-c only"); + let _ = tokio::signal::ctrl_c().await; + return; + } + }; + tokio::select! { + _ = tokio::signal::ctrl_c() => {} + _ = sigterm.recv() => {} + } + } + #[cfg(not(unix))] + { + let _ = tokio::signal::ctrl_c().await; + } } #[derive(Debug, Error)] @@ -138,7 +197,9 @@ pub(crate) enum ServerError { #[error(transparent)] Telemetry(#[from] centaur_telemetry::TelemetryError), #[error(transparent)] - ToolDiscovery(#[from] tool_discovery::ToolDiscoveryError), + ToolDiscovery(#[from] centaur_api_server::ToolDiscoveryError), + #[error(transparent)] + ActivitySummary(#[from] activity_summary::ActivitySummaryError), #[error("tool source error: {0}")] ToolSource(String), #[error("iron-proxy requires both firewall CA cert and key Secret names")] diff --git a/services/api-rs/crates/centaur-api-server/src/mcp.rs b/services/api-rs/crates/centaur-api-server/src/mcp.rs new file mode 100644 index 000000000..e47af033d --- /dev/null +++ b/services/api-rs/crates/centaur-api-server/src/mcp.rs @@ -0,0 +1,1260 @@ +use std::{ + collections::BTreeMap, + env, fs, + path::PathBuf, + sync::{Mutex, OnceLock}, + time::{Duration, Instant}, +}; + +use axum::{ + Json, + extract::State, + http::{HeaderMap, HeaderValue, StatusCode}, + response::{IntoResponse, Response}, +}; +use base64::{Engine as _, engine::general_purpose}; +use centaur_session_runtime::{SessionRuntime, ToolHostCallInput}; +use hmac::{Hmac, Mac}; +use serde::Deserialize; +use serde_json::{Value, json}; +use sha2::{Digest, Sha256}; +use time::OffsetDateTime; + +use crate::{ + ApiError, + api_jwt::jwt_signing_secret, + routes::{AppState, header_value}, + tool_discovery::{DiscoveredTool, ToolDiscoveryConfig, discover_tool_catalog}, +}; + +pub(crate) async fn mcp_get() -> Response { + ( + StatusCode::METHOD_NOT_ALLOWED, + Json(json!({ + "ok": false, + "error": "MCP Streamable HTTP requests must use POST for this endpoint", + })), + ) + .into_response() +} + +pub(crate) async fn mcp_protected_resource_metadata(headers: HeaderMap) -> Json { + let authorization_servers = mcp_authorization_server_url() + .into_iter() + .collect::>(); + Json(json!({ + "resource": mcp_resource_url(&headers), + "authorization_servers": authorization_servers, + "bearer_methods_supported": ["header"], + "scopes_supported": ["mcp:tools"], + })) +} + +#[derive(Debug, Deserialize)] +pub(crate) struct McpJsonRpcRequest { + jsonrpc: Option, + #[serde(default)] + id: Option, + method: String, + #[serde(default)] + params: Value, +} + +#[derive(Debug, Deserialize)] +struct McpToolCallParams { + name: String, + #[serde(default)] + arguments: Value, +} + +#[derive(Debug, Deserialize)] +struct CentaurToolMcpArguments { + method: String, + #[serde(default)] + arguments: Value, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +struct McpPrincipal { + token_id: String, + principal_id: String, + name: String, + scopes: Vec, + expires_at: Option, +} + +pub(crate) async fn mcp_post( + State(state): State, + headers: HeaderMap, + Json(request): Json, +) -> Result { + let Some(principal) = authenticate_mcp_bearer(&headers)? else { + return Ok(mcp_unauthorized(&headers)); + }; + if request.jsonrpc.as_deref().unwrap_or("2.0") != "2.0" { + return Ok(mcp_json_error( + request.id.unwrap_or(Value::Null), + -32600, + "invalid JSON-RPC version", + )); + } + let Some(id) = request.id.clone() else { + return Ok(StatusCode::NO_CONTENT.into_response()); + }; + + let result = match request.method.as_str() { + "initialize" => json!({ + "protocolVersion": requested_mcp_protocol_version(&request.params), + "capabilities": { + "tools": { + "listChanged": false, + }, + }, + "serverInfo": { + "name": "centaur", + "version": env!("CARGO_PKG_VERSION"), + }, + }), + "ping" => json!({}), + "tools/list" => { + ensure_mcp_scope(&principal.scopes, "mcp:tools")?; + let mut tools = vec![mcp_whoami_tool()]; + tools.extend(mcp_centaur_tool_entries()?); + json!({ + "tools": tools, + }) + } + "tools/call" => { + ensure_mcp_scope(&principal.scopes, "mcp:tools")?; + let params = serde_json::from_value::(request.params.clone()) + .map_err(|error| ApiError::BadRequest(error.to_string()))?; + if params.name == "centaur_whoami" { + mcp_whoami_result(&principal, params.arguments)? + } else { + let Some(tool) = mcp_find_centaur_tool(¶ms.name)? else { + return Ok(mcp_json_error(id, -32602, "unknown tool")); + }; + mcp_centaur_tool_result(&state, &principal, tool, params.arguments).await? + } + } + _ => return Ok(mcp_json_error(id, -32601, "method not found")), + }; + + Ok(Json(json!({ + "jsonrpc": "2.0", + "id": id, + "result": result, + })) + .into_response()) +} + +fn mcp_whoami_tool() -> Value { + json!({ + "name": "centaur_whoami", + "description": "Show the authenticated Centaur MCP principal and token metadata.", + "inputSchema": { + "type": "object", + "properties": {}, + "additionalProperties": false, + }, + }) +} + +fn mcp_centaur_tool_entries() -> Result, ApiError> { + let mut entries = Vec::new(); + for tool in mcp_centaur_tool_catalog()? { + let methods = mcp_tool_methods(&tool); + let signatures = methods + .iter() + .map(|method| method.signature.as_str()) + .collect::>(); + let names = methods + .iter() + .map(|method| method.name.as_str()) + .collect::>(); + let mut description = tool + .description + .clone() + .unwrap_or_else(|| format!("Centaur tool package {}", tool.package)); + if !methods.is_empty() { + description.push_str(" Available methods: "); + description.push_str(&signatures.join(", ")); + description.push_str(". Pass keyword arguments matching the method signature; call method=help for this list."); + } + let mut method_schema = json!({ + "type": "string", + "description": "Public method on the tool client to call. Use help to list available methods.", + }); + if !methods.is_empty() { + method_schema["enum"] = json!(names); + } + entries.push(json!({ + "name": tool.name, + "description": description, + "inputSchema": { + "type": "object", + "required": ["method"], + "properties": { + "method": method_schema, + "arguments": { + "type": "object", + "description": "Keyword arguments passed to the selected method.", + "additionalProperties": true, + }, + }, + "additionalProperties": false, + }, + })); + } + Ok(entries) +} + +struct McpToolMethod { + name: String, + signature: String, +} + +fn mcp_tool_methods(tool: &DiscoveredTool) -> Vec { + let mut methods = BTreeMap::from([("help".to_owned(), "help()".to_owned())]); + let path = tool.project_dir.join(&tool.client_module); + if let Ok(contents) = fs::read_to_string(&path) { + for line in contents.lines() { + let indent = line.chars().take_while(|ch| *ch == ' ').count(); + if indent != 0 && indent != 4 { + continue; + } + let trimmed = line.trim_start(); + let definition = trimmed + .strip_prefix("def ") + .or_else(|| trimmed.strip_prefix("async def ")); + let Some(definition) = definition else { + continue; + }; + let Some((name, params)) = definition.split_once('(') else { + continue; + }; + let name = name.trim(); + if name.is_empty() || name.starts_with('_') { + continue; + } + methods.insert(name.to_owned(), mcp_method_signature(name, params)); + } + } + methods + .into_iter() + .map(|(name, signature)| McpToolMethod { name, signature }) + .collect() +} + +/// Render `name(params)` from the text after the opening paren of a `def` +/// line, dropping a leading `self`. Multi-line parameter lists fall back to +/// `name(...)`. +fn mcp_method_signature(name: &str, params: &str) -> String { + let mut depth = 1usize; + let Some(end) = params.find(|ch| { + match ch { + '(' | '[' | '{' => depth += 1, + ')' | ']' | '}' => depth -= 1, + _ => {} + } + depth == 0 + }) else { + return format!("{name}(...)"); + }; + let mut params = params[..end].trim(); + if let Some(rest) = params.strip_prefix("self") { + params = rest.trim_start().trim_start_matches(',').trim_start(); + } + format!("{name}({params})") +} + +fn mcp_tool_help_result( + tool: &DiscoveredTool, + methods: &[McpToolMethod], +) -> Result { + Ok(mcp_text_result( + serde_json::to_string_pretty(&json!({ + "tool": tool.name, + "description": tool.description, + "methods": methods + .iter() + .map(|method| method.signature.as_str()) + .collect::>(), + "usage": "Call this tool with {\"method\": \"\", \"arguments\": {}}.", + }))?, + false, + )) +} + +fn mcp_centaur_tool_catalog() -> Result, ApiError> { + // Discovery scans the tool dirs and parses package metadata on every + // call; reuse a recent result so each MCP request does not redo that + // I/O while still picking up newly synced tools quickly. Tests point + // the discovery env vars at per-case temp dirs, so they read live. + const CATALOG_TTL: Duration = Duration::from_secs(10); + static CATALOG_CACHE: Mutex)>> = Mutex::new(None); + if !cfg!(test) + && let Some((discovered_at, tools)) = CATALOG_CACHE.lock().unwrap().as_ref() + && discovered_at.elapsed() < CATALOG_TTL + { + return Ok(tools.clone()); + } + + let dirs = ToolDiscoveryConfig { + tool_dirs: env::var("TOOL_DIRS").ok(), + public_tool_dirs: env::var("KUBERNETES_PUBLIC_TOOL_DIRS").ok(), + tools_path: env::var("TOOLS_PATH").ok().map(PathBuf::from), + tools_overlay_path: env::var("TOOLS_OVERLAY_PATH").ok().map(PathBuf::from), + plugins_dir: env::var("PLUGINS_DIR").ok().map(PathBuf::from), + tools_config: env::var("TOOLS_CONFIG").ok().map(PathBuf::from), + } + .resolve_tool_dirs() + .map_err(|error| ApiError::Internal(error.to_string()))?; + let tool_allowlist = effective_sandbox_env("TOOL_ALLOWLIST"); + let tool_blocklist = effective_sandbox_env("TOOL_BLOCKLIST"); + let tools = discover_tool_catalog(&dirs, tool_allowlist.as_deref(), tool_blocklist.as_deref()) + .map_err(|error| ApiError::Internal(error.to_string()))? + .tools; + if !cfg!(test) { + *CATALOG_CACHE.lock().unwrap() = Some((Instant::now(), tools.clone())); + } + Ok(tools) +} + +fn effective_sandbox_env(name: &str) -> Option { + env::var("SESSION_SANDBOX_EXTRA_ENV") + .ok() + .and_then(|raw| sandbox_extra_env_value(&raw, name)) + .or_else(|| env::var(name).ok()) +} + +fn sandbox_extra_env_value(raw: &str, name: &str) -> Option { + let parsed = serde_json::from_str::(raw).ok()?; + let entry = parsed.as_array()?.iter().rev().find(|item| { + item.get("name") + .and_then(Value::as_str) + .is_some_and(|candidate| candidate.trim() == name) + })?; + Some(match entry.get("value") { + None | Some(Value::Null) => String::new(), + Some(Value::String(value)) => value.clone(), + Some(value) => value.to_string(), + }) +} + +fn mcp_find_centaur_tool(name: &str) -> Result, ApiError> { + Ok(mcp_centaur_tool_catalog()? + .into_iter() + .find(|tool| tool.name == name)) +} + +fn mcp_whoami_result(principal: &McpPrincipal, arguments: Value) -> Result { + if !arguments.is_null() && !arguments.as_object().is_some_and(serde_json::Map::is_empty) { + return Err(ApiError::BadRequest( + "centaur_whoami does not accept arguments".to_owned(), + )); + } + Ok(mcp_text_result( + serde_json::to_string_pretty(&json!({ + "principal_id": principal.principal_id, + "token_id": principal.token_id, + "token_name": principal.name, + "scopes": principal.scopes, + "expires_at": principal + .expires_at + .map(|value| value.format(&time::format_description::well_known::Rfc3339)) + .transpose() + .map_err(|error| ApiError::Internal(error.to_string()))?, + }))?, + false, + )) +} + +async fn mcp_centaur_tool_result( + state: &AppState, + principal: &McpPrincipal, + tool: DiscoveredTool, + arguments: Value, +) -> Result { + let params = serde_json::from_value::(arguments) + .map_err(|error| ApiError::BadRequest(error.to_string()))?; + if params.method.trim().is_empty() { + return Err(ApiError::BadRequest("method is required".to_owned())); + } + let method = params.method.trim().to_owned(); + let methods = mcp_tool_methods(&tool); + if method == "help" { + return mcp_tool_help_result(&tool, &methods); + } + if !methods.iter().any(|candidate| candidate.name == method) { + return Ok(mcp_text_result( + format!( + "centaur tool {} has no method {method}. Available methods: {}", + tool.name, + methods + .iter() + .map(|method| method.signature.as_str()) + .collect::>() + .join(", ") + ), + true, + )); + } + run_tool_host_centaur_tool( + state.runtime()?, + principal, + &tool, + &method, + params.arguments, + ) + .await +} + +async fn run_tool_host_centaur_tool( + runtime: SessionRuntime, + principal: &McpPrincipal, + tool: &DiscoveredTool, + method: &str, + arguments: Value, +) -> Result { + let output = runtime + .run_tool_host_call(ToolHostCallInput { + principal_id: principal.principal_id.clone(), + token_id: Some(principal.token_id.clone()), + tool_name: tool.name.clone(), + method: method.to_owned(), + arguments, + timeout: Duration::from_secs(120), + }) + .await?; + if output.timed_out { + return Ok(mcp_text_result( + format!( + "centaur tool {}.{method} timed out in sandbox {}: {}", + tool.name, output.sandbox_id, output.stderr + ), + true, + )); + } + if output.exit_status != Some(0) { + let raw = if output.stderr.is_empty() { + &output.stdout + } else { + &output.stderr + }; + let detail = mcp_tool_failure_detail(raw); + return Ok(mcp_text_result( + format!( + "centaur tool {}.{method} failed in sandbox {} with status {:?}: {detail}\n\nCall the {} tool with method \"help\" to list available methods and their signatures.", + tool.name, output.sandbox_id, output.exit_status, tool.name + ), + true, + )); + } + let stdout = output.stdout.trim(); + if stdout.is_empty() { + return Ok(mcp_text_result("null".to_owned(), false)); + } + match serde_json::from_str::(stdout) { + Ok(value) => Ok(mcp_text_result( + serde_json::to_string_pretty(&value)?, + false, + )), + Err(error) => Ok(mcp_text_result( + format!( + "centaur tool {}.{method} returned non-json output in sandbox {}: {error}: {stdout}", + tool.name, output.sandbox_id + ), + true, + )), + } +} + +/// Reduce a Python traceback to its final exception message: agents act on +/// the error line, not on stack frames or build noise, so keep everything +/// from the last traceback's exception message to the end. +fn mcp_tool_failure_detail(raw: &str) -> String { + let trimmed = raw.trim(); + let Some(index) = trimmed.rfind("Traceback (most recent call last):") else { + return trimmed.to_owned(); + }; + let lines = trimmed[index..].lines().collect::>(); + let message_start = lines + .iter() + .skip(1) + .position(|line| !line.is_empty() && !line.starts_with(char::is_whitespace)); + match message_start { + Some(position) => lines[position + 1..].join("\n"), + None => trimmed.to_owned(), + } +} + +fn mcp_text_result(text: String, is_error: bool) -> Value { + json!({ + "content": [ + { + "type": "text", + "text": text, + }, + ], + "isError": is_error, + }) +} + +fn authenticate_mcp_bearer(headers: &HeaderMap) -> Result, ApiError> { + let Some(token) = bearer_token(headers) else { + return Ok(None); + }; + verify_mcp_jwt(&token, headers) +} + +#[derive(Debug, Deserialize)] +struct McpJwtHeader { + alg: String, +} + +#[derive(Debug, Deserialize)] +struct McpJwtClaims { + aud: Value, + exp: i64, + #[serde(default)] + iat: Option, + iss: String, + #[serde(default)] + jti: Option, + #[serde(default)] + name: Option, + #[serde(default)] + email: Option, + #[serde(default)] + nbf: Option, + principal_id: String, + #[serde(default)] + scope: Option, + #[serde(default)] + scopes: Option>, + #[serde(default)] + sub: Option, +} + +fn verify_mcp_jwt(token: &str, headers: &HeaderMap) -> Result, ApiError> { + let secret = jwt_signing_secret() + .filter(|secret| !secret.trim().is_empty()) + .ok_or_else(|| { + ApiError::ServiceUnavailable("CENTAUR_JWT_SIGNING_SECRET is not configured".to_owned()) + })?; + + let parts = token.split('.').collect::>(); + if parts.len() != 3 { + return Ok(None); + } + let Some(header) = decode_base64url_json::(parts[0]) else { + return Ok(None); + }; + if header.alg != "HS256" { + return Ok(None); + } + + let signing_input = format!("{}.{}", parts[0], parts[1]); + let mut mac = Hmac::::new_from_slice(secret.as_bytes()).map_err(|_| { + ApiError::Internal("CENTAUR_JWT_SIGNING_SECRET is not valid HMAC key material".to_owned()) + })?; + mac.update(signing_input.as_bytes()); + let expected = mac.finalize().into_bytes(); + let Some(presented) = decode_base64url(parts[2]) else { + return Ok(None); + }; + if !constant_time_eq(&presented, expected.as_slice()) { + return Ok(None); + } + + let Some(claims) = decode_base64url_json::(parts[1]) else { + return Ok(None); + }; + let now = OffsetDateTime::now_utc().unix_timestamp(); + if claims.exp <= now { + return Ok(None); + } + if claims.nbf.is_some_and(|nbf| nbf > now + 30) { + return Ok(None); + } + if claims.iat.is_some_and(|iat| iat > now + 30) { + return Ok(None); + } + let Some(issuer) = mcp_authorization_server_url() else { + return Ok(None); + }; + if !same_url(&claims.iss, &issuer) { + return Ok(None); + } + if !audience_contains(&claims.aud, &mcp_resource_url(headers)) { + return Ok(None); + } + if claims.principal_id.trim().is_empty() { + return Ok(None); + } + + let mut scopes = claims.scopes.unwrap_or_default(); + if let Some(scope) = claims.scope { + scopes.extend(scope.split_whitespace().map(ToOwned::to_owned)); + } + scopes = normalize_scope_list(scopes); + if scopes.is_empty() { + return Ok(None); + } + let expires_at = OffsetDateTime::from_unix_timestamp(claims.exp).ok(); + let token_id = claims.jti.unwrap_or_else(|| { + let digest = Sha256::digest(token.as_bytes()); + format!("mcp_jwt_{}", hex::encode(&digest[..12])) + }); + let name = first_non_empty_owned([ + claims.name, + claims.email, + claims.sub, + Some(claims.principal_id.clone()), + ]) + .unwrap_or_else(|| claims.principal_id.clone()); + + Ok(Some(McpPrincipal { + token_id, + principal_id: claims.principal_id, + name, + scopes, + expires_at, + })) +} + +fn decode_base64url_json Deserialize<'de>>(value: &str) -> Option { + let decoded = decode_base64url(value)?; + serde_json::from_slice(&decoded).ok() +} + +fn decode_base64url(value: &str) -> Option> { + general_purpose::URL_SAFE_NO_PAD + .decode(value) + .or_else(|_| general_purpose::URL_SAFE.decode(value)) + .ok() +} + +fn normalize_scope_list(scopes: Vec) -> Vec { + let mut scopes = scopes + .into_iter() + .map(|scope| scope.trim().to_owned()) + .filter(|scope| !scope.is_empty()) + .collect::>(); + scopes.sort(); + scopes.dedup(); + scopes +} + +fn first_non_empty_owned(values: impl IntoIterator>) -> Option { + values + .into_iter() + .flatten() + .map(|value| value.trim().to_owned()) + .find(|value| !value.is_empty()) +} + +fn audience_contains(audience: &Value, resource: &str) -> bool { + match audience { + Value::String(value) => same_url(value, resource), + Value::Array(values) => values + .iter() + .filter_map(Value::as_str) + .any(|value| same_url(value, resource)), + _ => false, + } +} + +fn same_url(left: &str, right: &str) -> bool { + normalize_public_url(left) + .is_some_and(|left| normalize_public_url(right).is_some_and(|right| left == right)) +} + +fn bearer_token(headers: &HeaderMap) -> Option { + let value = header_value(headers, "Authorization")?; + let token = value + .strip_prefix("Bearer ") + .or_else(|| value.strip_prefix("bearer ")) + .unwrap_or(value.as_str()) + .trim(); + (!token.is_empty()).then(|| token.to_owned()) +} + +fn ensure_mcp_scope(scopes: &[String], required: &str) -> Result<(), ApiError> { + if scopes + .iter() + .any(|scope| scope == "*" || scope == required || scope == "mcp:*") + { + Ok(()) + } else { + Err(ApiError::Forbidden(format!( + "missing required scope {required}" + ))) + } +} + +fn requested_mcp_protocol_version(params: &Value) -> &'static str { + const DEFAULT_PROTOCOL_VERSION: &str = "2025-06-18"; + match params + .get("protocolVersion") + .and_then(Value::as_str) + .filter(|version| !version.trim().is_empty()) + { + Some("2025-11-25") => "2025-11-25", + Some("2025-06-18") => "2025-06-18", + Some("2025-03-26") => "2025-03-26", + _ => DEFAULT_PROTOCOL_VERSION, + } +} + +fn mcp_json_error(id: Value, code: i64, message: &str) -> Response { + Json(json!({ + "jsonrpc": "2.0", + "id": id, + "error": { + "code": code, + "message": message, + }, + })) + .into_response() +} + +fn mcp_unauthorized(headers: &HeaderMap) -> Response { + let metadata = format!( + "{}/.well-known/oauth-protected-resource/mcp", + mcp_public_base_url(headers) + ); + let challenge = format!(r#"Bearer resource_metadata="{metadata}", scope="mcp:tools""#); + let mut response = ( + StatusCode::UNAUTHORIZED, + Json(json!({ + "ok": false, + "error": "missing or invalid MCP bearer token", + })), + ) + .into_response(); + if let Ok(value) = HeaderValue::from_str(&challenge) { + response.headers_mut().insert("WWW-Authenticate", value); + } + response +} + +fn mcp_resource_url(headers: &HeaderMap) -> String { + if let Some(url) = mcp_public_url_env() + .as_deref() + .and_then(normalize_mcp_endpoint_url) + { + return url; + } + format!("{}/mcp", request_base_url(headers)) +} + +fn mcp_authorization_server_url() -> Option { + [console_public_url_env(), iron_control_public_url_env()] + .into_iter() + .find_map(|url| url.as_deref().and_then(normalize_public_url)) +} + +fn mcp_public_base_url(headers: &HeaderMap) -> String { + if let Some(url) = mcp_public_url_env() + .as_deref() + .and_then(normalize_public_url) + { + return url.strip_suffix("/mcp").unwrap_or(&url).to_owned(); + } + request_base_url(headers) +} + +// The variables below are static deployment configuration, so each is resolved +// once per process. Tests mutate them per-case, so cfg!(test) reads live. +fn static_env(cell: &'static OnceLock>, name: &str) -> Option { + if cfg!(test) { + return env::var(name).ok(); + } + cell.get_or_init(|| env::var(name).ok()).clone() +} + +fn mcp_public_url_env() -> Option { + static CELL: OnceLock> = OnceLock::new(); + static_env(&CELL, "CENTAUR_MCP_PUBLIC_URL") +} + +fn console_public_url_env() -> Option { + static CELL: OnceLock> = OnceLock::new(); + static_env(&CELL, "CENTAUR_CONSOLE_PUBLIC_URL") +} + +fn iron_control_public_url_env() -> Option { + static CELL: OnceLock> = OnceLock::new(); + static_env(&CELL, "IRON_CONTROL_PUBLIC_URL") +} + +fn normalize_mcp_endpoint_url(value: &str) -> Option { + let mut url = normalize_public_url(value)?; + if !url.ends_with("/mcp") { + url.push_str("/mcp"); + } + Some(url) +} + +fn normalize_public_url(value: &str) -> Option { + let trimmed = value.trim().trim_end_matches('/'); + if trimmed.is_empty() { + return None; + } + Some(trimmed.to_owned()) +} + +fn request_base_url(headers: &HeaderMap) -> String { + let proto = header_value(headers, "X-Forwarded-Proto").unwrap_or_else(|| "http".to_owned()); + let host = header_value(headers, "X-Forwarded-Host") + .or_else(|| header_value(headers, "Host")) + .unwrap_or_else(|| "127.0.0.1:8080".to_owned()); + format!("{}://{}", proto.trim(), host.trim()) +} + +/// Compare two byte strings in constant time (modulo length, which is not +/// secret here). +fn constant_time_eq(actual: &[u8], expected: &[u8]) -> bool { + use subtle::ConstantTimeEq; + + actual.ct_eq(expected).into() +} + +#[cfg(test)] +mod mcp_tests { + use std::{ + sync::Mutex, + time::{SystemTime, UNIX_EPOCH}, + }; + + use futures_util::FutureExt; + + use super::*; + + static ENV_LOCK: Mutex<()> = Mutex::new(()); + + struct EnvGuard { + saved: Vec<(&'static str, Option)>, + } + + impl EnvGuard { + fn set(vars: &[(&'static str, &'static str)]) -> Self { + let saved = vars + .iter() + .map(|(name, _)| (*name, env::var(name).ok())) + .collect(); + for (name, value) in vars { + // SAFETY: tests that mutate process env hold ENV_LOCK for the + // duration of the guard, so concurrent tests in this module + // cannot observe partial mutations. + unsafe { + env::set_var(name, value); + } + } + Self { saved } + } + } + + impl Drop for EnvGuard { + fn drop(&mut self) { + for (name, value) in self.saved.drain(..) { + // SAFETY: see EnvGuard::set; the lock outlives the guard. + unsafe { + if let Some(value) = value { + env::set_var(name, value); + } else { + env::remove_var(name); + } + } + } + } + } + + fn temp_dir(prefix: &str) -> PathBuf { + let suffix = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); + env::temp_dir().join(format!("{prefix}-{}-{suffix}", std::process::id())) + } + + #[test] + fn tool_filters_follow_effective_sandbox_extra_env() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("TOOL_ALLOWLIST", "api-only"), + ( + "SESSION_SANDBOX_EXTRA_ENV", + r#"[{"name":"TOOL_ALLOWLIST","value":"old"},{"name":"TOOL_ALLOWLIST","value":"sandbox"},{"name":"TOOL_BLOCKLIST","value":"blocked"}]"#, + ), + ]); + + assert_eq!( + effective_sandbox_env("TOOL_ALLOWLIST").as_deref(), + Some("sandbox") + ); + assert_eq!( + effective_sandbox_env("TOOL_BLOCKLIST").as_deref(), + Some("blocked") + ); + } + + fn test_tool(project_dir: PathBuf) -> DiscoveredTool { + DiscoveredTool { + name: "demo".to_owned(), + package: "demo".to_owned(), + description: Some("Demo tool".to_owned()), + client_module: "client.py".to_owned(), + project_dir, + } + } + + fn test_jwt(secret: &str, claims: Value) -> String { + let header = general_purpose::URL_SAFE_NO_PAD + .encode(serde_json::to_vec(&json!({"alg": "HS256", "typ": "JWT"})).unwrap()); + let payload = general_purpose::URL_SAFE_NO_PAD.encode(serde_json::to_vec(&claims).unwrap()); + let signing_input = format!("{header}.{payload}"); + let mut mac = Hmac::::new_from_slice(secret.as_bytes()).unwrap(); + mac.update(signing_input.as_bytes()); + let signature = general_purpose::URL_SAFE_NO_PAD.encode(mac.finalize().into_bytes()); + format!("{signing_input}.{signature}") + } + + fn mcp_auth_headers(token: &str) -> HeaderMap { + let mut headers = HeaderMap::new(); + headers.insert( + "Authorization", + HeaderValue::from_str(&format!("Bearer {token}")).unwrap(), + ); + headers + } + + #[test] + fn mcp_tool_method_names_include_public_client_methods_and_help() { + let temp = temp_dir("centaur-api-rs-mcp-methods"); + fs::create_dir_all(&temp).unwrap(); + fs::write( + temp.join("client.py"), + r#" +def search(query, limit=20): + return [] + +def _hidden(): + return None + +class DemoClient: + def list_channels(self, limit=200): + def nested_helper(): + return None + return [] + + async def search_messages(self, query): + return [] +"#, + ) + .unwrap(); + + let parsed = mcp_tool_methods(&test_tool(temp.clone())); + let methods = parsed + .iter() + .map(|method| method.name.clone()) + .collect::>(); + + assert!(methods.contains(&"help".to_owned())); + assert!(methods.contains(&"search".to_owned())); + assert!(methods.contains(&"list_channels".to_owned())); + assert!(methods.contains(&"search_messages".to_owned())); + assert!(!methods.contains(&"_hidden".to_owned())); + assert!(!methods.contains(&"nested_helper".to_owned())); + + let signatures = parsed + .into_iter() + .map(|method| method.signature) + .collect::>(); + assert!(signatures.contains(&"search(query, limit=20)".to_owned())); + assert!(signatures.contains(&"list_channels(limit=200)".to_owned())); + assert!(signatures.contains(&"search_messages(query)".to_owned())); + assert!(signatures.contains(&"help()".to_owned())); + + let _ = fs::remove_dir_all(temp); + } + + #[test] + fn mcp_tool_failure_detail_keeps_final_exception_from_chained_traceback() { + let stderr = r#"Building twitter @ file:///tools/comms/twitter +Installed 16 packages in 66ms +Traceback (most recent call last): + File "/tools/comms/twitter/client.py", line 53, in _request + response.raise_for_status() +httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.x.com/2/tweets/search/recent' + +The above exception was the direct cause of the following exception: + +Traceback (most recent call last): + File "", line 45, in + File "/tools/comms/twitter/client.py", line 229, in search_tweets + tweets, meta, includes = self._paged( +RuntimeError: X API error: 401 - { + "title": "Unauthorized", + "status": 401 +}"#; + + let detail = mcp_tool_failure_detail(stderr); + + assert!(detail.starts_with("RuntimeError: X API error: 401")); + assert!(detail.contains("\"title\": \"Unauthorized\"")); + assert!(!detail.contains("Traceback")); + assert!(!detail.contains("Installed 16 packages")); + + let plain = "invalid arguments for search_tweets(query, limit=10): got an unexpected keyword argument 'max_results'"; + assert_eq!(mcp_tool_failure_detail(plain), plain); + } + + #[tokio::test] + async fn mcp_unknown_method_returns_available_methods_without_running_tool() { + let temp = temp_dir("centaur-api-rs-mcp-unknown-method"); + fs::create_dir_all(&temp).unwrap(); + fs::write( + temp.join("client.py"), + r#" +def search(query, limit=20): + return [] +"#, + ) + .unwrap(); + + let result = mcp_centaur_tool_result( + &AppState::unready(), + &McpPrincipal { + principal_id: "mcp:test".to_owned(), + token_id: "mcp_tok_test".to_owned(), + name: "test".to_owned(), + scopes: vec!["mcp:tools".to_owned()], + expires_at: None, + }, + test_tool(temp.clone()), + json!({"method": "missing", "arguments": {}}), + ) + .await + .unwrap(); + + assert_eq!(result["isError"], true); + let text = result["content"][0]["text"].as_str().unwrap(); + assert!(text.contains("has no method missing")); + assert!(text.contains("search")); + + let _ = fs::remove_dir_all(temp); + } + + #[tokio::test] + async fn mcp_unknown_method_is_rejected_when_tool_has_no_public_methods() { + let temp = temp_dir("centaur-api-rs-mcp-no-methods"); + fs::create_dir_all(&temp).unwrap(); + fs::write(temp.join("client.py"), "def _hidden():\n return None\n").unwrap(); + + let result = mcp_centaur_tool_result( + &AppState::unready(), + &McpPrincipal { + principal_id: "mcp:test".to_owned(), + token_id: "mcp_tok_test".to_owned(), + name: "test".to_owned(), + scopes: vec!["mcp:tools".to_owned()], + expires_at: None, + }, + test_tool(temp.clone()), + json!({"method": "missing", "arguments": {}}), + ) + .await + .unwrap(); + + assert_eq!(result["isError"], true); + let text = result["content"][0]["text"].as_str().unwrap(); + assert!(text.contains("has no method missing")); + + let _ = fs::remove_dir_all(temp); + } + + #[test] + fn mcp_jwt_authenticates_console_principal() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_JWT_SIGNING_SECRET", "test-secret"), + ("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000/mcp"), + ("CENTAUR_CONSOLE_PUBLIC_URL", "http://localhost:3001"), + ]); + let token = test_jwt( + "test-secret", + json!({ + "iss": "http://localhost:3001", + "sub": "usr_test", + "aud": "http://localhost:3000/mcp", + "exp": OffsetDateTime::now_utc().unix_timestamp() + 3600, + "iat": OffsetDateTime::now_utc().unix_timestamp(), + "jti": "mcpjwt_test", + "scope": "mcp:tools", + "principal_id": "prn_test", + "email": "test@example.com", + }), + ); + + let principal = authenticate_mcp_bearer(&mcp_auth_headers(&token)) + .unwrap() + .unwrap(); + + assert_eq!(principal.token_id, "mcpjwt_test"); + assert_eq!(principal.principal_id, "prn_test"); + assert_eq!(principal.name, "test@example.com"); + assert_eq!(principal.scopes, vec!["mcp:tools"]); + assert!(principal.expires_at.is_some()); + } + + #[test] + fn mcp_jwt_rejects_wrong_audience() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_JWT_SIGNING_SECRET", "test-secret"), + ("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000/mcp"), + ("CENTAUR_CONSOLE_PUBLIC_URL", "http://localhost:3001"), + ]); + let token = test_jwt( + "test-secret", + json!({ + "iss": "http://localhost:3001", + "aud": "http://other.example/mcp", + "exp": OffsetDateTime::now_utc().unix_timestamp() + 3600, + "principal_id": "prn_test", + "scope": "mcp:tools", + }), + ); + + assert!( + authenticate_mcp_bearer(&mcp_auth_headers(&token)) + .unwrap() + .is_none() + ); + } + + #[test] + fn mcp_jwt_rejects_issued_at_in_the_future() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_JWT_SIGNING_SECRET", "test-secret"), + ("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000/mcp"), + ("CENTAUR_CONSOLE_PUBLIC_URL", "http://localhost:3001"), + ]); + let token = test_jwt( + "test-secret", + json!({ + "iss": "http://localhost:3001", + "aud": "http://localhost:3000/mcp", + "exp": OffsetDateTime::now_utc().unix_timestamp() + 3600, + "iat": OffsetDateTime::now_utc().unix_timestamp() + 600, + "principal_id": "prn_test", + "scope": "mcp:tools", + }), + ); + + assert!( + authenticate_mcp_bearer(&mcp_auth_headers(&token)) + .unwrap() + .is_none() + ); + } + + #[test] + fn mcp_jwt_rejects_internal_console_control_plane_issuer() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_JWT_SIGNING_SECRET", "test-secret"), + ("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000/mcp"), + ("CENTAUR_CONSOLE_PUBLIC_URL", ""), + ("IRON_CONTROL_PUBLIC_URL", ""), + ("CENTAUR_CONSOLE_URL", "http://centaur-console:3000"), + ("IRON_CONTROL_URL", "http://centaur-console:3000"), + ]); + let token = test_jwt( + "test-secret", + json!({ + "iss": "http://centaur-console:3000", + "aud": "http://localhost:3000/mcp", + "exp": OffsetDateTime::now_utc().unix_timestamp() + 3600, + "principal_id": "prn_test", + "scope": "mcp:tools", + }), + ); + + assert!( + authenticate_mcp_bearer(&mcp_auth_headers(&token)) + .unwrap() + .is_none() + ); + } + + #[test] + fn mcp_non_jwt_bearer_values_are_not_accepted() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[("CENTAUR_JWT_SIGNING_SECRET", "test-secret")]); + + assert!( + authenticate_mcp_bearer(&mcp_auth_headers("not-a-jwt-token")) + .unwrap() + .is_none() + ); + } + + #[test] + fn mcp_protected_resource_metadata_uses_configured_urls() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000"), + ("CENTAUR_CONSOLE_PUBLIC_URL", "http://localhost:3001"), + ]); + + let Json(metadata) = mcp_protected_resource_metadata(HeaderMap::new()) + .now_or_never() + .unwrap(); + + assert_eq!(metadata["resource"], "http://localhost:3000/mcp"); + assert_eq!( + metadata["authorization_servers"][0], + "http://localhost:3001" + ); + } + + #[test] + fn mcp_protected_resource_metadata_ignores_internal_console_control_plane_url() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[ + ("CENTAUR_CONSOLE_PUBLIC_URL", ""), + ("IRON_CONTROL_PUBLIC_URL", ""), + ("CENTAUR_CONSOLE_URL", "http://centaur-console:3000"), + ("IRON_CONTROL_URL", "http://centaur-console:3000"), + ]); + let Json(metadata) = mcp_protected_resource_metadata(HeaderMap::new()) + .now_or_never() + .unwrap(); + + assert_eq!(metadata["authorization_servers"], json!([])); + } + + #[test] + fn mcp_unauthorized_challenge_uses_public_metadata_url() { + let _lock = ENV_LOCK.lock().unwrap(); + let _env = EnvGuard::set(&[("CENTAUR_MCP_PUBLIC_URL", "http://localhost:3000/mcp")]); + + let response = mcp_unauthorized(&HeaderMap::new()); + let challenge = response + .headers() + .get("WWW-Authenticate") + .unwrap() + .to_str() + .unwrap(); + + assert!(challenge.contains( + r#"resource_metadata="http://localhost:3000/.well-known/oauth-protected-resource/mcp""# + )); + assert!(!challenge.contains("/mcp/.well-known")); + } +} diff --git a/services/api-rs/crates/centaur-api-server/src/routes.rs b/services/api-rs/crates/centaur-api-server/src/routes.rs index ece7253c4..47f5b98f2 100644 --- a/services/api-rs/crates/centaur-api-server/src/routes.rs +++ b/services/api-rs/crates/centaur-api-server/src/routes.rs @@ -1,5 +1,5 @@ use std::{ - collections::BTreeMap, + collections::{BTreeMap, BTreeSet}, convert::Infallible, convert::TryFrom, env, @@ -17,8 +17,8 @@ use aws_sdk_s3::{ use axum::{ Json, Router, body::{Body, Bytes}, - extract::{DefaultBodyLimit, MatchedPath, Path, Query, Request, State}, - http::{HeaderMap, Method, StatusCode, Uri}, + extract::{DefaultBodyLimit, FromRequestParts, MatchedPath, Path, Query, Request, State}, + http::{HeaderMap, Method, StatusCode, Uri, request::Parts}, middleware::{self, Next}, response::{ IntoResponse, Response, Sse, @@ -29,8 +29,8 @@ use axum::{ use base64::{Engine as _, engine::general_purpose}; use centaur_session_core::ThreadKey; use centaur_session_runtime::{ - ExecuteSessionInput, HarnessConflictPolicy, PersonaSummary, SandboxRuntime, SessionRuntime, - thread_trace_id, thread_trace_parent_span_id, + DrainReport, ExecuteSessionInput, HarnessConflictPolicy, PersonaSummary, SandboxRuntime, + SessionRuntime, thread_trace_id, thread_trace_parent_span_id, }; use centaur_session_sqlx::PgSessionStore; use centaur_telemetry::{ @@ -38,8 +38,9 @@ use centaur_telemetry::{ record_http_request_started, set_span_parent_trace, }; use centaur_workflows::{ - CreateWorkflowRunRequest, WebhookFilter, WorkflowRuntime, WorkflowWebhookAuth, - WorkflowWebhookSpec, WorkflowWebhookTriggerKey, + CreateWorkflowRunRequest, WebhookFilter, WorkflowRun, WorkflowRuntime, WorkflowWebhookAuth, + WorkflowWebhookSpec, WorkflowWebhookTriggerKey, decode_workflow_task_token, + workflow_task_signing_key_from_env, }; use futures_util::{Stream, StreamExt}; use hmac::{Hmac, Mac}; @@ -54,11 +55,15 @@ use uuid::Uuid; use crate::{ ApiError, + api_jwt::{bearer_jwt_from_headers, bearer_token, decode_jwt_payload, verify_console_jwt}, + mcp::{mcp_get, mcp_post, mcp_protected_resource_metadata}, + slack_proxy::slack_proxy_router, types::{ AppendMessagesRequest, AppendMessagesResponse, CreateSessionRequest, CreateSessionResponse, EmitWorkflowEventRequest, EventsQuery, ExecuteSessionRequest, ExecuteSessionResponse, - ListWorkflowRunsQuery, OnHarnessConflict, SessionContextResponse, SessionSseEvent, - SlackThreadContext, stream_error_sse, + InterruptSessionExecutionRequest, InterruptSessionExecutionResponse, ListWorkflowRunsQuery, + OnHarnessConflict, ReleaseThreadRequest, ReleaseThreadResponse, SessionContextResponse, + SessionSseEvent, SlackThreadContext, stream_error_sse, }, }; @@ -125,7 +130,22 @@ impl AppState { self.initialized().is_some() } - fn runtime(&self) -> Result { + /// The session runtime, if initialization completed. Unlike the private + /// request-path accessor this does not error while starting; the + /// shutdown path uses it to skip the execution handoff when the runtime + /// never came up. + pub fn session_runtime(&self) -> Option { + self.initialized().map(|initialized| initialized.runtime) + } + + /// The workflow runtime, if initialization completed. Shutdown and the + /// admin drain use this to stop queue claims before fencing sandboxes. + pub fn workflow_runtime(&self) -> Option { + self.initialized() + .and_then(|initialized| initialized.workflows) + } + + pub(crate) fn runtime(&self) -> Result { self.initialized() .map(|initialized| initialized.runtime) .ok_or_else(|| ApiError::ServiceUnavailable("api-rs is still starting".to_owned())) @@ -151,6 +171,16 @@ impl AppState { } const MAX_WEBHOOK_BODY_BYTES: usize = 1024 * 1024; +const SESSION_API_SERVICE_KEY_ENVS: &[&str] = &[ + "CENTAUR_CONTROL_API_KEY", + "SLACKBOT_API_KEY", + "GITHUBBOT_API_KEY", + "LINEARBOT_API_KEY", + "DISCORDBOT_API_KEY", + "TEAMSBOT_API_KEY", +]; +const WORKFLOW_API_SERVICE_KEY_ENVS: &[&str] = &["CENTAUR_CONTROL_API_KEY", "WORKFLOW_API_KEY"]; +const ADMIN_API_SERVICE_KEY_ENVS: &[&str] = &["CENTAUR_CONTROL_API_KEY"]; const REDACTED_WEBHOOK_HEADERS: &[&str] = &[ "authorization", "cookie", @@ -189,6 +219,15 @@ pub fn build_router_with_app_state(state: AppState) -> Router { .route("/readyz", get(readyz)) .route("/metrics", get(metrics)) .route("/api/personas", get(list_personas)) + .route("/mcp", post(mcp_post).get(mcp_get)) + .route( + "/.well-known/oauth-protected-resource", + get(mcp_protected_resource_metadata), + ) + .route( + "/.well-known/oauth-protected-resource/mcp", + get(mcp_protected_resource_metadata), + ) .route( "/api/session/{thread_key}", post(create_or_get_session).get(get_session_context), @@ -201,37 +240,25 @@ pub fn build_router_with_app_state(state: AppState) -> Router { "/api/session/{thread_key}/execute", post(execute_session).layer(DefaultBodyLimit::disable()), ) - .route("/api/session/{thread_key}/events", get(stream_events)) + .route( + "/api/session/{thread_key}/interrupt", + post(interrupt_session_execution), + ) .route("/api/session/{thread_key}/release", post(release_thread)) - .route("/agent/threads/{thread_key}/release", post(release_thread)) + .route("/api/session/{thread_key}/events", get(stream_events)) .route("/api/sandboxes/drain", post(drain_sandboxes)) + .merge(slack_proxy_router()) .route("/api/workflows/schedules", get(list_workflow_schedules)) - .route("/workflows/schedules", get(list_workflow_schedules)) .route( "/api/workflows/runs", post(create_workflow_run).get(list_workflow_runs), ) - .route( - "/workflows/runs", - post(create_workflow_run).get(list_workflow_runs), - ) - .route( - "/api/workflows/runs/{run_id}/checkpoints", - get(get_workflow_run_checkpoints), - ) - .route( - "/workflows/runs/{run_id}/checkpoints", - get(get_workflow_run_checkpoints), - ) .route("/api/workflows/runs/{run_id}", get(get_workflow_run)) - .route("/workflows/runs/{run_id}", get(get_workflow_run)) .route( "/api/workflows/runs/{run_id}/cancel", post(cancel_workflow_run), ) - .route("/workflows/runs/{run_id}/cancel", post(cancel_workflow_run)) .route("/api/workflows/events", post(emit_workflow_event)) - .route("/workflows/events", post(emit_workflow_event)) .route( "/api/admin/slack/archive-imports", get(list_slack_archive_imports).post(presign_slack_archive_import), @@ -329,8 +356,30 @@ pub fn build_router_with_app_state(state: AppState) -> Router { .with_state(state) } -async fn healthz() -> Json { - Json(json!({"ok": true})) +async fn healthz(headers: HeaderMap) -> Json { + let mut body = json!({"ok": true}); + if let Some(token) = bearer_jwt_from_headers(&headers) { + body["slack_client_jwt"] = match decode_jwt_payload(token) { + Ok(claims) => { + let mut jwt = json!({ "claims": claims }); + match verify_console_jwt::(token) { + Ok(_) => { + jwt["valid"] = json!(true); + } + Err(error) => { + jwt["valid"] = json!(false); + jwt["error"] = json!(error.to_string()); + } + } + jwt + } + Err(error) => json!({ + "valid": false, + "error": error, + }), + }; + } + Json(body) } async fn readyz(State(state): State) -> impl IntoResponse { @@ -394,24 +443,44 @@ fn session_thread_key_from_path(path: &str) -> Option { async fn create_or_get_session( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, Json(request): Json, ) -> Result, ApiError> { + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; + ensure_session_create_authorized(&authorization, &thread_key)?; let on_harness_conflict = match request.on_harness_conflict { Some(OnHarnessConflict::Restart) => HarnessConflictPolicy::Restart, Some(OnHarnessConflict::Reject) | None => HarnessConflictPolicy::Reject, }; - let outcome = state - .runtime()? - .create_or_get_session( - &thread_key, - &request.harness_type, - request.persona_id.as_deref(), - request.metadata, - on_harness_conflict, - ) - .await?; + let outcome = match &authorization { + WorkflowApiAuthorization::FeedbackImprovement(claims) => { + runtime + .create_or_get_session_for_principal( + &thread_key, + &request.harness_type, + request.persona_id.as_deref(), + request.metadata, + on_harness_conflict, + claims.principal_id().ok_or_else(|| { + ApiError::Forbidden("feedback JWT has no principal subject".to_owned()) + })?, + ) + .await? + } + WorkflowApiAuthorization::Service | WorkflowApiAuthorization::Principal(_) => { + runtime + .create_or_get_session( + &thread_key, + &request.harness_type, + request.persona_id.as_deref(), + request.metadata, + on_harness_conflict, + ) + .await? + } + }; Ok(Json(CreateSessionResponse { session: outcome.session, harness_switched: outcome.harness_switched, @@ -420,12 +489,26 @@ async fn create_or_get_session( async fn get_session_context( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, ) -> Result, ApiError> { - let _runtime = state.runtime()?; + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let title = match runtime.session_title(&thread_key).await { + Ok(title) => title, + Err(error) => { + tracing::warn!( + thread_key = %thread_key, + %error, + "failed to load optional session title" + ); + None + } + }; Ok(Json(SessionContextResponse { slack: slack_thread_context(&thread_key), + title, thread_key, })) } @@ -461,12 +544,14 @@ fn is_slack_conversation_id(value: &str) -> bool { async fn append_messages( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, Json(request): Json, ) -> Result, ApiError> { + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; - let message_ids = state - .runtime()? + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let message_ids = runtime .append_messages(&thread_key, &request.messages) .await?; Ok(Json(AppendMessagesResponse { @@ -477,12 +562,14 @@ async fn append_messages( async fn execute_session( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, Json(request): Json, ) -> Result, ApiError> { + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; - let execution = state - .runtime()? + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let execution = runtime .execute_session( &thread_key, ExecuteSessionInput { @@ -502,24 +589,54 @@ async fn execute_session( })) } +async fn interrupt_session_execution( + State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, + Path(raw_thread_key): Path, + Json(request): Json, +) -> Result, ApiError> { + let runtime = state.runtime()?; + let thread_key = ThreadKey::try_from(raw_thread_key)?; + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let reason = request + .reason + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or("Interrupted from Slack"); + let outcome = runtime + .interrupt_active_execution(&thread_key, reason) + .await?; + Ok(Json(InterruptSessionExecutionResponse { + ok: true, + interrupted: outcome.interrupted, + execution_id: outcome.execution_id, + thread_key, + })) +} + async fn release_thread( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, - Json(request): Json, -) -> Result, ApiError> { + Json(request): Json, +) -> Result, ApiError> { + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; - let outcome = state - .runtime()? + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let outcome = runtime .release_thread( &thread_key, request.release_id.as_deref(), + request.expected_sandbox_id.as_deref(), request.cancel_inflight, ) .await?; - Ok(Json(crate::types::ReleaseThreadResponse { + Ok(Json(ReleaseThreadResponse { ok: true, session: outcome.session, release_id: outcome.release_id, + expected_sandbox_id: request.expected_sandbox_id, cancel_inflight: outcome.cancel_inflight, sandbox_released: outcome.sandbox_released, sandbox_release_error: outcome.sandbox_release_error, @@ -528,29 +645,50 @@ async fn release_thread( })) } -async fn drain_sandboxes(State(state): State) -> Result, ApiError> { - let report = state.runtime()?.drain().await?; +async fn drain_sandboxes( + State(state): State, + _authorization: AdminServiceAuthorization, +) -> Result<(StatusCode, Json), ApiError> { + if let Some(workflows) = state.workflow_runtime() { + workflows.close_workers().await?; + } + let runtime = state.runtime()?; + let report = runtime.drain().await?; + Ok(drain_http_response(report)) +} + +fn drain_http_response(report: DrainReport) -> (StatusCode, Json) { + let status = if report.failed.is_empty() { + StatusCode::OK + } else { + StatusCode::SERVICE_UNAVAILABLE + }; let failed = report .failed .iter() .map(|failure| json!({ "sandbox_id": failure.sandbox_id, "error": failure.error })) .collect::>(); - Ok(Json(json!({ + ( + status, + Json(json!({ "ok": report.failed.is_empty(), "stopped_count": report.stopped.len(), "stopped": report.stopped, "failed": failed, - }))) + })), + ) } async fn stream_events( State(state): State, + SessionApiAuthorization(authorization): SessionApiAuthorization, Path(raw_thread_key): Path, Query(query): Query, ) -> Result>>, ApiError> { + let runtime = state.runtime()?; let thread_key = ThreadKey::try_from(raw_thread_key)?; - let events = state - .runtime()? + ensure_session_resource_authorized(&runtime, &thread_key, &authorization).await?; + let events = runtime .stream_events( &thread_key, query.after_event_id.unwrap_or(0), @@ -1098,6 +1236,7 @@ struct SlackArchiveUploadConfig { async fn list_slack_archive_imports( State(state): State, + _authorization: AdminServiceAuthorization, Query(query): Query, ) -> Result, ApiError> { let pool = db_pool(&state)?; @@ -1121,6 +1260,7 @@ async fn list_slack_archive_imports( async fn get_slack_archive_import( State(state): State, + _authorization: AdminServiceAuthorization, Path(import_id): Path, ) -> Result, ApiError> { let pool = db_pool(&state)?; @@ -1132,6 +1272,7 @@ async fn get_slack_archive_import( async fn presign_slack_archive_import( State(state): State, + _authorization: AdminServiceAuthorization, Json(request): Json, ) -> Result<(StatusCode, Json), ApiError> { let pool = db_pool(&state)?; @@ -1192,6 +1333,7 @@ async fn presign_slack_archive_import( async fn refresh_slack_archive_import_upload_url( State(state): State, + _authorization: AdminServiceAuthorization, Path(import_id): Path, ) -> Result<(StatusCode, Json), ApiError> { let pool = db_pool(&state)?; @@ -1223,10 +1365,12 @@ async fn refresh_slack_archive_import_upload_url( async fn create_slack_archive_import_download_url( State(state): State, + authorization: ArchiveDownloadAuthorization, Path(import_id): Path, ) -> Result, ApiError> { let pool = db_pool(&state)?; let import = load_slack_archive_import(&pool, &import_id).await?; + ensure_archive_download_authorized(&authorization, &import)?; ensure_archive_import_status( &import.status, &["uploaded", "importing", "failed"], @@ -1245,6 +1389,7 @@ async fn create_slack_archive_import_download_url( async fn delete_slack_archive_import( State(state): State, + _authorization: AdminServiceAuthorization, Path(import_id): Path, ) -> Result, ApiError> { let pool = db_pool(&state)?; @@ -1279,6 +1424,7 @@ async fn delete_slack_archive_import( async fn start_slack_archive_import( State(state): State, + _authorization: AdminServiceAuthorization, Path(import_id): Path, ) -> Result<(StatusCode, Json), ApiError> { let pool = db_pool(&state)?; @@ -1323,6 +1469,7 @@ async fn start_slack_archive_import( async fn retry_slack_archive_import( State(state): State, + _authorization: AdminServiceAuthorization, Path(import_id): Path, ) -> Result<(StatusCode, Json), ApiError> { let pool = db_pool(&state)?; @@ -1371,6 +1518,7 @@ async fn retry_slack_archive_import( async fn list_slack_dm_sync_checkpoints( State(state): State, + _authorization: AdminServiceAuthorization, Query(query): Query, ) -> Result, ApiError> { let pool = db_pool(&state)?; @@ -1397,6 +1545,7 @@ async fn list_slack_dm_sync_checkpoints( async fn ingest_slack_dm_sync_batch( State(state): State, + _authorization: AdminServiceAuthorization, Json(request): Json, ) -> Result, ApiError> { validate_slack_dm_sync_batch(&request)?; @@ -1630,6 +1779,7 @@ async fn ingest_slack_dm_sync_batch( async fn get_google_docs_sync_checkpoint( State(state): State, + _authorization: AdminServiceAuthorization, Query(query): Query, ) -> Result, ApiError> { let pool = db_pool(&state)?; @@ -1649,6 +1799,7 @@ async fn get_google_docs_sync_checkpoint( async fn ingest_google_docs_sync_batch( State(state): State, + _authorization: AdminServiceAuthorization, Json(request): Json, ) -> Result, ApiError> { validate_google_docs_sync_batch(&request)?; @@ -1913,84 +2064,484 @@ async fn ingest_google_docs_sync_batch( }))) } +#[derive(Debug, Default, Deserialize)] +struct WorkflowApiClaims { + #[serde(default)] + sub: String, + #[serde(default)] + slack: WorkflowApiSlackClaims, +} + +#[derive(Debug, Default, Deserialize)] +struct WorkflowApiSlackClaims { + #[serde(default)] + upload_channels: Vec, +} + +#[derive(Debug)] +struct AdminServiceAuthorization; + +#[derive(Debug, PartialEq, Eq)] +enum ArchiveDownloadAuthorization { + Service, + WorkflowTask { run_id: String, task_id: String }, +} + +impl FromRequestParts for AdminServiceAuthorization { + type Rejection = ApiError; + + async fn from_request_parts( + parts: &mut Parts, + _state: &AppState, + ) -> Result { + let token = bearer_token(&parts.headers)?; + if token_matches_configured_env(token, ADMIN_API_SERVICE_KEY_ENVS) { + return Ok(Self); + } + Err(ApiError::Unauthorized( + "invalid admin service token".to_owned(), + )) + } +} + +impl FromRequestParts for ArchiveDownloadAuthorization { + type Rejection = ApiError; + + async fn from_request_parts( + parts: &mut Parts, + _state: &AppState, + ) -> Result { + let service_authorized = bearer_token(&parts.headers) + .ok() + .is_some_and(|token| token_matches_configured_env(token, ADMIN_API_SERVICE_KEY_ENVS)); + if service_authorized { + return authorize_archive_download_headers( + &parts.headers, + None, + OffsetDateTime::now_utc().unix_timestamp(), + true, + ); + } + let signing_key = workflow_task_signing_key_from_env() + .map_err(|error| ApiError::Internal(error.to_string()))?; + authorize_archive_download_headers( + &parts.headers, + Some(&signing_key), + OffsetDateTime::now_utc().unix_timestamp(), + false, + ) + } +} + +fn authorize_archive_download_headers( + headers: &HeaderMap, + signing_key: Option<&[u8]>, + now_unix: i64, + service_authorized: bool, +) -> Result { + if service_authorized { + return Ok(ArchiveDownloadAuthorization::Service); + } + + let token = header_value(headers, "X-Centaur-Workflow-Task-Token") + .map(|value| value.trim().to_owned()) + .filter(|value| !value.is_empty()) + .ok_or_else(|| { + ApiError::Unauthorized( + "archive download requires signed workflow-task authorization".to_owned(), + ) + })?; + let signing_key = signing_key.ok_or_else(|| { + ApiError::Internal("workflow task signing key is not configured".to_owned()) + })?; + let identity = decode_workflow_task_token(signing_key, &token, now_unix).ok_or_else(|| { + ApiError::Unauthorized("invalid or expired workflow-task authorization".to_owned()) + })?; + Ok(ArchiveDownloadAuthorization::WorkflowTask { + run_id: identity.run_id, + task_id: identity.task_id, + }) +} + +fn ensure_archive_download_authorized( + authorization: &ArchiveDownloadAuthorization, + import: &SlackArchiveImportRow, +) -> Result<(), ApiError> { + match authorization { + ArchiveDownloadAuthorization::Service => Ok(()), + ArchiveDownloadAuthorization::WorkflowTask { run_id, task_id } + if import.workflow_run_id.as_deref() == Some(run_id.as_str()) + && import.workflow_task_id.as_deref() == Some(task_id.as_str()) => + { + Ok(()) + } + ArchiveDownloadAuthorization::WorkflowTask { .. } => Err(ApiError::Forbidden( + "workflow task is not authorized for this archive import".to_owned(), + )), + } +} + +#[derive(Debug)] +enum WorkflowApiAuthorization { + Service, + FeedbackImprovement(WorkflowApiClaims), + Principal(WorkflowApiClaims), +} + +impl FromRequestParts for WorkflowApiAuthorization { + type Rejection = ApiError; + + async fn from_request_parts( + parts: &mut Parts, + _state: &AppState, + ) -> Result { + authorize_workflow_api(&parts.headers) + } +} + +#[derive(Debug)] +struct SessionApiAuthorization(WorkflowApiAuthorization); + +impl WorkflowApiClaims { + fn allows_channel(&self, channel_id: &str) -> bool { + self.slack + .upload_channels + .iter() + .any(|allowed| allowed == channel_id) + } + + fn has_channels(&self) -> bool { + !self.slack.upload_channels.is_empty() + } + + fn principal_id(&self) -> Option<&str> { + let sub = self.sub.trim(); + (!sub.is_empty()).then_some(sub) + } +} + +fn configured_workflow_api_names(env_name: &str) -> BTreeSet { + env::var(env_name) + .unwrap_or_default() + .split(|character: char| character == ',' || character.is_ascii_whitespace()) + .map(str::trim) + .filter(|name| !name.is_empty()) + .map(ToOwned::to_owned) + .collect() +} + +fn ensure_workflow_api_name_allowed( + workflow_name: &str, + allowed_names: &BTreeSet, +) -> Result<(), ApiError> { + let workflow_name = workflow_name.trim(); + if !workflow_name.is_empty() + && (allowed_names.contains("*") || allowed_names.contains(workflow_name)) + { + return Ok(()); + } + Err(ApiError::Forbidden( + "workflow is not allowed through the sandbox workflow API".to_owned(), + )) +} + +fn authorize_workflow_api(headers: &HeaderMap) -> Result { + let token = bearer_token(headers)?; + if token_matches_configured_env(token, WORKFLOW_API_SERVICE_KEY_ENVS) { + return Ok(WorkflowApiAuthorization::Service); + } + let claims: WorkflowApiClaims = verify_console_jwt(token)?; + if !claims.has_channels() { + return Err(ApiError::Forbidden( + "JWT has no Slack upload channel permissions".to_owned(), + )); + } + Ok(WorkflowApiAuthorization::Principal(claims)) +} + +fn authorize_session_api(headers: &HeaderMap) -> Result { + if let Some(presented) = header_value(headers, "X-Centaur-Feedback-Key") { + if !token_matches_configured_env(presented.trim(), &["SLACK_FEEDBACK_API_KEY"]) { + return Err(ApiError::Unauthorized( + "invalid feedback service token".to_owned(), + )); + } + let claims: WorkflowApiClaims = verify_console_jwt(bearer_token(headers)?)?; + if claims.principal_id().is_none() { + return Err(ApiError::Forbidden( + "feedback JWT has no principal subject".to_owned(), + )); + } + return Ok(WorkflowApiAuthorization::FeedbackImprovement(claims)); + } + + let slackbot_key = env::var("SLACKBOT_API_KEY").unwrap_or_default(); + let slackbot_key = slackbot_key.trim(); + if !slackbot_key.is_empty() + && header_value(headers, "X-Api-Key").is_some_and(|presented| { + constant_time_eq(presented.trim().as_bytes(), slackbot_key.as_bytes()) + }) + { + return Ok(WorkflowApiAuthorization::Service); + } + + let token = bearer_token(headers)?; + if token_matches_configured_env(token, SESSION_API_SERVICE_KEY_ENVS) { + return Ok(WorkflowApiAuthorization::Service); + } + let claims: WorkflowApiClaims = verify_console_jwt(token)?; + if claims.principal_id().is_none() { + return Err(ApiError::Forbidden( + "JWT has no principal subject".to_owned(), + )); + } + Ok(WorkflowApiAuthorization::Principal(claims)) +} + +fn token_matches_configured_env(token: &str, env_names: &[&str]) -> bool { + env_names.iter().any(|env_name| { + env::var(env_name).is_ok_and(|expected| { + let expected = expected.trim(); + !expected.is_empty() && constant_time_eq(token.as_bytes(), expected.as_bytes()) + }) + }) +} + +impl FromRequestParts for SessionApiAuthorization { + type Rejection = ApiError; + + async fn from_request_parts( + parts: &mut Parts, + _state: &AppState, + ) -> Result { + authorize_session_api(&parts.headers).map(Self) + } +} + +fn ensure_session_create_authorized( + authorization: &WorkflowApiAuthorization, + thread_key: &ThreadKey, +) -> Result<(), ApiError> { + match authorization { + WorkflowApiAuthorization::Service => Ok(()), + WorkflowApiAuthorization::FeedbackImprovement(_) + if thread_key.as_str().starts_with("feedback-improvement:") => + { + Ok(()) + } + WorkflowApiAuthorization::FeedbackImprovement(_) => Err(ApiError::Forbidden( + "feedback key is restricted to feedback-improvement sessions".to_owned(), + )), + WorkflowApiAuthorization::Principal(_) => Err(ApiError::Forbidden( + "session creation requires trusted service authorization".to_owned(), + )), + } +} + +async fn ensure_session_resource_authorized( + runtime: &SessionRuntime, + thread_key: &ThreadKey, + authorization: &WorkflowApiAuthorization, +) -> Result<(), ApiError> { + let claims = match authorization { + WorkflowApiAuthorization::Service => return Ok(()), + WorkflowApiAuthorization::FeedbackImprovement(claims) + if thread_key.as_str().starts_with("feedback-improvement:") => + { + let session = runtime.get_session(thread_key).await?; + if session.iron_control_principal.as_deref() == claims.principal_id() { + return Ok(()); + } + return Err(ApiError::Forbidden( + "feedback JWT is not authorized for this improvement session".to_owned(), + )); + } + WorkflowApiAuthorization::FeedbackImprovement(_) => { + return Err(ApiError::Forbidden( + "feedback key is restricted to feedback-improvement sessions".to_owned(), + )); + } + WorkflowApiAuthorization::Principal(claims) => claims, + }; + let session = runtime.get_session(thread_key).await?; + if claims_owns_session(claims, session.iron_control_principal.as_deref()) { + return Ok(()); + } + Err(ApiError::Forbidden( + "JWT is not authorized for this session".to_owned(), + )) +} + +fn claims_owns_session(claims: &WorkflowApiClaims, bound_principal: Option<&str>) -> bool { + claims.principal_id().is_some_and(|principal_id| { + bound_principal + .is_some_and(|bound| constant_time_eq(principal_id.as_bytes(), bound.as_bytes())) + }) +} + +fn workflow_input_thread_context(input: &Value) -> Result { + let raw_thread_key = input + .get("thread_key") + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .ok_or_else(|| ApiError::BadRequest("workflow input.thread_key is required".to_owned()))?; + let thread_key = ThreadKey::try_from(raw_thread_key.to_owned())?; + let context = slack_thread_context(&thread_key).ok_or_else(|| { + ApiError::BadRequest("workflow input.thread_key must identify a Slack thread".to_owned()) + })?; + if let Some(input_channel) = input + .get("channel") + .and_then(Value::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + && input_channel != context.channel_id + { + return Err(ApiError::BadRequest( + "workflow input.channel must match input.thread_key".to_owned(), + )); + } + Ok(context) +} + +fn ensure_workflow_input_authorized( + authorization: &WorkflowApiAuthorization, + input: &Value, +) -> Result, ApiError> { + let WorkflowApiAuthorization::Principal(claims) = authorization else { + return Ok(None); + }; + let context = workflow_input_thread_context(input)?; + if !claims.allows_channel(&context.channel_id) { + return Err(ApiError::Forbidden( + "JWT is not authorized for the workflow Slack channel".to_owned(), + )); + } + Ok(Some(context)) +} + +fn ensure_workflow_run_authorized( + authorization: &WorkflowApiAuthorization, + run: &WorkflowRun, + allowed_names: &BTreeSet, +) -> Result<(), ApiError> { + if matches!(authorization, WorkflowApiAuthorization::Principal(_)) { + ensure_workflow_api_name_allowed(&run.workflow_name, allowed_names)?; + } + ensure_workflow_input_authorized(authorization, &run.input)?; + Ok(()) +} + +fn ensure_workflow_service_authorized( + authorization: &WorkflowApiAuthorization, + operation: &str, +) -> Result<(), ApiError> { + if matches!(authorization, WorkflowApiAuthorization::Service) { + return Ok(()); + } + Err(ApiError::Forbidden(format!( + "workflow {operation} requires trusted service authorization" + ))) +} + async fn create_workflow_run( State(state): State, + authorization: WorkflowApiAuthorization, Json(request): Json, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; + if matches!(authorization, WorkflowApiAuthorization::Principal(_)) { + ensure_workflow_api_name_allowed( + &request.workflow_name, + &configured_workflow_api_names("WORKFLOW_API_ALLOWED_NAMES"), + )?; + } + ensure_workflow_input_authorized(&authorization, &request.input)?; let run = workflows.create_run(request).await?; Ok(Json(serde_json::to_value(run)?)) } async fn list_workflow_runs( State(state): State, + authorization: WorkflowApiAuthorization, Query(query): Query, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; + let workflow_name = query + .workflow_name + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()); + let thread_key = query + .thread_key + .as_deref() + .map(str::trim) + .filter(|value| !value.is_empty()); + if matches!(authorization, WorkflowApiAuthorization::Principal(_)) { + let workflow_name = workflow_name + .ok_or_else(|| ApiError::BadRequest("workflow_name query is required".to_owned()))?; + let thread_key = thread_key + .ok_or_else(|| ApiError::BadRequest("thread_key query is required".to_owned()))?; + ensure_workflow_api_name_allowed( + workflow_name, + &configured_workflow_api_names("WORKFLOW_API_ALLOWED_NAMES"), + )?; + ensure_workflow_input_authorized(&authorization, &json!({ "thread_key": thread_key }))?; + } let runs = workflows - .list_runs_filtered( - query.limit.unwrap_or(50), - query.workflow_name.as_deref(), - query.thread_key.as_deref(), - query.status.as_deref(), - query.parent_run_id.as_deref(), - ) + .list_runs(query.limit.unwrap_or(50), workflow_name, thread_key) .await?; - Ok(Json( - json!({ "ok": true, "runs": runs.clone(), "items": runs }), - )) + Ok(Json(json!({ "ok": true, "runs": runs }))) } async fn list_workflow_schedules( State(state): State, + authorization: WorkflowApiAuthorization, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; + ensure_workflow_service_authorized(&authorization, "schedules")?; let schedules = workflows.list_schedules(); Ok(Json(json!({ "ok": true, "schedules": schedules }))) } async fn get_workflow_run( State(state): State, + authorization: WorkflowApiAuthorization, Path(run_id): Path, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; let run = workflows.get_run(&run_id).await?; - Ok(Json(workflow_run_response(run)?)) -} - -async fn get_workflow_run_checkpoints( - State(state): State, - Path(run_id): Path, -) -> Result, ApiError> { - let workflows = workflow_runtime(&state)?; - let checkpoints = workflows.get_run_checkpoints(&run_id).await?; - Ok(Json(json!({ "ok": true, "checkpoints": checkpoints }))) + ensure_workflow_run_authorized( + &authorization, + &run, + &configured_workflow_api_names("WORKFLOW_API_ALLOWED_NAMES"), + )?; + Ok(Json(json!({ "ok": true, "run": run }))) } async fn cancel_workflow_run( State(state): State, + authorization: WorkflowApiAuthorization, Path(run_id): Path, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; + let run = workflows.get_run(&run_id).await?; + ensure_workflow_run_authorized( + &authorization, + &run, + &configured_workflow_api_names("WORKFLOW_API_ALLOWED_NAMES"), + )?; workflows.cancel_run(&run_id).await?; Ok(Json(json!({ "ok": true, "status": "cancelled" }))) } -fn workflow_run_response( - run: centaur_workflows::WorkflowRun, -) -> Result { - let run_value = serde_json::to_value(&run)?; - let mut response = run_value.as_object().cloned().unwrap_or_default(); - response.insert("ok".to_owned(), json!(true)); - response.insert("run".to_owned(), run_value); - Ok(serde_json::Value::Object(response)) -} - async fn emit_workflow_event( State(state): State, + authorization: WorkflowApiAuthorization, Json(request): Json, ) -> Result, ApiError> { let workflows = workflow_runtime(&state)?; + ensure_workflow_service_authorized(&authorization, "events")?; workflows .emit_event(&request.event_name, request.payload) .await?; @@ -2343,14 +2894,14 @@ fn slack_archive_upload_config() -> Result { }) } -fn non_empty_env(name: &str) -> Option { +pub(crate) fn non_empty_env(name: &str) -> Option { env::var(name) .ok() .map(|value| value.trim().to_owned()) .filter(|value| !value.is_empty()) } -fn positive_env_u64(name: &str, default: u64) -> u64 { +pub(crate) fn positive_env_u64(name: &str, default: u64) -> u64 { env::var(name) .ok() .and_then(|value| value.parse::().ok()) @@ -2940,7 +3491,7 @@ fn signature_header_name(auth: &WorkflowWebhookAuth) -> Option<&str> { } } -fn header_value(headers: &HeaderMap, name: &str) -> Option { +pub(crate) fn header_value(headers: &HeaderMap, name: &str) -> Option { headers .get(name) .and_then(|value| value.to_str().ok()) @@ -3000,6 +3551,156 @@ fn webhook_filter_matches(filter: &WebhookFilter, headers: &HeaderMap, body: &Va } } +#[cfg(test)] +mod drain_response_tests { + use centaur_session_runtime::DrainFailure; + + use super::*; + + #[test] + fn partial_drain_failure_is_a_non_success_http_response() { + let (status, Json(body)) = drain_http_response(DrainReport { + stopped: vec!["sbx-stopped".to_owned()], + failed: vec![DrainFailure { + sandbox_id: "sbx-live".to_owned(), + error: "stop failed".to_owned(), + }], + }); + + assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE); + assert_eq!(body["ok"], false); + assert_eq!(body["stopped_count"], 1); + assert_eq!(body["failed"][0]["sandbox_id"], "sbx-live"); + } + + #[test] + fn complete_drain_remains_successful() { + let (status, Json(body)) = drain_http_response(DrainReport { + stopped: vec!["sbx-stopped".to_owned()], + failed: Vec::new(), + }); + + assert_eq!(status, StatusCode::OK); + assert_eq!(body["ok"], true); + } +} + +#[cfg(test)] +mod workflow_api_tests { + use super::*; + + fn principal(channel_id: &str) -> WorkflowApiAuthorization { + WorkflowApiAuthorization::Principal(WorkflowApiClaims { + sub: "prn_test".to_owned(), + slack: WorkflowApiSlackClaims { + upload_channels: vec![channel_id.to_owned()], + }, + }) + } + + #[test] + fn workflow_api_requires_an_explicit_allowed_name() { + let allowed = BTreeSet::from(["reminder".to_owned()]); + ensure_workflow_api_name_allowed("reminder", &allowed).unwrap(); + assert!(matches!( + ensure_workflow_api_name_allowed("compliance_cdd_research", &allowed), + Err(ApiError::Forbidden(_)) + )); + assert!(matches!( + ensure_workflow_api_name_allowed("reminder", &BTreeSet::new()), + Err(ApiError::Forbidden(_)) + )); + } + + #[test] + fn workflow_api_scopes_input_to_the_jwt_slack_channel() { + let input = json!({ + "thread_key": "slack:T123:C123:1780000000.000100", + "channel": "C123" + }); + let context = ensure_workflow_input_authorized(&principal("C123"), &input) + .unwrap() + .expect("principal context"); + assert_eq!(context.channel_id, "C123"); + + assert!(matches!( + ensure_workflow_input_authorized(&principal("C999"), &input), + Err(ApiError::Forbidden(_)) + )); + } + + #[test] + fn workflow_api_rejects_missing_or_mismatched_thread_context() { + assert!(matches!( + ensure_workflow_input_authorized(&principal("C123"), &json!({})), + Err(ApiError::BadRequest(_)) + )); + assert!(matches!( + ensure_workflow_input_authorized( + &principal("C123"), + &json!({ + "thread_key": "slack:C123:1780000000.000100", + "channel": "C999" + }), + ), + Err(ApiError::BadRequest(_)) + )); + } + + #[test] + fn workflow_service_authorization_can_operate_non_slack_runs() { + assert!( + ensure_workflow_input_authorized( + &WorkflowApiAuthorization::Service, + &json!({"metadata": {"reason": "operator"}}), + ) + .unwrap() + .is_none() + ); + assert!(constant_time_eq(b"service-token", b"service-token")); + assert!(!constant_time_eq(b"service-token", b"other-token")); + } + + #[test] + fn workflow_principals_cannot_list_schedules_or_emit_global_events() { + for operation in ["schedules", "events"] { + assert!(matches!( + ensure_workflow_service_authorized(&principal("C123"), operation), + Err(ApiError::Forbidden(_)) + )); + ensure_workflow_service_authorized(&WorkflowApiAuthorization::Service, operation) + .unwrap(); + } + } + + #[test] + fn feedback_key_can_create_only_namespaced_improvement_sessions() { + let allowed = ThreadKey::parse("feedback-improvement:20260711:abcdef12").unwrap(); + let denied = ThreadKey::parse("slack:C123:1780000000.000100").unwrap(); + let feedback = WorkflowApiAuthorization::FeedbackImprovement(WorkflowApiClaims { + sub: "prn_feedback".to_owned(), + slack: WorkflowApiSlackClaims::default(), + }); + ensure_session_create_authorized(&feedback, &allowed).unwrap(); + assert!(matches!( + ensure_session_create_authorized(&feedback, &denied), + Err(ApiError::Forbidden(_)) + )); + } + + #[test] + fn channel_grant_does_not_authorize_another_principals_session() { + let claims = match principal("C123") { + WorkflowApiAuthorization::Principal(claims) => claims, + _ => unreachable!(), + }; + assert!(claims.allows_channel("C123")); + assert!(claims_owns_session(&claims, Some("prn_test"))); + assert!(!claims_owns_session(&claims, Some("prn_other"))); + assert!(!claims_owns_session(&claims, None)); + } +} + #[cfg(test)] mod slack_archive_import_tests { use super::*; @@ -3095,6 +3796,85 @@ mod slack_archive_import_tests { } } + #[test] + fn archive_download_uses_task_header_when_proxy_authorization_coexists() { + let signing_key = b"workflow-signing-key"; + let now = 1_700_000_000; + let task_token = centaur_workflows::mint_workflow_task_token( + signing_key, + "wfr_expected", + "wft_expected", + now + 300, + ) + .unwrap(); + let mut headers = HeaderMap::new(); + headers.insert( + "authorization", + "Bearer proxy-injected-principal-jwt".parse().unwrap(), + ); + headers.insert("x-centaur-workflow-task-token", task_token.parse().unwrap()); + + assert_eq!( + authorize_archive_download_headers(&headers, Some(signing_key), now, false).unwrap(), + ArchiveDownloadAuthorization::WorkflowTask { + run_id: "wfr_expected".to_owned(), + task_id: "wft_expected".to_owned(), + } + ); + + headers.remove("x-centaur-workflow-task-token"); + assert!(matches!( + authorize_archive_download_headers(&headers, Some(signing_key), now, false), + Err(ApiError::Unauthorized(_)) + )); + } + + #[test] + fn archive_download_service_authorization_precedes_task_capability() { + assert_eq!( + authorize_archive_download_headers(&HeaderMap::new(), None, 1_700_000_000, true) + .unwrap(), + ArchiveDownloadAuthorization::Service + ); + } + + #[test] + fn archive_download_capability_is_bound_to_the_exact_workflow_task() { + let mut row = archive_row("importing"); + row.workflow_run_id = Some("wfr_expected".to_owned()); + row.workflow_task_id = Some("wft_expected".to_owned()); + + ensure_archive_download_authorized( + &ArchiveDownloadAuthorization::WorkflowTask { + run_id: "wfr_expected".to_owned(), + task_id: "wft_expected".to_owned(), + }, + &row, + ) + .unwrap(); + ensure_archive_download_authorized(&ArchiveDownloadAuthorization::Service, &row).unwrap(); + assert!(matches!( + ensure_archive_download_authorized( + &ArchiveDownloadAuthorization::WorkflowTask { + run_id: "wfr_expected".to_owned(), + task_id: "wft_other".to_owned(), + }, + &row, + ), + Err(ApiError::Forbidden(_)) + )); + assert!(matches!( + ensure_archive_download_authorized( + &ArchiveDownloadAuthorization::WorkflowTask { + run_id: "wfr_other".to_owned(), + task_id: "wft_expected".to_owned(), + }, + &row, + ), + Err(ApiError::Forbidden(_)) + )); + } + #[test] fn archive_import_bucket_must_match_current_upload_config() { let import = archive_row("upload_pending"); diff --git a/services/api-rs/crates/centaur-api-server/src/slack_proxy.rs b/services/api-rs/crates/centaur-api-server/src/slack_proxy.rs new file mode 100644 index 000000000..83d6a9407 --- /dev/null +++ b/services/api-rs/crates/centaur-api-server/src/slack_proxy.rs @@ -0,0 +1,1787 @@ +use std::{collections::BTreeSet, sync::OnceLock, time::Duration}; + +use axum::{ + Json, Router, + body::Body, + extract::{DefaultBodyLimit, Path, Query}, + http::{HeaderMap, HeaderValue, header}, + response::{IntoResponse, Response}, + routing::{get, post}, +}; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; + +use crate::{ + ApiError, + api_jwt::{bearer_token, verify_console_jwt}, + routes::{AppState, non_empty_env, positive_env_u64}, +}; + +const DEFAULT_SLACK_API_URL: &str = "https://slack.com/api"; +const DEFAULT_MAX_UPLOAD_BYTES: u64 = 100 * 1024 * 1024; +const DEFAULT_SLACK_FILES_LIST_LIMIT: u16 = 100; +const MAX_SLACK_FILES_LIST_LIMIT: u16 = 200; +const HTTP_CONNECT_TIMEOUT: Duration = Duration::from_secs(10); +const HTTP_READ_TIMEOUT: Duration = Duration::from_secs(60); + +fn http_client() -> &'static reqwest::Client { + static CLIENT: OnceLock = OnceLock::new(); + CLIENT.get_or_init(|| { + reqwest::Client::builder() + .connect_timeout(HTTP_CONNECT_TIMEOUT) + .read_timeout(HTTP_READ_TIMEOUT) + .build() + .expect("reqwest client configuration is valid") + }) +} + +pub(crate) fn slack_proxy_router() -> Router { + Router::new() + .route("/api/slack/files", get(get_slack_files)) + .route( + "/api/slack/files/upload", + post(upload_slack_file).layer(DefaultBodyLimit::disable()), + ) + .route( + "/api/slack/files/{file_id}/download", + get(download_slack_file), + ) + .route("/api/slack/files/{file_id}/info", get(get_slack_file_info)) + .route("/api/slack/channels", get(get_slack_channels)) + .route( + "/api/slack/channels/{channel_id}/history", + get(get_slack_channel_history), + ) + .route( + "/api/slack/channels/{channel_id}/members", + get(get_slack_channel_members), + ) + .route( + "/api/slack/channels/{channel_id}/threads/{thread_ts}/replies", + get(get_slack_thread_replies), + ) +} + +#[derive(Debug, Deserialize)] +struct SlackFileUploadQuery { + channel_id: String, + filename: String, + #[serde(default)] + thread_ts: Option, + #[serde(default)] + title: Option, + #[serde(default)] + initial_comment: Option, + #[serde(default)] + content_type: Option, + #[serde(default)] + alt_txt: Option, + #[serde(default)] + snippet_type: Option, +} + +#[derive(Debug, Deserialize)] +struct SlackFileDownloadQuery { + channel_id: String, +} + +#[derive(Debug, Deserialize)] +struct SlackFileInfoQuery { + channel_id: String, +} + +#[derive(Debug, Deserialize)] +struct SlackFilesListQuery { + #[serde(default)] + channel_id: Option, + #[serde(default)] + limit: Option, + #[serde(default)] + page: Option, +} + +#[derive(Debug, Deserialize)] +struct SlackChannelHistoryQuery { + #[serde(default)] + latest: Option, + #[serde(default)] + oldest: Option, + #[serde(default)] + inclusive: Option, + #[serde(default)] + include_all_metadata: Option, + #[serde(default)] + limit: Option, + #[serde(default)] + cursor: Option, +} + +#[derive(Debug, Deserialize)] +struct SlackChannelMembersQuery { + #[serde(default)] + limit: Option, + #[serde(default)] + cursor: Option, +} + +#[derive(Debug, Deserialize)] +struct SlackFileProxyClaims { + slack: SlackProxyClaims, +} + +#[derive(Debug, Deserialize)] +struct SlackProxyClaims { + #[serde(default)] + upload_channels: Vec, + #[serde(default)] + download_channels: Vec, + #[serde(default)] + history_channels: Vec, +} + +#[derive(Debug, Serialize)] +struct SlackFileUploadResponse { + ok: bool, + file_id: String, + channel_id: String, + thread_ts: Option, + file: Value, +} + +#[derive(Debug, Serialize)] +struct SlackChannelsResponse { + ok: bool, + channels: Vec, + count: usize, +} + +#[derive(Debug, Serialize)] +struct SlackFilesListResponse { + ok: bool, + files: Vec, + count: usize, + page: u32, + paging: Option, + has_more: bool, +} + +#[derive(Debug, Serialize)] +struct SlackFileInfoResponse { + ok: bool, + file_id: String, + channel_id: String, + file: Value, +} + +#[derive(Debug, Serialize)] +struct SlackChannelItem { + id: String, + name: String, + purpose: String, + topic: String, + member_count: u64, + is_private: bool, + is_member: bool, + can_upload: bool, + can_download: bool, + can_read_history: bool, +} + +async fn upload_slack_file( + headers: HeaderMap, + Query(query): Query, + body: Body, +) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + ensure_upload_channel_allowed(&claims, &query.channel_id)?; + validate_slack_channel_id(&query.channel_id)?; + validate_filename(&query.filename)?; + if let Some(thread_ts) = query.thread_ts.as_deref() { + validate_slack_thread_ts(thread_ts)?; + } + if let Some(content_type) = query.content_type.as_deref() { + validate_content_type(content_type)?; + } + let config = slack_proxy_config()?; + let content_length = content_length(&headers)?; + ensure_upload_size(content_length, config.max_upload_bytes)?; + let client = http_client(); + let upload_ticket = get_upload_url( + client, + config, + &query.filename, + content_length, + query.alt_txt.as_deref(), + query.snippet_type.as_deref(), + ) + .await?; + upload_file_bytes( + client, + &upload_ticket.upload_url, + body, + content_length, + query.content_type.as_deref(), + ) + .await?; + let file = complete_upload( + client, + config, + &upload_ticket.file_id, + &query.channel_id, + query.thread_ts.as_deref(), + query.title.as_deref().unwrap_or(&query.filename), + query.initial_comment.as_deref(), + ) + .await?; + + Ok(Json(SlackFileUploadResponse { + ok: true, + file_id: upload_ticket.file_id, + channel_id: query.channel_id, + thread_ts: query.thread_ts, + file, + })) +} + +async fn download_slack_file( + headers: HeaderMap, + Path(file_id): Path, + Query(query): Query, +) -> Result { + let client = http_client(); + let (config, file) = + authorized_slack_file_info(&headers, client, &file_id, &query.channel_id).await?; + let download_url = file + .get("url_private_download") + .or_else(|| file.get("url_private")) + .and_then(Value::as_str) + .ok_or_else(|| ApiError::BadRequest("Slack file has no download URL".to_owned()))?; + + let upstream = client + .get(download_url) + .bearer_auth(&config.bot_token) + .send() + .await + .map_err(|error| ApiError::Internal(format!("Slack file download failed: {error}")))?; + if !upstream.status().is_success() { + return Err(ApiError::BadRequest(format!( + "Slack file download failed with status {}", + upstream.status().as_u16() + ))); + } + + let file_mimetype = file.get("mimetype").and_then(Value::as_str); + // Slack's file host serves login/error pages with a 200 status; without this + // check they would stream through labeled as the file's real mimetype. + let upstream_content_type = upstream + .headers() + .get(header::CONTENT_TYPE) + .and_then(|value| value.to_str().ok()); + if upstream_body_is_unexpected_html(upstream_content_type, file_mimetype) { + return Err(ApiError::Internal( + "Slack file download returned an HTML page instead of the file contents".to_owned(), + )); + } + + let upstream_content_length = upstream.headers().get(header::CONTENT_LENGTH).cloned(); + let mut response = Body::from_stream(upstream.bytes_stream()).into_response(); + let headers = response.headers_mut(); + if let Some(value) = file_mimetype.and_then(|value| value.parse().ok()) { + headers.insert(header::CONTENT_TYPE, value); + } + if let Some(value) = upstream_content_length { + headers.insert(header::CONTENT_LENGTH, value); + } + headers.insert( + header::X_CONTENT_TYPE_OPTIONS, + HeaderValue::from_static("nosniff"), + ); + let filename = file + .get("name") + .or_else(|| file.get("title")) + .and_then(Value::as_str) + .unwrap_or(&file_id); + if let Ok(value) = content_disposition_filename(filename).parse::() { + headers.insert(header::CONTENT_DISPOSITION, value); + } + Ok(response) +} + +async fn get_slack_files( + headers: HeaderMap, + Query(query): Query, +) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + validate_slack_files_list_query(&query)?; + let channel_id = query + .channel_id + .as_deref() + .expect("validate_slack_files_list_query requires channel_id"); + ensure_download_channel_allowed(&claims, channel_id)?; + let effective_page = slack_files_list_page(&query); + + let config = slack_proxy_config()?; + let client = http_client(); + let mut value = slack_files_list(client, config, channel_id, &query).await?; + let mut files = Vec::new(); + let mut seen_file_ids = BTreeSet::new(); + let paging = value.get("paging").cloned(); + let has_more = slack_files_list_has_more(&value); + let file_values = value + .get_mut("files") + .and_then(Value::as_array_mut) + .map(std::mem::take) + .unwrap_or_default(); + for file in file_values { + let Some(file_id) = file.get("id").and_then(Value::as_str).map(str::to_owned) else { + continue; + }; + if seen_file_ids.insert(file_id) { + files.push(file); + } + } + files.sort_by(|left, right| { + slack_file_created(right) + .cmp(&slack_file_created(left)) + .then_with(|| slack_file_id(left).cmp(slack_file_id(right))) + }); + + Ok(Json(SlackFilesListResponse { + ok: true, + count: files.len(), + files, + page: effective_page, + paging, + has_more, + })) +} + +async fn get_slack_file_info( + headers: HeaderMap, + Path(file_id): Path, + Query(query): Query, +) -> Result, ApiError> { + let (_, file) = + authorized_slack_file_info(&headers, http_client(), &file_id, &query.channel_id).await?; + + Ok(Json(SlackFileInfoResponse { + ok: true, + file_id, + channel_id: query.channel_id, + file, + })) +} + +async fn authorized_slack_file_info( + headers: &HeaderMap, + client: &reqwest::Client, + file_id: &str, + channel_id: &str, +) -> Result<(&'static SlackFileProxyConfig, Value), ApiError> { + let claims = authorize_slack_file_proxy(headers)?; + ensure_download_channel_allowed(&claims, channel_id)?; + validate_slack_channel_id(channel_id)?; + validate_slack_file_id(file_id)?; + + let config = slack_proxy_config()?; + let file = slack_file_info(client, config, file_id).await?; + if !slack_file_in_channel(&file, channel_id) { + return Err(ApiError::Forbidden( + "file is not shared in an allowed Slack channel".to_owned(), + )); + } + Ok((config, file)) +} + +async fn get_slack_channels(headers: HeaderMap) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + let channel_ids = slack_channel_ids_from_claims(&claims)?; + + let config = slack_proxy_config()?; + let client = http_client(); + let mut channels = Vec::with_capacity(channel_ids.len()); + for channel_id in channel_ids { + match slack_channel_info(client, config, &channel_id).await { + Ok(channel) => channels.push(slack_channel_item(&claims, &channel_id, &channel)), + Err(error) => { + tracing::warn!( + channel_id, + error = %error, + "skipping Slack channel whose metadata could not be fetched" + ); + } + } + } + channels.sort_by(|left, right| { + left.name + .to_ascii_lowercase() + .cmp(&right.name.to_ascii_lowercase()) + .then_with(|| left.id.cmp(&right.id)) + }); + + Ok(Json(SlackChannelsResponse { + ok: true, + count: channels.len(), + channels, + })) +} + +async fn get_slack_channel_history( + headers: HeaderMap, + Path(channel_id): Path, + Query(query): Query, +) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + ensure_history_channel_allowed(&claims, &channel_id)?; + validate_slack_channel_id(&channel_id)?; + validate_slack_channel_history_query(&query)?; + + let config = slack_proxy_config()?; + let value = slack_channel_history(http_client(), config, &channel_id, &query).await?; + Ok(Json(value)) +} + +async fn get_slack_channel_members( + headers: HeaderMap, + Path(channel_id): Path, + Query(query): Query, +) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + ensure_history_channel_allowed(&claims, &channel_id)?; + validate_slack_channel_id(&channel_id)?; + validate_slack_channel_members_query(&query)?; + + let config = slack_proxy_config()?; + let value = slack_channel_members(http_client(), config, &channel_id, &query).await?; + Ok(Json(value)) +} + +async fn get_slack_thread_replies( + headers: HeaderMap, + Path((channel_id, thread_ts)): Path<(String, String)>, + Query(query): Query, +) -> Result, ApiError> { + let claims = authorize_slack_file_proxy(&headers)?; + ensure_history_channel_allowed(&claims, &channel_id)?; + validate_slack_channel_id(&channel_id)?; + validate_slack_thread_ts(&thread_ts)?; + validate_slack_channel_history_query(&query)?; + + let config = slack_proxy_config()?; + let value = + slack_thread_replies(http_client(), config, &channel_id, &thread_ts, &query).await?; + Ok(Json(value)) +} + +fn upstream_body_is_unexpected_html( + upstream_content_type: Option<&str>, + file_mimetype: Option<&str>, +) -> bool { + let upstream_is_html = upstream_content_type.is_some_and(|value| { + value + .trim_start() + .to_ascii_lowercase() + .starts_with("text/html") + }); + let file_is_html = file_mimetype.is_some_and(|value| value.eq_ignore_ascii_case("text/html")); + upstream_is_html && !file_is_html +} + +// No Debug derive: bot_token must not end up in logs via {:?} formatting. +struct SlackFileProxyConfig { + api_url: String, + bot_token: String, + max_upload_bytes: u64, +} + +fn slack_proxy_config() -> Result<&'static SlackFileProxyConfig, ApiError> { + static CELL: OnceLock = OnceLock::new(); + if let Some(config) = CELL.get() { + return Ok(config); + } + let config = SlackFileProxyConfig::from_env()?; + Ok(CELL.get_or_init(|| config)) +} + +impl SlackFileProxyConfig { + fn from_env() -> Result { + let bot_token = non_empty_env("SLACK_BOT_TOKEN") + .ok_or_else(|| ApiError::Internal("SLACK_BOT_TOKEN is not configured".to_owned()))?; + Ok(Self { + api_url: non_empty_env("SLACK_API_URL") + .unwrap_or_else(|| DEFAULT_SLACK_API_URL.to_owned()) + .trim_end_matches('/') + .to_owned(), + bot_token, + max_upload_bytes: positive_env_u64( + "SLACK_FILE_PROXY_MAX_UPLOAD_BYTES", + DEFAULT_MAX_UPLOAD_BYTES, + ), + }) + } +} + +#[derive(Debug)] +struct SlackUploadTicket { + upload_url: String, + file_id: String, +} + +async fn get_upload_url( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + filename: &str, + length: u64, + alt_txt: Option<&str>, + snippet_type: Option<&str>, +) -> Result { + let form = slack_get_upload_url_form(filename, length, alt_txt, snippet_type); + let value = slack_api_post_form(client, config, "files.getUploadURLExternal", &form).await?; + Ok(SlackUploadTicket { + upload_url: required_slack_string(&value, "upload_url")?, + file_id: required_slack_string(&value, "file_id")?, + }) +} + +fn slack_get_upload_url_form( + filename: &str, + length: u64, + alt_txt: Option<&str>, + snippet_type: Option<&str>, +) -> Vec<(&'static str, String)> { + let mut form = vec![ + ("filename", filename.to_owned()), + ("length", length.to_string()), + ("alt_txt", alt_txt.unwrap_or("").to_owned()), + ("snippet_type", snippet_type.unwrap_or("").to_owned()), + ]; + form.retain(|(_, value)| !value.is_empty()); + form +} + +async fn upload_file_bytes( + client: &reqwest::Client, + upload_url: &str, + body: Body, + content_length: u64, + content_type: Option<&str>, +) -> Result<(), ApiError> { + let response = client + .post(upload_url) + .header( + header::CONTENT_TYPE, + content_type.unwrap_or("application/octet-stream"), + ) + .header(header::CONTENT_LENGTH, content_length) + .body(reqwest::Body::wrap_stream(body.into_data_stream())) + .send() + .await + .map_err(|error| ApiError::Internal(format!("Slack upload failed: {error}")))?; + if !response.status().is_success() { + return Err(ApiError::BadRequest(format!( + "Slack upload failed with status {}", + response.status().as_u16() + ))); + } + Ok(()) +} + +async fn complete_upload( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + file_id: &str, + channel_id: &str, + thread_ts: Option<&str>, + title: &str, + initial_comment: Option<&str>, +) -> Result { + let files = json!([{ "id": file_id, "title": title }]).to_string(); + let mut form = vec![ + ("files", files), + ("channel_id", channel_id.to_owned()), + ("thread_ts", thread_ts.unwrap_or("").to_owned()), + ("initial_comment", initial_comment.unwrap_or("").to_owned()), + ]; + form.retain(|(_, value)| !value.is_empty()); + let value = slack_api_post_form(client, config, "files.completeUploadExternal", &form).await?; + value + .get("files") + .and_then(Value::as_array) + .and_then(|files| files.first()) + .cloned() + .ok_or_else(|| { + ApiError::BadRequest("Slack upload response did not include file".to_owned()) + }) +} + +async fn slack_file_info( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + file_id: &str, +) -> Result { + let value = + slack_api_post_form(client, config, "files.info", &slack_file_info_form(file_id)).await?; + value.get("file").cloned().ok_or_else(|| { + ApiError::BadRequest("Slack file info response did not include file".to_owned()) + }) +} + +async fn slack_channel_info( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + channel_id: &str, +) -> Result { + let value = slack_api_post_form( + client, + config, + "conversations.info", + &slack_channel_info_form(channel_id), + ) + .await?; + value.get("channel").cloned().ok_or_else(|| { + ApiError::BadRequest("Slack channel info response did not include channel".to_owned()) + }) +} + +fn slack_channel_info_form(channel_id: &str) -> Vec<(&'static str, String)> { + vec![ + ("channel", channel_id.to_owned()), + ("include_num_members", "true".to_owned()), + ] +} + +async fn slack_channel_history( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + channel_id: &str, + query: &SlackChannelHistoryQuery, +) -> Result { + let form = slack_channel_history_form(channel_id, query); + slack_api_post_form(client, config, "conversations.history", &form).await +} + +async fn slack_thread_replies( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + channel_id: &str, + thread_ts: &str, + query: &SlackChannelHistoryQuery, +) -> Result { + let form = slack_thread_replies_form(channel_id, thread_ts, query); + slack_api_post_form(client, config, "conversations.replies", &form).await +} + +async fn slack_files_list( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + channel_id: &str, + query: &SlackFilesListQuery, +) -> Result { + let form = slack_files_list_form(channel_id, query); + slack_api_post_form(client, config, "files.list", &form).await +} + +async fn slack_channel_members( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + channel_id: &str, + query: &SlackChannelMembersQuery, +) -> Result { + let form = slack_channel_members_form(channel_id, query); + slack_api_post_form(client, config, "conversations.members", &form).await +} + +fn slack_files_list_form( + channel_id: &str, + query: &SlackFilesListQuery, +) -> Vec<(&'static str, String)> { + vec![ + ("channel", channel_id.to_owned()), + ("count", slack_files_list_limit(query).to_string()), + ("page", slack_files_list_page(query).to_string()), + ] +} + +fn slack_channel_members_form( + channel_id: &str, + query: &SlackChannelMembersQuery, +) -> Vec<(&'static str, String)> { + let mut form = vec![ + ("channel", channel_id.to_owned()), + ( + "limit", + query + .limit + .map(|value| value.to_string()) + .unwrap_or_default(), + ), + ("cursor", query.cursor.clone().unwrap_or_default()), + ]; + form.retain(|(_, value)| !value.is_empty()); + form +} + +fn slack_file_info_form(file_id: &str) -> Vec<(&'static str, String)> { + vec![("file", file_id.to_owned())] +} + +fn slack_channel_history_form( + channel_id: &str, + query: &SlackChannelHistoryQuery, +) -> Vec<(&'static str, String)> { + let mut form = vec![ + ("channel", channel_id.to_owned()), + ("latest", query.latest.clone().unwrap_or_default()), + ("oldest", query.oldest.clone().unwrap_or_default()), + ( + "inclusive", + query + .inclusive + .map(|value| value.to_string()) + .unwrap_or_default(), + ), + ( + "include_all_metadata", + query + .include_all_metadata + .map(|value| value.to_string()) + .unwrap_or_default(), + ), + ( + "limit", + query + .limit + .map(|value| value.to_string()) + .unwrap_or_default(), + ), + ("cursor", query.cursor.clone().unwrap_or_default()), + ]; + form.retain(|(_, value)| !value.is_empty()); + form +} + +fn slack_thread_replies_form( + channel_id: &str, + thread_ts: &str, + query: &SlackChannelHistoryQuery, +) -> Vec<(&'static str, String)> { + let mut form = slack_channel_history_form(channel_id, query); + form.push(("ts", thread_ts.to_owned())); + form +} + +fn slack_files_list_limit(query: &SlackFilesListQuery) -> u16 { + query.limit.unwrap_or(DEFAULT_SLACK_FILES_LIST_LIMIT) +} + +fn slack_files_list_page(query: &SlackFilesListQuery) -> u32 { + query.page.unwrap_or(1) +} + +fn slack_files_list_has_more(value: &Value) -> bool { + value + .get("paging") + .is_some_and(|paging| slack_paging_page(paging) < slack_paging_pages(paging)) +} + +fn slack_paging_page(paging: &Value) -> u64 { + paging + .get("page") + .and_then(Value::as_u64) + .unwrap_or_default() +} + +fn slack_paging_pages(paging: &Value) -> u64 { + paging + .get("pages") + .and_then(Value::as_u64) + .unwrap_or_default() +} + +async fn slack_api_post_form( + client: &reqwest::Client, + config: &SlackFileProxyConfig, + method: &str, + form: &[(&str, String)], +) -> Result { + let response = client + .post(format!("{}/{}", config.api_url, method)) + .bearer_auth(&config.bot_token) + .form(form) + .send() + .await + .map_err(|error| ApiError::Internal(format!("Slack API request failed: {error}")))?; + let status = response.status(); + let value = response + .json::() + .await + .map_err(|error| ApiError::Internal(format!("Slack API response was not JSON: {error}")))?; + if !status.is_success() || value.get("ok") != Some(&Value::Bool(true)) { + let slack_error = value + .get("error") + .and_then(Value::as_str) + .unwrap_or("unknown_error"); + return Err(ApiError::BadRequest(format!( + "Slack {method} failed: {slack_error}" + ))); + } + Ok(value) +} + +fn authorize_slack_file_proxy(headers: &HeaderMap) -> Result { + let token = bearer_token(headers)?; + verify_console_jwt(token) +} + +fn ensure_upload_channel_allowed( + claims: &SlackFileProxyClaims, + channel_id: &str, +) -> Result<(), ApiError> { + ensure_channel_allowed( + &claims.slack.upload_channels, + channel_id, + "JWT is not authorized to upload to this Slack channel", + ) +} + +fn ensure_download_channel_allowed( + claims: &SlackFileProxyClaims, + channel_id: &str, +) -> Result<(), ApiError> { + ensure_channel_allowed( + &claims.slack.download_channels, + channel_id, + "JWT is not authorized to download from this Slack channel", + ) +} + +fn ensure_history_channel_allowed( + claims: &SlackFileProxyClaims, + channel_id: &str, +) -> Result<(), ApiError> { + ensure_channel_allowed( + &claims.slack.history_channels, + channel_id, + "JWT is not authorized to read history from this Slack channel", + ) +} + +fn ensure_channel_allowed( + allowed_channels: &[String], + channel_id: &str, + message: &str, +) -> Result<(), ApiError> { + if allowed_channels.iter().any(|allowed| allowed == channel_id) { + return Ok(()); + } + Err(ApiError::Forbidden(message.to_owned())) +} + +fn slack_channel_ids_from_claims(claims: &SlackFileProxyClaims) -> Result, ApiError> { + validated_channel_ids( + claims + .slack + .upload_channels + .iter() + .chain(claims.slack.download_channels.iter()) + .chain(claims.slack.history_channels.iter()), + ) +} + +fn validated_channel_ids<'a>( + raw_channel_ids: impl IntoIterator, +) -> Result, ApiError> { + let mut channel_ids: BTreeSet = BTreeSet::new(); + for channel_id in raw_channel_ids { + validate_slack_channel_id(channel_id)?; + channel_ids.insert(channel_id.to_owned()); + } + Ok(channel_ids.into_iter().collect()) +} + +fn slack_channel_item( + claims: &SlackFileProxyClaims, + channel_id: &str, + channel: &Value, +) -> SlackChannelItem { + SlackChannelItem { + id: channel_id.to_owned(), + name: channel + .get("name") + .and_then(Value::as_str) + .filter(|name| !name.is_empty()) + .unwrap_or(channel_id) + .to_owned(), + purpose: slack_channel_text_field(channel, "purpose"), + topic: slack_channel_text_field(channel, "topic"), + member_count: channel + .get("num_members") + .and_then(Value::as_u64) + .unwrap_or_default(), + is_private: channel + .get("is_private") + .and_then(Value::as_bool) + .unwrap_or_else(|| channel_id.starts_with('G')), + is_member: channel + .get("is_member") + .and_then(Value::as_bool) + .unwrap_or_default(), + can_upload: claims + .slack + .upload_channels + .iter() + .any(|allowed| allowed == channel_id), + can_download: claims + .slack + .download_channels + .iter() + .any(|allowed| allowed == channel_id), + can_read_history: claims + .slack + .history_channels + .iter() + .any(|allowed| allowed == channel_id), + } +} + +fn slack_channel_text_field(channel: &Value, field: &str) -> String { + channel + .get(field) + .and_then(|value| value.get("value")) + .and_then(Value::as_str) + .unwrap_or_default() + .to_owned() +} + +fn slack_file_in_channel(file: &Value, channel_id: &str) -> bool { + slack_file_channel_ids(file).contains(channel_id) +} + +fn slack_file_channel_ids(file: &Value) -> BTreeSet { + let mut channels = BTreeSet::new(); + for key in ["channels", "groups", "ims"] { + if let Some(values) = file.get(key).and_then(Value::as_array) { + for value in values { + if let Some(channel) = value.as_str() { + channels.insert(channel.to_owned()); + } + } + } + } + if let Some(shares) = file.get("shares").and_then(Value::as_object) { + for share_type in shares.values().filter_map(Value::as_object) { + for (channel, _shares) in share_type { + channels.insert(channel.to_owned()); + } + } + } + channels +} + +fn slack_file_created(file: &Value) -> u64 { + file.get("created") + .and_then(Value::as_u64) + .unwrap_or_default() +} + +fn slack_file_id(file: &Value) -> &str { + file.get("id").and_then(Value::as_str).unwrap_or_default() +} + +fn required_slack_string(value: &Value, field: &str) -> Result { + value + .get(field) + .and_then(Value::as_str) + .map(str::to_owned) + .ok_or_else(|| ApiError::BadRequest(format!("Slack response missing {field}"))) +} + +fn content_length(headers: &HeaderMap) -> Result { + headers + .get(header::CONTENT_LENGTH) + .and_then(|value| value.to_str().ok()) + .and_then(|value| value.parse::().ok()) + .ok_or_else(|| ApiError::BadRequest("Content-Length header is required".to_owned())) +} + +fn ensure_upload_size(len: u64, max: u64) -> Result<(), ApiError> { + if len == 0 { + return Err(ApiError::BadRequest( + "file body must not be empty".to_owned(), + )); + } + if len > max { + return Err(ApiError::PayloadTooLarge(format!( + "file body exceeds {max} byte limit" + ))); + } + Ok(()) +} + +fn validate_slack_channel_id(channel_id: &str) -> Result<(), ApiError> { + if channel_id.len() >= 9 + && matches!(channel_id.as_bytes().first(), Some(b'C' | b'D' | b'G')) + && channel_id + .bytes() + .all(|byte| byte.is_ascii_uppercase() || byte.is_ascii_digit()) + { + return Ok(()); + } + Err(ApiError::BadRequest("invalid Slack channel ID".to_owned())) +} + +fn validate_slack_file_id(file_id: &str) -> Result<(), ApiError> { + if file_id.len() >= 9 + && file_id.starts_with('F') + && file_id + .bytes() + .all(|byte| byte.is_ascii_uppercase() || byte.is_ascii_digit()) + { + return Ok(()); + } + Err(ApiError::BadRequest("invalid Slack file ID".to_owned())) +} + +fn validate_slack_channel_history_query(query: &SlackChannelHistoryQuery) -> Result<(), ApiError> { + if let Some(latest) = query.latest.as_deref() { + validate_slack_timestamp(latest)?; + } + if let Some(oldest) = query.oldest.as_deref() { + validate_slack_timestamp(oldest)?; + } + if let Some(limit) = query.limit + && !(1..=999).contains(&limit) + { + return Err(ApiError::BadRequest( + "Slack history limit must be between 1 and 999".to_owned(), + )); + } + if let Some(cursor) = query.cursor.as_deref() { + validate_slack_cursor(cursor)?; + } + Ok(()) +} + +fn validate_slack_channel_members_query(query: &SlackChannelMembersQuery) -> Result<(), ApiError> { + if let Some(limit) = query.limit + && !(1..=1000).contains(&limit) + { + return Err(ApiError::BadRequest( + "Slack channel members limit must be between 1 and 1000".to_owned(), + )); + } + if let Some(cursor) = query.cursor.as_deref() { + validate_slack_cursor(cursor)?; + } + Ok(()) +} + +fn validate_slack_files_list_query(query: &SlackFilesListQuery) -> Result<(), ApiError> { + if let Some(limit) = query.limit + && !(1..=MAX_SLACK_FILES_LIST_LIMIT).contains(&limit) + { + return Err(ApiError::BadRequest(format!( + "Slack files.list limit must be between 1 and {MAX_SLACK_FILES_LIST_LIMIT}" + ))); + } + let Some(channel_id) = query.channel_id.as_deref() else { + return Err(ApiError::BadRequest( + "Slack files.list channel_id is required".to_owned(), + )); + }; + validate_slack_channel_id(channel_id)?; + if let Some(page) = query.page + && page == 0 + { + return Err(ApiError::BadRequest( + "Slack files.list page must be greater than 0".to_owned(), + )); + } + Ok(()) +} + +fn validate_slack_thread_ts(thread_ts: &str) -> Result<(), ApiError> { + let Some((seconds, micros)) = thread_ts.split_once('.') else { + return Err(ApiError::BadRequest("invalid Slack thread_ts".to_owned())); + }; + if !seconds.is_empty() + && !micros.is_empty() + && seconds.bytes().all(|byte| byte.is_ascii_digit()) + && micros.bytes().all(|byte| byte.is_ascii_digit()) + { + return Ok(()); + } + Err(ApiError::BadRequest("invalid Slack thread_ts".to_owned())) +} + +fn validate_slack_timestamp(timestamp: &str) -> Result<(), ApiError> { + if !timestamp.is_empty() + && timestamp + .split_once('.') + .map(|(seconds, micros)| { + !seconds.is_empty() + && !micros.is_empty() + && seconds.bytes().all(|byte| byte.is_ascii_digit()) + && micros.bytes().all(|byte| byte.is_ascii_digit()) + }) + .unwrap_or_else(|| timestamp.bytes().all(|byte| byte.is_ascii_digit())) + { + return Ok(()); + } + Err(ApiError::BadRequest("invalid Slack timestamp".to_owned())) +} + +fn validate_slack_cursor(cursor: &str) -> Result<(), ApiError> { + if cursor.is_empty() || cursor.len() > 4096 || cursor.chars().any(|ch| ch.is_ascii_control()) { + return Err(ApiError::BadRequest("invalid Slack cursor".to_owned())); + } + Ok(()) +} + +fn validate_filename(filename: &str) -> Result<(), ApiError> { + let filename = filename.trim(); + if filename.is_empty() || filename.contains('/') || filename.contains('\\') { + return Err(ApiError::BadRequest("invalid filename".to_owned())); + } + Ok(()) +} + +fn validate_content_type(content_type: &str) -> Result<(), ApiError> { + if content_type.trim().is_empty() || content_type.parse::().is_err() { + return Err(ApiError::BadRequest("invalid content_type".to_owned())); + } + Ok(()) +} + +fn content_disposition_filename(filename: &str) -> String { + let sanitized = filename + .chars() + .map(|ch| { + if ch.is_ascii_alphanumeric() || matches!(ch, '.' | '_' | '-') { + ch + } else { + '_' + } + }) + .collect::(); + format!("attachment; filename=\"{sanitized}\"") +} + +#[cfg(test)] +mod tests { + use super::*; + use jsonwebtoken::{Algorithm, EncodingKey, Header, encode}; + + fn test_jwt(secret: &[u8], claims: Value) -> String { + encode( + &Header::new(Algorithm::HS256), + &claims, + &EncodingKey::from_secret(secret), + ) + .unwrap() + } + + #[test] + fn verifies_hs256_jwt_and_separate_slack_channel_claims() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "user_123", + "aud": "centaur-api", + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C987654321"], + "history_channels": ["C111111111"] + } + }), + ); + let claims = crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console", + ) + .unwrap(); + ensure_upload_channel_allowed(&claims, "C123456789").unwrap(); + ensure_download_channel_allowed(&claims, "C987654321").unwrap(); + ensure_history_channel_allowed(&claims, "C111111111").unwrap(); + assert!(matches!( + ensure_upload_channel_allowed(&claims, "C987654321").unwrap_err(), + ApiError::Forbidden(_) + )); + assert!(matches!( + ensure_download_channel_allowed(&claims, "C123456789").unwrap_err(), + ApiError::Forbidden(_) + )); + assert!(matches!( + ensure_history_channel_allowed(&claims, "C123456789").unwrap_err(), + ApiError::Forbidden(_) + )); + } + + #[test] + fn extracts_deduped_channel_ids_from_all_slack_claims() { + let claims = SlackFileProxyClaims { + slack: SlackProxyClaims { + upload_channels: vec!["C123456789".to_owned()], + download_channels: vec!["G123456789".to_owned(), "C123456789".to_owned()], + history_channels: vec!["D123456789".to_owned(), "G123456789".to_owned()], + }, + }; + + assert_eq!( + slack_channel_ids_from_claims(&claims).unwrap(), + vec![ + "C123456789".to_owned(), + "D123456789".to_owned(), + "G123456789".to_owned(), + ] + ); + } + + #[test] + fn channel_item_enriches_slack_metadata_with_permissions() { + let claims = SlackFileProxyClaims { + slack: SlackProxyClaims { + upload_channels: vec!["C123456789".to_owned()], + download_channels: vec![], + history_channels: vec!["C123456789".to_owned()], + }, + }; + let channel = json!({ + "id": "C123456789", + "name": "general", + "purpose": {"value": "Company updates"}, + "topic": {"value": "Announcements"}, + "num_members": 42, + "is_private": false, + "is_member": true + }); + + let item = slack_channel_item(&claims, "C123456789", &channel); + + assert_eq!(item.id, "C123456789"); + assert_eq!(item.name, "general"); + assert_eq!(item.purpose, "Company updates"); + assert_eq!(item.topic, "Announcements"); + assert_eq!(item.member_count, 42); + assert!(!item.is_private); + assert!(item.is_member); + assert!(item.can_upload); + assert!(!item.can_download); + assert!(item.can_read_history); + } + + #[test] + fn channel_info_form_requests_member_counts() { + assert_eq!( + slack_channel_info_form("C123456789"), + vec![ + ("channel", "C123456789".to_owned()), + ("include_num_members", "true".to_owned()), + ] + ); + } + + #[test] + fn files_list_form_maps_proxy_query_to_slack_params() { + let query = SlackFilesListQuery { + channel_id: Some("C123456789".to_owned()), + limit: Some(20), + page: Some(3), + }; + + assert_eq!( + slack_files_list_form("C123456789", &query), + vec![ + ("channel", "C123456789".to_owned()), + ("count", "20".to_owned()), + ("page", "3".to_owned()), + ] + ); + } + + #[test] + fn files_list_form_defaults_to_capped_first_page() { + let query = SlackFilesListQuery { + channel_id: Some("C123456789".to_owned()), + limit: None, + page: None, + }; + + assert_eq!( + slack_files_list_form("C123456789", &query), + vec![ + ("channel", "C123456789".to_owned()), + ("count", DEFAULT_SLACK_FILES_LIST_LIMIT.to_string()), + ("page", "1".to_owned()), + ] + ); + } + + #[test] + fn channel_members_form_maps_proxy_query_to_slack_params() { + let query = SlackChannelMembersQuery { + limit: Some(500), + cursor: Some("cursor-1".to_owned()), + }; + + assert_eq!( + slack_channel_members_form("C123456789", &query), + vec![ + ("channel", "C123456789".to_owned()), + ("limit", "500".to_owned()), + ("cursor", "cursor-1".to_owned()), + ] + ); + } + + #[test] + fn file_info_form_maps_proxy_query_to_slack_params() { + assert_eq!( + slack_file_info_form("F123456789"), + vec![("file", "F123456789".to_owned())] + ); + } + + #[test] + fn files_list_has_more_reads_legacy_paging() { + assert!(slack_files_list_has_more(&json!({ + "paging": {"page": 1, "pages": 2} + }))); + assert!(!slack_files_list_has_more(&json!({ + "paging": {"page": 2, "pages": 2} + }))); + } + + #[test] + fn validates_files_list_query() { + validate_slack_files_list_query(&SlackFilesListQuery { + channel_id: Some("C123456789".to_owned()), + limit: Some(200), + page: Some(1), + }) + .unwrap(); + + assert!(matches!( + validate_slack_files_list_query(&SlackFilesListQuery { + channel_id: None, + limit: Some(20), + page: None, + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + assert!(matches!( + validate_slack_files_list_query(&SlackFilesListQuery { + channel_id: Some("C123456789".to_owned()), + limit: Some(201), + page: None, + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + assert!(matches!( + validate_slack_files_list_query(&SlackFilesListQuery { + channel_id: Some("C123456789".to_owned()), + limit: Some(20), + page: Some(0), + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + } + + #[test] + fn validates_channel_members_query() { + validate_slack_channel_members_query(&SlackChannelMembersQuery { + limit: Some(1000), + cursor: Some("cursor-1".to_owned()), + }) + .unwrap(); + + assert!(matches!( + validate_slack_channel_members_query(&SlackChannelMembersQuery { + limit: Some(1001), + cursor: None, + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + assert!(matches!( + validate_slack_channel_members_query(&SlackChannelMembersQuery { + limit: Some(10), + cursor: Some("\n".to_owned()), + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + } + + #[tokio::test] + async fn file_info_authorizes_before_reading_slack_config() { + let headers = HeaderMap::new(); + let result = + authorized_slack_file_info(&headers, http_client(), "F123456789", "C123456789").await; + + assert!(matches!(result, Err(ApiError::Unauthorized(_)))); + } + + #[test] + fn rejects_invalid_jwt_signature() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "user_123", + "aud": "centaur-api", + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + assert!(matches!( + crate::api_jwt::verify_hs256_jwt::( + &token, + b"other-secret", + "centaur-api", + "centaur-console" + ) + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } + + #[test] + fn rejects_expired_jwt() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "user_123", + "aud": "centaur-api", + "iat": 1i64, + "exp": 1i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + assert!(matches!( + crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console" + ) + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } + + #[test] + fn rejects_wrong_jwt_audience() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "user_123", + "aud": "other-api", + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + assert!(matches!( + crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console" + ) + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } + + #[test] + fn accepts_jwt_audience_array() { + let token = test_jwt( + b"secret", + json!({ + "iss": "centaur-console", + "sub": "user_123", + "aud": ["other-api", "centaur-api"], + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + let claims = crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console", + ) + .unwrap(); + ensure_upload_channel_allowed(&claims, "C123456789").unwrap(); + ensure_download_channel_allowed(&claims, "C123456789").unwrap(); + } + + #[test] + fn rejects_missing_standard_jwt_claims() { + let token = test_jwt( + b"secret", + json!({ + "aud": "centaur-api", + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + assert!(matches!( + crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console" + ) + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } + + #[test] + fn extracts_channels_from_file_metadata() { + let file = json!({ + "channels": ["C111111111"], + "groups": ["G111111111"], + "ims": ["D111111111"], + "shares": { + "public": { + "C222222222": [{"ts": "1.000001"}] + }, + "private": { + "G222222222": [{"ts": "1.000002"}] + } + } + }); + let channels = slack_file_channel_ids(&file); + assert!(channels.contains("C111111111")); + assert!(channels.contains("G111111111")); + assert!(channels.contains("D111111111")); + assert!(channels.contains("C222222222")); + assert!(channels.contains("G222222222")); + } + + #[test] + fn upload_requires_content_length() { + let headers = HeaderMap::new(); + assert!(matches!( + content_length(&headers).unwrap_err(), + ApiError::BadRequest(_) + )); + + let mut headers = HeaderMap::new(); + headers.insert(header::CONTENT_LENGTH, "42".parse().unwrap()); + assert_eq!(content_length(&headers).unwrap(), 42); + } + + #[test] + fn rejects_wrong_jwt_issuer() { + let token = test_jwt( + b"secret", + json!({ + "iss": "other-issuer", + "sub": "user_123", + "aud": "centaur-api", + "iat": 1_700_000_000i64, + "exp": 4_102_444_800i64, + "slack": { + "upload_channels": ["C123456789"], + "download_channels": ["C123456789"] + } + }), + ); + assert!(matches!( + crate::api_jwt::verify_hs256_jwt::( + &token, + b"secret", + "centaur-api", + "centaur-console" + ) + .unwrap_err(), + ApiError::Unauthorized(_) + )); + } + + #[test] + fn detects_unexpected_html_download_body() { + assert!(upstream_body_is_unexpected_html( + Some("text/html; charset=utf-8"), + Some("image/png"), + )); + assert!(upstream_body_is_unexpected_html(Some("TEXT/HTML"), None)); + assert!(!upstream_body_is_unexpected_html( + Some("text/html"), + Some("text/html"), + )); + assert!(!upstream_body_is_unexpected_html( + Some("image/png"), + Some("image/png"), + )); + assert!(!upstream_body_is_unexpected_html(None, Some("image/png"))); + } + + #[test] + fn validates_content_type() { + validate_content_type("application/pdf").unwrap(); + validate_content_type("text/plain; charset=utf-8").unwrap(); + for content_type in ["", " ", "a\nb", "a\rb", "a\0b"] { + assert!(matches!( + validate_content_type(content_type).unwrap_err(), + ApiError::BadRequest(_) + )); + } + } + + #[test] + fn upload_url_form_includes_alt_text_and_snippet_type() { + let form = slack_get_upload_url_form("notes.txt", 42, Some("Release notes"), Some("text")); + assert_eq!( + form, + vec![ + ("filename", "notes.txt".to_owned()), + ("length", "42".to_owned()), + ("alt_txt", "Release notes".to_owned()), + ("snippet_type", "text".to_owned()), + ] + ); + + let form = slack_get_upload_url_form("notes.txt", 42, None, None); + assert_eq!( + form, + vec![ + ("filename", "notes.txt".to_owned()), + ("length", "42".to_owned()), + ] + ); + } + + #[test] + fn validates_slack_channel_history_query() { + validate_slack_channel_history_query(&SlackChannelHistoryQuery { + latest: Some("1700000000.000002".to_owned()), + oldest: Some("0".to_owned()), + inclusive: Some(true), + include_all_metadata: Some(true), + limit: Some(999), + cursor: Some("next_cursor".to_owned()), + }) + .unwrap(); + + assert!(matches!( + validate_slack_channel_history_query(&SlackChannelHistoryQuery { + latest: None, + oldest: None, + inclusive: None, + include_all_metadata: None, + limit: Some(1000), + cursor: None, + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + assert!(matches!( + validate_slack_channel_history_query(&SlackChannelHistoryQuery { + latest: Some("not-a-ts".to_owned()), + oldest: None, + inclusive: None, + include_all_metadata: None, + limit: None, + cursor: None, + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + assert!(matches!( + validate_slack_channel_history_query(&SlackChannelHistoryQuery { + latest: None, + oldest: None, + inclusive: None, + include_all_metadata: None, + limit: None, + cursor: Some("bad\ncursor".to_owned()), + }) + .unwrap_err(), + ApiError::BadRequest(_) + )); + } + + #[test] + fn channel_history_form_omits_empty_query_params() { + let form = slack_channel_history_form( + "C123456789", + &SlackChannelHistoryQuery { + latest: Some("1700000000.000002".to_owned()), + oldest: None, + inclusive: Some(false), + include_all_metadata: Some(true), + limit: Some(15), + cursor: None, + }, + ); + assert_eq!( + form, + vec![ + ("channel", "C123456789".to_owned()), + ("latest", "1700000000.000002".to_owned()), + ("inclusive", "false".to_owned()), + ("include_all_metadata", "true".to_owned()), + ("limit", "15".to_owned()), + ] + ); + } + + #[test] + fn thread_replies_form_includes_thread_ts() { + let form = slack_thread_replies_form( + "C123456789", + "1700000000.000001", + &SlackChannelHistoryQuery { + latest: None, + oldest: Some("0".to_owned()), + inclusive: Some(true), + include_all_metadata: None, + limit: Some(25), + cursor: Some("next".to_owned()), + }, + ); + assert_eq!( + form, + vec![ + ("channel", "C123456789".to_owned()), + ("oldest", "0".to_owned()), + ("inclusive", "true".to_owned()), + ("limit", "25".to_owned()), + ("cursor", "next".to_owned()), + ("ts", "1700000000.000001".to_owned()), + ] + ); + } +} diff --git a/services/api-rs/crates/centaur-api-server/src/tool_discovery.rs b/services/api-rs/crates/centaur-api-server/src/tool_discovery.rs index 2972b1db0..5e5c6f8b0 100644 --- a/services/api-rs/crates/centaur-api-server/src/tool_discovery.rs +++ b/services/api-rs/crates/centaur-api-server/src/tool_discovery.rs @@ -41,23 +41,38 @@ const HTTP_METHODS: &[&str] = &[ ]; #[derive(Clone, Debug, Default)] -pub(crate) struct ToolDiscoveryConfig { - pub(crate) tool_dirs: Option, - pub(crate) tools_path: Option, - pub(crate) tools_overlay_path: Option, - pub(crate) plugins_dir: Option, - pub(crate) tools_config: Option, +pub struct ToolDiscoveryConfig { + pub tool_dirs: Option, + pub public_tool_dirs: Option, + pub tools_path: Option, + pub tools_overlay_path: Option, + pub plugins_dir: Option, + pub tools_config: Option, } #[derive(Clone, Debug)] -pub(crate) struct DiscoveredToolProxyFragment { - pub(crate) fragment: ProxyFragment, - pub(crate) tool_count: usize, - pub(crate) secret_count: usize, +pub struct DiscoveredToolProxyFragment { + pub fragment: ProxyFragment, + pub tool_count: usize, + pub secret_count: usize, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub(crate) struct DiscoveredToolCatalog { + pub(crate) tools: Vec, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub(crate) struct DiscoveredTool { + pub(crate) name: String, + pub(crate) package: String, + pub(crate) description: Option, + pub(crate) client_module: String, + pub(crate) project_dir: PathBuf, } #[derive(Debug, Error)] -pub(crate) enum ToolDiscoveryError { +pub enum ToolDiscoveryError { #[error("failed to read {path}: {source}")] Read { path: PathBuf, @@ -75,7 +90,7 @@ pub(crate) enum ToolDiscoveryError { } impl ToolDiscoveryConfig { - pub(crate) fn resolve_tool_dirs(&self) -> Result, ToolDiscoveryError> { + pub fn resolve_tool_dirs(&self) -> Result, ToolDiscoveryError> { if let Some(tool_dirs) = clean_optional_str(self.tool_dirs.as_deref()) { return Ok(split_tool_dirs(&tool_dirs)); } @@ -114,9 +129,17 @@ impl ToolDiscoveryConfig { }); Ok(vec![root.join("tools")]) } + + pub fn resolve_public_tool_dirs(&self) -> Vec { + self.public_tool_dirs + .as_deref() + .and_then(|value| clean_optional_str(Some(value))) + .map(|value| split_tool_dirs(&value)) + .unwrap_or_default() + } } -pub(crate) fn discover_tool_proxy_fragment( +pub fn discover_tool_proxy_fragment( tool_dirs: &[PathBuf], ) -> Result { let tools = collect_plugin_metadata(tool_dirs)?.tools; @@ -139,7 +162,7 @@ pub(crate) fn discover_tool_proxy_fragment( }) } -pub(crate) fn discover_persona_registry( +pub fn discover_persona_registry( tool_dirs: &[PathBuf], default_persona_id: Option, ) -> Result { @@ -148,6 +171,74 @@ pub(crate) fn discover_persona_registry( .map_err(ToolDiscoveryError::Invalid) } +pub(crate) fn discover_tool_catalog( + tool_dirs: &[PathBuf], + tool_allowlist: Option<&str>, + tool_blocklist: Option<&str>, +) -> Result { + let allowlist = parse_tool_name_filter(tool_allowlist); + let blocklist = parse_tool_name_filter(tool_blocklist); + let mut tools = BTreeMap::new(); + + // Match services/sandbox/install_tool_shims.py: scan TOOL_DIRS in order, + // filter by package-directory, project, or script name, and let the last + // package that declares a script name own that script. + for base_dir in tool_dirs { + if !base_dir.exists() { + continue; + } + for tool_dir in candidate_tool_dirs(base_dir)? { + let pyproject_path = tool_dir.join("pyproject.toml"); + let Some(LoadedPluginMeta::Tool(tool)) = + load_plugin_meta(base_dir, &tool_dir, &pyproject_path)? + else { + continue; + }; + let identifiers = tool_identifiers(&tool); + if !allowlist.is_empty() && identifiers.is_disjoint(&allowlist) { + continue; + } + if !identifiers.is_disjoint(&blocklist) { + continue; + } + for script_name in tool.script_names { + if blocklist.contains(&script_name) { + continue; + } + tools.insert( + script_name.clone(), + DiscoveredTool { + name: script_name, + package: tool.package.clone(), + description: tool.description.clone(), + client_module: tool.client_module.clone(), + project_dir: tool.dir.clone(), + }, + ); + } + } + } + Ok(DiscoveredToolCatalog { + tools: tools.into_values().collect(), + }) +} + +fn parse_tool_name_filter(value: Option<&str>) -> BTreeSet { + value + .unwrap_or_default() + .split(',') + .map(str::trim) + .filter(|name| !name.is_empty()) + .map(str::to_owned) + .collect() +} + +fn tool_identifiers(tool: &LoadedToolMeta) -> BTreeSet { + let mut identifiers = BTreeSet::from([tool.name.clone(), tool.package.clone()]); + identifiers.extend(tool.script_names.iter().cloned()); + identifiers +} + fn split_tool_dirs(value: &str) -> Vec { value .split(':') @@ -284,6 +375,11 @@ fn parse_toml(path: &Path, contents: &str) -> Result, + client_module: String, + script_names: Vec, secrets: Vec, } @@ -439,7 +535,7 @@ fn load_plugin_meta( .and_then(|value| value.get("centaur")) .unwrap_or(&default_tool_conf); if tool_conf.get("type").and_then(TomlValue::as_str) != Some("persona") { - return load_tool_meta(source_root, plugin_dir, tool_conf) + return load_tool_meta(source_root, plugin_dir, &pyproject, tool_conf) .map(|meta| meta.map(LoadedPluginMeta::Tool)); } let id = plugin_dir @@ -476,6 +572,7 @@ fn load_plugin_meta( fn load_tool_meta( source_root: &Path, tool_dir: &Path, + pyproject: &TomlValue, tool_conf: &TomlValue, ) -> Result, ToolDiscoveryError> { let name = tool_dir @@ -485,6 +582,39 @@ fn load_tool_meta( ToolDiscoveryError::Invalid(format!("invalid tool path {}", tool_dir.display())) })? .to_owned(); + let default_project_conf = TomlValue::Table(Default::default()); + let project_conf = pyproject.get("project").unwrap_or(&default_project_conf); + let package = project_conf + .get("name") + .and_then(TomlValue::as_str) + .map(str::to_owned) + .unwrap_or_else(|| name.clone()); + let description = project_conf + .get("description") + .and_then(TomlValue::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_owned); + let client_module = tool_conf + .get("module") + .and_then(TomlValue::as_str) + .map(str::trim) + .filter(|value| !value.is_empty()) + .unwrap_or("client.py") + .to_owned(); + let script_names = project_conf + .get("scripts") + .and_then(TomlValue::as_table) + .map(|scripts| { + let mut names = scripts + .keys() + .filter(|script| !script.contains('/') && !script.contains('\0')) + .cloned() + .collect::>(); + names.sort(); + names + }) + .unwrap_or_default(); let default_hosts = string_array(tool_conf.get("hosts")); let labels = tool_labels(&name, &overlay_name_for_root(source_root)); let secrets = match parse_secret_list(tool_conf.get("secrets"), &default_hosts, &labels) @@ -506,7 +636,15 @@ fn load_tool_meta( return Ok(None); } }; - Ok(Some(LoadedToolMeta { name, secrets })) + Ok(Some(LoadedToolMeta { + name, + dir: tool_dir.to_path_buf(), + package, + description, + client_module, + script_names, + secrets, + })) } #[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)] @@ -1585,6 +1723,22 @@ mod tests { ); } + #[test] + fn resolves_public_tool_dirs_from_explicit_env_string() { + let config = ToolDiscoveryConfig { + public_tool_dirs: Some("/public-base:/public-overlay".to_owned()), + ..Default::default() + }; + + assert_eq!( + config.resolve_public_tool_dirs(), + vec![ + PathBuf::from("/public-base"), + PathBuf::from("/public-overlay") + ] + ); + } + #[test] fn resolves_sandbox_style_tools_path_and_overlay_path() { let config = ToolDiscoveryConfig { @@ -1599,6 +1753,89 @@ mod tests { ); } + #[test] + fn tool_catalog_matches_sandbox_filters_and_script_precedence() { + let temp = temp_dir("api-rs-tool-catalog"); + let base = temp.join("base"); + let overlay = temp.join("overlay"); + write_tool( + &base.join("category").join("alpha-dir"), + r#" +[project] +name = "alpha-project" + +[project.scripts] +alpha = "alpha:main" +shared = "alpha:main" +"#, + ); + write_tool( + &base.join("category").join("beta-dir"), + r#" +[project] +name = "beta-project" + +[project.scripts] +beta = "beta:main" +"#, + ); + write_tool( + &base.join("category").join("blocked-dir"), + r#" +[project] +name = "blocked-project" + +[project.scripts] +blocked = "blocked:main" +safe-sibling = "blocked:main" +"#, + ); + write_tool( + &base.join("category").join("phantom"), + r#" +[project] +name = "phantom-project" +"#, + ); + write_tool( + &overlay.join("category").join("replacement"), + r#" +[project] +name = "overlay-project" + +[project.scripts] +overlay = "overlay:main" +shared = "overlay:main" +"#, + ); + + let catalog = discover_tool_catalog( + &[base.clone(), overlay.clone()], + Some("alpha-dir,beta-project,overlay,blocked-dir,phantom"), + Some("blocked"), + ) + .unwrap(); + + assert_eq!( + catalog + .tools + .iter() + .map(|tool| tool.name.as_str()) + .collect::>(), + vec!["alpha", "beta", "overlay", "shared"] + ); + let shared = catalog + .tools + .iter() + .find(|tool| tool.name == "shared") + .expect("shared script"); + assert_eq!(shared.package, "overlay-project"); + assert_eq!(shared.project_dir, overlay.join("category/replacement")); + assert!(catalog.tools.iter().all(|tool| tool.name != "safe-sibling")); + + let _ = fs::remove_dir_all(temp); + } + #[test] fn postgres_listeners_retain_sandbox_env_name_and_database() { // api-rs bakes the sandbox PG DSNs from `sandbox_env`, so the listener diff --git a/services/api-rs/crates/centaur-api-server/src/types.rs b/services/api-rs/crates/centaur-api-server/src/types.rs index 09924cbea..9f995403f 100644 --- a/services/api-rs/crates/centaur-api-server/src/types.rs +++ b/services/api-rs/crates/centaur-api-server/src/types.rs @@ -37,6 +37,8 @@ pub struct CreateSessionResponse { pub struct SessionContextResponse { pub thread_key: ThreadKey, #[serde(skip_serializing_if = "Option::is_none")] + pub title: Option, + #[serde(skip_serializing_if = "Option::is_none")] pub slack: Option, } @@ -75,9 +77,12 @@ pub struct ExecuteSessionResponse { pub status: String, } -#[derive(Clone, Debug, Deserialize, Serialize)] +#[derive(Clone, Debug, Default, Deserialize, Serialize)] pub struct ReleaseThreadRequest { pub release_id: Option, + /// Optional caller-side compare-and-swap fence. When present, release is + /// rejected unless the thread is still assigned to this exact sandbox. + pub expected_sandbox_id: Option, #[serde(default)] pub cancel_inflight: bool, } @@ -88,6 +93,7 @@ pub struct ReleaseThreadResponse { #[serde(flatten)] pub session: Session, pub release_id: Option, + pub expected_sandbox_id: Option, pub cancel_inflight: bool, pub sandbox_released: bool, pub sandbox_release_error: Option, @@ -95,6 +101,19 @@ pub struct ReleaseThreadResponse { pub execution_cancelled: bool, } +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct InterruptSessionExecutionRequest { + pub reason: Option, +} + +#[derive(Clone, Debug, Serialize)] +pub struct InterruptSessionExecutionResponse { + pub ok: bool, + pub interrupted: bool, + pub execution_id: Option, + pub thread_key: ThreadKey, +} + #[derive(Clone, Debug, Deserialize)] pub struct EventsQuery { pub after_event_id: Option, @@ -106,8 +125,6 @@ pub struct ListWorkflowRunsQuery { pub limit: Option, pub workflow_name: Option, pub thread_key: Option, - pub status: Option, - pub parent_run_id: Option, } #[derive(Clone, Debug, Deserialize, Serialize)] diff --git a/services/api-rs/crates/centaur-iron-control/src/client.rs b/services/api-rs/crates/centaur-iron-control/src/client.rs index 3cc1b5313..346e03ec3 100644 --- a/services/api-rs/crates/centaur-iron-control/src/client.rs +++ b/services/api-rs/crates/centaur-iron-control/src/client.rs @@ -16,7 +16,7 @@ use crate::models::{ AwsAuthSecretInput, BrokerCredentialInput, BrokerCredentialRecord, DataEnvelope, EffectiveConfig, GcpAuthSecretInput, GcpIdTokenSecretInput, Grant, GrantSecret, Grantee, HmacSecretInput, IdentityInput, OAuthTokenSecretInput, PgDsnSecretInput, Principal, Proxy, - ProxyInput, Role, SecretRecord, StaticSecretInput, + ProxyInput, Role, SecretRecord, SlackChannelPermissionInput, StaticSecretInput, }; const API_PREFIX: &str = "/api/v1"; @@ -179,6 +179,20 @@ impl IronControlClient { .await } + /// Create or update one Slack channel permission row on a principal without + /// replacing that principal's other Slack permissions. + pub async fn upsert_slack_channel_permission( + &self, + principal_id: &str, + input: &SlackChannelPermissionInput, + ) -> Result<()> { + let path = format!( + "{API_PREFIX}/principals/{}/slack_channel_permissions", + urlencoding::encode(principal_id) + ); + self.write_unit(Method::POST, &path, input).await + } + /// List the roles assigned to a principal (by OID; this sub-resource route /// does not resolve ``foreign_id``s — pass the OID from [`Self::get_principal`]). pub async fn list_principal_roles(&self, principal: &str) -> Result> { diff --git a/services/api-rs/crates/centaur-iron-control/src/models.rs b/services/api-rs/crates/centaur-iron-control/src/models.rs index ed1b48d81..d6f485772 100644 --- a/services/api-rs/crates/centaur-iron-control/src/models.rs +++ b/services/api-rs/crates/centaur-iron-control/src/models.rs @@ -479,9 +479,20 @@ pub struct Principal { #[serde(default)] pub labels: BTreeMap, #[serde(default = "default_true")] - pub sandbox_repo_cache_enabled: bool, - #[serde(default = "default_true")] pub sandbox_observability_enabled: bool, + #[serde(default = "default_true")] + pub sandbox_api_server_enabled: bool, +} + +/// Request body for creating/updating one Slack permission row on a principal. +#[derive(Clone, Debug, PartialEq, Eq, Serialize)] +pub struct SlackChannelPermissionInput { + pub channel_id: String, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub channel_name: Option, + pub upload_enabled: bool, + pub download_enabled: bool, + pub history_enabled: bool, } /// A principal's effective config — the same secrets/postgres the principal's @@ -701,7 +712,7 @@ pub struct Proxy { #[cfg(test)] mod tests { - use super::normalize_gcp_id_token_header; + use super::{SlackChannelPermissionInput, normalize_gcp_id_token_header}; #[test] fn normalizes_supported_gcp_id_token_headers() { @@ -715,4 +726,20 @@ mod tests { ); assert_eq!(normalize_gcp_id_token_header("x-other"), None); } + + #[test] + fn slack_channel_permission_serializes_false_values() { + let value = serde_json::to_value(SlackChannelPermissionInput { + channel_id: "C0123456789".to_owned(), + channel_name: None, + upload_enabled: false, + download_enabled: true, + history_enabled: false, + }) + .unwrap(); + + assert_eq!(value["upload_enabled"], false); + assert_eq!(value["download_enabled"], true); + assert_eq!(value["history_enabled"], false); + } } diff --git a/services/api-rs/crates/centaur-iron-control/src/principal.rs b/services/api-rs/crates/centaur-iron-control/src/principal.rs index 461a8e69e..32b525569 100644 --- a/services/api-rs/crates/centaur-iron-control/src/principal.rs +++ b/services/api-rs/crates/centaur-iron-control/src/principal.rs @@ -69,6 +69,17 @@ pub fn derive_principal( thread_key: &str, actor_user_id: Option<&str>, conversation_name: Option<&str>, +) -> PrincipalRef { + derive_principal_with_slack_team(thread_key, actor_user_id, None, conversation_name) +} + +/// Resolve the principal for a thread, allowing ingress metadata to supply the +/// Slack team id when legacy DM thread keys omit it. +pub fn derive_principal_with_slack_team( + thread_key: &str, + actor_user_id: Option<&str>, + slack_team_id: Option<&str>, + conversation_name: Option<&str>, ) -> PrincipalRef { let display_name = conversation_name .map(str::trim) @@ -144,7 +155,20 @@ pub fn derive_principal( }; } - let (team_id, conversation_id) = parse_slack_segments(thread_key); + let (thread_team_id, conversation_id) = parse_slack_segments(thread_key); + let metadata_team_id = slack_team_id.map(str::trim).filter(|team| !team.is_empty()); + // Channel principals must never be scoped by the message-derived metadata + // team: in Slack Connect shared channels that team can identify an external + // requester's workspace, which would fork a separate principal from the + // host channel's. A Slack channel id is globally unique, so an un-teamed + // scope is correct and collision-free. DM principals stay team-scoped so + // legacy DM keys that omit the team still resolve to the right per-user + // identity (see #882). + let team_id = if is_direct_message(conversation_id) { + thread_team_id.or(metadata_team_id) + } else { + thread_team_id + }; let mut labels = BTreeMap::new(); if let Some(team) = team_id { labels.insert("slack_team_id".to_owned(), team.to_owned()); @@ -208,6 +232,11 @@ fn parse_slack_segments(thread_key: &str) -> (Option<&str>, Option<&str>) { (team, conversation) } +/// The first Slack conversation id (``C``/``D``/``G``) in a thread key. +pub(crate) fn slack_conversation_id(thread_key: &str) -> Option<&str> { + parse_slack_segments(thread_key).1 +} + /// The guild and (optional) channel segments of a ``discord::`` /// thread key, or ``None`` when the key is not a Discord thread. The discordbot /// encodes session threads as ``discord::[:]``, @@ -260,7 +289,7 @@ fn parse_teams_adapter_segments(thread_key: &str) -> Option<(String, String, Opt } /// Slack direct-message conversation ids start with ``D``. -fn is_direct_message(conversation_id: Option<&str>) -> bool { +pub(crate) fn is_direct_message(conversation_id: Option<&str>) -> bool { conversation_id .and_then(|id| id.chars().next()) .is_some_and(|first| first.eq_ignore_ascii_case(&'d')) @@ -326,6 +355,59 @@ mod tests { assert_eq!(principal.name, "Slack User U07ABC (team T123)"); } + #[test] + fn metadata_team_id_is_folded_into_legacy_dm_user_key() { + let principal = derive_principal_with_slack_team( + "slack:D9:ts", + Some("U07ABC"), + Some("T123"), + Some("Ada Lovelace"), + ); + assert_eq!(principal.foreign_id, "slack-user-t123-u07abc"); + assert_eq!(principal.name, "Slack DM @Ada Lovelace"); + assert_eq!( + principal.labels.get("slack_team_id").map(String::as_str), + Some("T123") + ); + assert_eq!( + principal.labels.get("slack_user_id").map(String::as_str), + Some("U07ABC") + ); + } + + #[test] + fn metadata_team_id_is_not_folded_into_channel_key() { + // A Slack Connect requester's team must not scope a channel principal: + // the channel is globally unique and belongs to the host workspace. + let principal = derive_principal_with_slack_team( + "slack:C456:1780000000.0001", + Some("U07ABC"), + Some("T_EXTERNAL"), + None, + ); + assert_eq!(principal.foreign_id, "slack-channel-c456"); + assert_eq!(principal.labels.get("slack_team_id"), None); + assert_eq!( + principal.labels.get("slack_channel_id").map(String::as_str), + Some("C456") + ); + } + + #[test] + fn thread_key_team_id_wins_over_metadata_team_id() { + let principal = derive_principal_with_slack_team( + "slack:T_FROM_KEY:D9:ts", + Some("U07ABC"), + Some("T_FROM_METADATA"), + None, + ); + assert_eq!(principal.foreign_id, "slack-user-t-from-key-u07abc"); + assert_eq!( + principal.labels.get("slack_team_id").map(String::as_str), + Some("T_FROM_KEY") + ); + } + #[test] fn non_slack_thread_keys_slug_the_whole_key() { let principal = derive_principal("api", None, None); diff --git a/services/api-rs/crates/centaur-iron-control/src/registry.rs b/services/api-rs/crates/centaur-iron-control/src/registry.rs index af4b6f51e..942ac31ab 100644 --- a/services/api-rs/crates/centaur-iron-control/src/registry.rs +++ b/services/api-rs/crates/centaur-iron-control/src/registry.rs @@ -4,9 +4,8 @@ //! Today the proxy config is rendered from fragments and baked into a //! per-sandbox ConfigMap. Under iron-control the same fragments become durable //! control-plane state: each fragment's secrets are upserted as typed secret -//! resources and granted to a role. api-rs currently folds infra, harness, and -//! discovered tool fragments into the single shared infra role so each sandbox -//! principal only needs one assignment. +//! resources and granted to a role. api-rs registers infra and harness +//! fragments against the shared infra role. //! //! [`secret_inputs_from_fragment`] is the pure translation (fragment → secret //! inputs) and is unit-tested without a server; [`register_role`] drives the @@ -403,7 +402,7 @@ pub fn source_from_placeholder( ) -> SecretSource { match policy.kind { SourceKind::Env => { - let mut config = json!({ "var": placeholder }); + let mut config = json!({ "var": format!("{}{}", policy.env_prefix, placeholder) }); insert_json_key(&mut config, json_key); SecretSource { source_type: "env".to_owned(), @@ -1036,12 +1035,27 @@ pub fn unique_foreign_id(candidate: String, used: &mut BTreeSet) -> Stri #[cfg(test)] mod tests { use super::*; - use centaur_iron_proxy::load_fragment_str; + use centaur_iron_proxy::{infra_fragment, load_fragment_str}; fn env_policy() -> SourcePolicy { SourcePolicy::env() } + #[test] + fn env_policy_prefixes_the_resolved_environment_key() { + let source = source_from_placeholder( + &SourcePolicy::env().with_env_prefix("CENTAUR_"), + "SLACK_FEEDBACK_API_KEY", + None, + ); + + assert_eq!(source.source_type, "env"); + assert_eq!( + source.config, + json!({ "var": "CENTAUR_SLACK_FEEDBACK_API_KEY" }) + ); + } + #[test] fn translates_replace_secret_with_derived_env_source() { let fragment = load_fragment_str( @@ -1198,6 +1212,46 @@ transforms: assert!(input.replace_config.is_none()); } + #[test] + fn built_in_github_broker_becomes_infra_replace_secret() { + let fragment = infra_fragment().expect("built-in infra fragment parses"); + let inputs = + secret_inputs_from_fragment("default", "infra", &fragment, &env_policy()).unwrap(); + let input = inputs + .iter() + .find_map(|input| match input { + SecretInput::Static(input) if input.foreign_id == "infra-github-app" => Some(input), + _ => None, + }) + .expect("built-in GitHub broker static secret"); + + assert_eq!(input.name, "github-app"); + assert_eq!(input.source.source_type, "token_broker"); + assert_eq!( + input.source.config, + json!({ + "credential_id": "github-app", + "credential_namespace": "default", + }) + ); + assert!(input.inject_config.is_none()); + let replace = input.replace_config.as_ref().expect("replace config"); + assert_eq!(replace.proxy_value, "GITHUB_TOKEN"); + assert_eq!(replace.match_headers, vec!["Authorization"]); + assert_eq!( + input + .rules + .iter() + .filter_map(|rule| rule.host.as_deref()) + .collect::>(), + vec!["github.com", "api.github.com"] + ); + assert_eq!( + input.labels.get("managed-by").map(String::as_str), + Some("centaur") + ); + } + #[test] fn placeholder_inject_secret_derives_source() { let fragment = load_fragment_str( diff --git a/services/api-rs/crates/centaur-iron-control/src/session.rs b/services/api-rs/crates/centaur-iron-control/src/session.rs index bf2568be9..348548234 100644 --- a/services/api-rs/crates/centaur-iron-control/src/session.rs +++ b/services/api-rs/crates/centaur-iron-control/src/session.rs @@ -8,15 +8,19 @@ //! principal is derived from the thread key (see [`crate::derive_principal`]). use serde_json::Value; +use std::collections::BTreeMap; use crate::IronControlClient; use crate::error::{IronControlError, Result}; -use crate::models::Principal; -use crate::principal::derive_principal; +use crate::models::{Principal, SlackChannelPermissionInput}; +use crate::principal::{ + derive_principal_with_slack_team, is_direct_message, slack_conversation_id, +}; #[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] struct SessionPrincipalMetadata<'a> { actor_user_id: Option<&'a str>, + slack_team_id: Option<&'a str>, conversation_name: Option<&'a str>, } @@ -31,6 +35,7 @@ impl<'a> SessionPrincipalMetadata<'a> { .or_else(|| metadata.get("aad_object_id")) .or_else(|| metadata.get("user_id")) .and_then(Value::as_str), + slack_team_id: metadata.get("slack_team_id").and_then(Value::as_str), conversation_name: metadata .get("slack_conversation_name") .or_else(|| metadata.get("discord_conversation_name")) @@ -82,22 +87,39 @@ impl SessionRegistrar { metadata: Option<&Value>, ) -> Result { let metadata = SessionPrincipalMetadata::from_session_metadata(metadata); - let principal = derive_principal( + let principal = derive_principal_with_slack_team( thread_key, metadata.actor_user_id, + metadata.slack_team_id, metadata.conversation_name, ); - let input = principal.to_identity_input(&self.namespace); - let exists = match self + let mut input = principal.to_identity_input(&self.namespace); + let existing = match self .client .get_principal(&self.namespace, &input.foreign_id) .await { - Ok(_) => true, - Err(error) if is_status(&error, 404) => false, + Ok(existing) => Some(existing), + Err(error) if is_status(&error, 404) => None, Err(error) => return Err(error), }; + let exists = existing.is_some(); + if let Some(existing) = existing { + let mut labels = existing.labels; + labels.extend(input.labels); + input.labels = labels; + } + let slack_permission = slack_permission_for_thread(thread_key, &input.labels); + let should_upsert_slack_permission = !exists + || slack_permission + .as_ref() + .is_some_and(|permission| is_direct_message(Some(&permission.channel_id))); let record = self.client.upsert_principal(&input).await?; + if should_upsert_slack_permission && let Some(permission) = slack_permission { + self.client + .upsert_slack_channel_permission(&record.id, &permission) + .await?; + } if !exists { for role_id in &self.assign_role_ids { match self.client.assign_role(&record.id, role_id).await { @@ -115,6 +137,39 @@ impl SessionRegistrar { } } +fn slack_permission_for_thread( + thread_key: &str, + labels: &BTreeMap, +) -> Option { + if let Some(channel_id) = labels.get("slack_channel_id") { + let channel_id = channel_id.trim(); + return (!is_direct_message(Some(channel_id))) + .then(|| slack_permission(channel_id.to_owned(), None)); + } + + let user_id = labels.get("slack_user_id")?; + let conversation_id = slack_conversation_id(thread_key)?; + is_direct_message(Some(conversation_id)).then(|| { + slack_permission( + conversation_id.to_owned(), + Some(user_id.trim().to_owned()).filter(|value| !value.is_empty()), + ) + }) +} + +fn slack_permission( + channel_id: String, + channel_name: Option, +) -> SlackChannelPermissionInput { + SlackChannelPermissionInput { + channel_id, + channel_name, + upload_enabled: true, + download_enabled: true, + history_enabled: true, + } +} + fn is_status(err: &IronControlError, code: u16) -> bool { matches!(err, IronControlError::Status { status, .. } if *status == code) } @@ -167,6 +222,17 @@ mod tests { ); } + #[test] + fn session_principal_metadata_carries_slack_team_id() { + assert_eq!( + SessionPrincipalMetadata::from_session_metadata(Some(&json!({ + "slack_team_id": "T123" + }))) + .slack_team_id, + Some("T123") + ); + } + #[tokio::test] async fn register_session_seeds_roles_for_new_principal() { let (base_url, requests, server) = spawn_iron_control_stub(false).await; @@ -177,6 +243,7 @@ mod tests { ); let metadata = json!({ "slack_user_id": "U123", + "slack_team_id": "T123", "slack_conversation_name": "general" }); @@ -192,6 +259,11 @@ mod tests { ) ); assert!(requests.contains(&"PUT /api/v1/principals/slack-channel-t123-c123".to_owned())); + assert!( + requests.contains( + &"POST /api/v1/principals/prn_channel/slack_channel_permissions".to_owned() + ) + ); assert!(requests.contains(&"POST /api/v1/principals/prn_channel/roles".to_owned())); server.abort(); } @@ -206,6 +278,7 @@ mod tests { ); let metadata = json!({ "slack_user_id": "U123", + "slack_team_id": "T123", "slack_conversation_name": "general" }); @@ -221,6 +294,12 @@ mod tests { ) ); assert!(requests.contains(&"PUT /api/v1/principals/slack-channel-t123-c123".to_owned())); + assert!( + !requests + .iter() + .any(|request| request.ends_with("/slack_channel_permissions")), + "existing principals must not have Slack permissions reset" + ); assert!( !requests .iter() @@ -230,6 +309,76 @@ mod tests { server.abort(); } + #[tokio::test] + async fn register_session_upserts_slack_dm_permission_for_new_user_principal() { + let (base_url, requests, server) = spawn_iron_control_stub(false).await; + let registrar = SessionRegistrar::new( + IronControlClient::new(base_url, "test-key"), + "default", + vec![], + ); + let metadata = json!({ + "slack_user_id": "U123", + "slack_team_id": "T123", + "slack_conversation_name": "Ada Lovelace" + }); + + registrar + .register_session("slack:T123:D123:1773364194.179929", Some(&metadata)) + .await + .unwrap(); + + let requests = requests.lock().unwrap(); + assert!(requests.contains(&"PUT /api/v1/principals/slack-user-t123-u123".to_owned())); + assert!( + requests + .contains(&"POST /api/v1/principals/prn_user/slack_channel_permissions".to_owned()) + ); + server.abort(); + } + + #[tokio::test] + async fn register_session_upserts_slack_dm_permission_for_existing_user_principal() { + let (base_url, requests, server) = spawn_iron_control_stub(true).await; + let registrar = SessionRegistrar::new( + IronControlClient::new(base_url, "test-key"), + "default", + vec!["role_infra".to_owned()], + ); + let metadata = json!({ + "slack_user_id": "U123", + "slack_team_id": "T123", + "slack_conversation_name": "Ada Lovelace" + }); + + registrar + .register_session("slack:T123:D123:1773364194.179929", Some(&metadata)) + .await + .unwrap(); + + let requests = requests.lock().unwrap(); + assert!(requests.contains(&"PUT /api/v1/principals/slack-user-t123-u123".to_owned())); + assert!( + requests + .contains(&"POST /api/v1/principals/prn_user/slack_channel_permissions".to_owned()) + ); + assert!( + !requests + .iter() + .any(|request| request == "POST /api/v1/principals/prn_user/roles"), + "existing DM principals must not have manually removed roles restored" + ); + server.abort(); + } + + #[test] + fn slack_permission_for_thread_skips_dm_channel_fallback_without_user() { + let mut labels = BTreeMap::new(); + labels.insert("slack_channel_id".to_owned(), "D123".to_owned()); + + assert_eq!(slack_permission_for_thread("slack:D123:ts", &labels), None); + } + async fn spawn_iron_control_stub( principal_exists: bool, ) -> (String, Arc>>, tokio::task::JoinHandle<()>) { @@ -261,14 +410,28 @@ mod tests { ("GET", "/api/v1/principals/lookup/default/slack-channel-t123-c123") if principal_exists => { - ("200 OK", principal_body()) + ("200 OK", channel_principal_body()) + } + ("GET", "/api/v1/principals/lookup/default/slack-user-t123-u123") + if principal_exists => + { + ("200 OK", user_principal_body()) } - ("GET", "/api/v1/principals/lookup/default/slack-channel-t123-c123") => { + ("GET", "/api/v1/principals/lookup/default/slack-channel-t123-c123") + | ("GET", "/api/v1/principals/lookup/default/slack-user-t123-u123") => { ("404 Not Found", r#"{"error":"not found"}"#.to_owned()) } ("PUT", "/api/v1/principals/slack-channel-t123-c123") => { - ("200 OK", principal_body()) + ("200 OK", channel_principal_body()) + } + ("PUT", "/api/v1/principals/slack-user-t123-u123") => { + ("200 OK", user_principal_body()) } + ( + "POST", + "/api/v1/principals/prn_channel/slack_channel_permissions" + | "/api/v1/principals/prn_user/slack_channel_permissions", + ) => ("200 OK", r#"{"data":{"ok":true}}"#.to_owned()), ("POST", "/api/v1/principals/prn_channel/roles") => { ("200 OK", r#"{"data":{"ok":true}}"#.to_owned()) } @@ -288,7 +451,11 @@ mod tests { (base_url, requests, handle) } - fn principal_body() -> String { + fn channel_principal_body() -> String { r#"{"data":{"id":"prn_channel","namespace":"default","foreign_id":"slack-channel-t123-c123","name":"Slack Channel #general","labels":{}}}"#.to_owned() } + + fn user_principal_body() -> String { + r#"{"data":{"id":"prn_user","namespace":"default","foreign_id":"slack-user-t123-u123","name":"Slack DM @Ada Lovelace","labels":{}}}"#.to_owned() + } } diff --git a/services/api-rs/crates/centaur-iron-proxy/src/fragment.rs b/services/api-rs/crates/centaur-iron-proxy/src/fragment.rs index f0a58b7b0..242d3f0f4 100644 --- a/services/api-rs/crates/centaur-iron-proxy/src/fragment.rs +++ b/services/api-rs/crates/centaur-iron-proxy/src/fragment.rs @@ -32,6 +32,7 @@ pub fn harness_auth_fragment(engine: &str, auth_mode: &str) -> Result CODEX_API_KEY_FRAGMENT, ("codex", "access_token") => CODEX_ACCESS_TOKEN_FRAGMENT, ("openrouter", "api_key") => OPENROUTER_API_KEY_FRAGMENT, + ("meta-ai", "api_key") => META_AI_API_KEY_FRAGMENT, ("claude-code", "api_key") => CLAUDE_CODE_API_KEY_FRAGMENT, ("claude-code", "access_token") => CLAUDE_CODE_ACCESS_TOKEN_FRAGMENT, _ => return Ok(None), @@ -162,11 +163,9 @@ transforms: config: secrets: - id: OPENAI_API_KEY_AUTHORIZATION - source: - placeholder: OPENAI_API_KEY - inject: - header: Authorization - formatter: "Bearer {{.Value}}" + replace: + proxy_value: OPENAI_API_KEY + match_headers: ["Authorization"] rules: [{ host: api.openai.com }] "#; @@ -176,14 +175,24 @@ transforms: config: secrets: - id: OPENROUTER_API_KEY_AUTHORIZATION - source: - placeholder: OPENROUTER_API_KEY - inject: - header: Authorization - formatter: "Bearer {{.Value}}" + replace: + proxy_value: OPENROUTER_API_KEY + match_headers: ["Authorization"] rules: [{ host: openrouter.ai }] "#; +const META_AI_API_KEY_FRAGMENT: &str = r#" +transforms: + - name: secrets + config: + secrets: + - id: META_AI_API_KEY_AUTHORIZATION + replace: + proxy_value: META_AI_API_KEY + match_headers: ["Authorization"] + rules: [{ host: api.ai.meta.com }] +"#; + // The `openai-codex` broker credential this references is managed by // iron-control and provisioned out of band (see `centaur-perms broker create`). const CODEX_ACCESS_TOKEN_FRAGMENT: &str = r#" diff --git a/services/api-rs/crates/centaur-iron-proxy/src/infra.yaml b/services/api-rs/crates/centaur-iron-proxy/src/infra.yaml index 8ecd8d25b..522582938 100644 --- a/services/api-rs/crates/centaur-iron-proxy/src/infra.yaml +++ b/services/api-rs/crates/centaur-iron-proxy/src/infra.yaml @@ -8,33 +8,14 @@ transforms: - name: secrets config: secrets: - - replace: - proxy_value: XAI_API_KEY - match_headers: ["Authorization"] - rules: [{ host: api.x.ai }] - - replace: - proxy_value: GEMINI_API_KEY - match_headers: ["X-Goog-Api-Key"] - rules: [{ host: generativelanguage.googleapis.com }] - - replace: - proxy_value: AMP_API_KEY - match_headers: ["Authorization"] - rules: [{ host: ampcode.com }] - # GitHub App installation tokens expire ~1h after mint, so the static - # value baked into the sandbox at start goes stale within a long-lived - # (~48h) sandbox and git/gh start failing with HTTP 401. Source the - # replacement value from the `github-app` broker credential (a - # github_app_installation BrokerCredential managed by iron-control, - # provisioned by the Helm chart's tokenBroker.githubApp bootstrap or - # manually via `centaur-perms broker create`), which - # mints and rotates the installation token so every request gets a live - # one. Keep `replace` (not `inject`) so the request's existing auth - # scheme is preserved: git over HTTPS keeps its Basic `x-access-token` - # form (github.com rejects `Bearer` for git transport) while the REST - # API keeps Bearer -- both just get the placeholder swapped for the - # current token. The `GITHUB_TOKEN` placeholder is still injected into - # the sandbox env (it is this entry's proxy_value). - - source: + # The Helm bootstrap guarantees the canonical `github-app` credential + # exists before api-rs starts. Register this source on the shared infra + # role so every new sandbox principal receives the rotating token. + # Replace the placeholder instead of injecting a header: git HTTPS uses + # Basic x-access-token auth while the REST API uses Bearer, and both + # must preserve their caller-selected scheme. + - id: github-app + source: type: token_broker credential_id: github-app replace: diff --git a/services/api-rs/crates/centaur-iron-proxy/src/source.rs b/services/api-rs/crates/centaur-iron-proxy/src/source.rs index 2a18a0c7f..780399adb 100644 --- a/services/api-rs/crates/centaur-iron-proxy/src/source.rs +++ b/services/api-rs/crates/centaur-iron-proxy/src/source.rs @@ -9,6 +9,10 @@ pub struct SourcePolicy { pub kind: SourceKind, pub op_vault: String, pub ttl: String, + /// Prefix applied to Kubernetes Secret keys exposed through envFrom. + /// Placeholder names remain canonical in tool manifests; env-backed + /// iron-control sources resolve the actual prefixed environment key. + pub env_prefix: String, } impl SourcePolicy { @@ -24,11 +28,17 @@ impl SourcePolicy { Self::new(SourceKind::OnePasswordConnect, op_vault, ttl) } + pub fn with_env_prefix(mut self, prefix: impl Into) -> Self { + self.env_prefix = prefix.into(); + self + } + fn new(kind: SourceKind, op_vault: impl Into, ttl: impl Into) -> Self { Self { kind, op_vault: op_vault.into(), ttl: ttl.into(), + env_prefix: String::new(), } } } diff --git a/services/api-rs/crates/centaur-iron-proxy/src/tests.rs b/services/api-rs/crates/centaur-iron-proxy/src/tests.rs index 3c2ff801b..18a9a85a2 100644 --- a/services/api-rs/crates/centaur-iron-proxy/src/tests.rs +++ b/services/api-rs/crates/centaur-iron-proxy/src/tests.rs @@ -3,7 +3,11 @@ use super::*; #[test] fn harness_auth_fragments_are_baked_in() { let codex = harness_auth_fragment("codex", "api_key").unwrap().unwrap(); - assert!(placeholder_env(&[codex]).is_empty()); + let codex_placeholders = placeholder_env(&[codex]); + assert_eq!( + codex_placeholders.get("OPENAI_API_KEY").map(String::as_str), + Some("OPENAI_API_KEY") + ); // access_token carries the token-broker credential, not a replace // placeholder, so it contributes no sandbox placeholder env. @@ -15,7 +19,35 @@ fn harness_auth_fragments_are_baked_in() { let openrouter = harness_auth_fragment("openrouter", "api_key") .unwrap() .unwrap(); - assert!(placeholder_env(&[openrouter]).is_empty()); + let openrouter_placeholders = placeholder_env(&[openrouter]); + assert_eq!( + openrouter_placeholders + .get("OPENROUTER_API_KEY") + .map(String::as_str), + Some("OPENROUTER_API_KEY") + ); + + let meta_ai = harness_auth_fragment("meta-ai", "api_key") + .unwrap() + .unwrap(); + let meta_ai_placeholders = placeholder_env(&[meta_ai]); + assert_eq!( + meta_ai_placeholders + .get("META_AI_API_KEY") + .map(String::as_str), + Some("META_AI_API_KEY") + ); + + let claude_code = harness_auth_fragment("claude-code", "api_key") + .unwrap() + .unwrap(); + let claude_code_placeholders = placeholder_env(&[claude_code]); + assert_eq!( + claude_code_placeholders + .get("ANTHROPIC_API_KEY") + .map(String::as_str), + Some("ANTHROPIC_API_KEY") + ); assert!(harness_auth_fragment("codex", "bogus").unwrap().is_none()); @@ -25,12 +57,15 @@ fn harness_auth_fragments_are_baked_in() { Some("120s") ); let placeholders = placeholder_env(&[infra]); - for name in ["AMP_API_KEY", "GITHUB_TOKEN"] { - assert_eq!(placeholders.get(name).map(String::as_str), Some(name)); - } - assert!( - !placeholders.contains_key("SLACK_BOT_TOKEN"), - "agent sessions must not receive the broad Slack bot-token placeholder" + assert_eq!( + placeholders.get("GITHUB_TOKEN").map(String::as_str), + Some("GITHUB_TOKEN") + ); + // Slack's bot credential is control-plane-only and is not part of the + // shared infra fragment. + assert_eq!( + placeholders.get("SLACK_BOT_TOKEN").map(String::as_str), + None ); } @@ -93,3 +128,21 @@ fn shipped_proxy_allowlist_preserves_railway_project_tokens() { .any(|header| header.as_str() == Some("project-access-token")) ); } + +#[test] +fn shipped_proxy_allowlist_preserves_workflow_task_capabilities() { + let config: serde_yaml::Value = + serde_yaml::from_str(include_str!("../../../../iron-proxy/iron-proxy.yaml")).unwrap(); + let transforms = config["transforms"].as_sequence().unwrap(); + let header_allowlist = transforms + .iter() + .find(|transform| transform["name"].as_str() == Some("header_allowlist")) + .unwrap(); + let headers = header_allowlist["config"]["headers"].as_sequence().unwrap(); + + assert!( + headers + .iter() + .any(|header| { header.as_str() == Some("x-centaur-workflow-task-token") }) + ); +} diff --git a/services/api-rs/crates/centaur-perms/src/main.rs b/services/api-rs/crates/centaur-perms/src/main.rs index 2f4abcb11..014e25157 100644 --- a/services/api-rs/crates/centaur-perms/src/main.rs +++ b/services/api-rs/crates/centaur-perms/src/main.rs @@ -62,6 +62,10 @@ struct Cli { #[arg(long, default_value = "10m")] op_ttl: String, + /// Prefix on env-backed Secret keys (for Helm secretManager.envPrefix). + #[arg(long, env = "FIREWALL_MANAGER_SECRET_ENV_PREFIX", default_value = "")] + env_prefix: String, + #[command(subcommand)] command: Command, } @@ -966,7 +970,8 @@ fn build_source_policy(cli: &Cli) -> Result { SourcePolicy::onepassword_connect(vault, cli.op_ttl.clone()) } } - }) + } + .with_env_prefix(cli.env_prefix.clone())) } fn role_identity(role: &RoleSpec, namespace: &str) -> IdentityInput { diff --git a/services/api-rs/crates/centaur-perms/src/tests.rs b/services/api-rs/crates/centaur-perms/src/tests.rs index f93ad99a9..3a93dbaba 100644 --- a/services/api-rs/crates/centaur-perms/src/tests.rs +++ b/services/api-rs/crates/centaur-perms/src/tests.rs @@ -1003,6 +1003,69 @@ fn real_slack_tool_parses_and_translates() { ), "expected the SLACK_BOT_TOKEN static secret" ); + + let etl_inputs = out + .inputs + .iter() + .filter_map(|input| match input { + SecretInput::Static(secret) if secret.name == "SLACK_ETL_TOKEN" => Some(secret), + _ => None, + }) + .collect::>(); + assert_eq!(etl_inputs.len(), 2); + for secret in &etl_inputs { + let replace = secret + .replace_config + .as_ref() + .expect("expected header replacement for the Slack ETL token"); + assert_eq!(replace.match_headers, vec!["Authorization".to_owned()]); + } + + let slack_api = etl_inputs + .iter() + .find(|secret| { + secret + .rules + .iter() + .any(|rule| rule.host.as_deref() == Some("slack.com")) + }) + .expect("expected Slack Web API ETL token rules"); + let mut slack_hosts = slack_api + .rules + .iter() + .map(|rule| rule.host.as_deref().unwrap_or_default().to_owned()) + .collect::>(); + slack_hosts.sort(); + assert_eq!( + slack_hosts, + vec!["slack.com".to_owned(), "www.slack.com".to_owned()] + ); + for rule in &slack_api.rules { + assert_eq!(rule.http_methods, vec!["GET".to_owned(), "POST".to_owned()]); + assert_eq!( + rule.paths, + vec![ + "/api/conversations.list".to_owned(), + "/api/conversations.history".to_owned(), + "/api/conversations.replies".to_owned(), + "/api/users.list".to_owned(), + ] + ); + } + + let files = etl_inputs + .iter() + .find(|secret| { + secret + .rules + .iter() + .any(|rule| rule.host.as_deref() == Some("files.slack.com")) + }) + .expect("expected Slack file download ETL token rule"); + assert_eq!(files.rules.len(), 1); + assert_eq!(files.rules[0].host.as_deref(), Some("files.slack.com")); + assert_eq!(files.rules[0].http_methods, vec!["GET".to_owned()]); + assert!(files.rules[0].paths.is_empty()); } #[test] diff --git a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/iron_proxy.rs b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/iron_proxy.rs index 0be0c87dd..eb7a895a0 100644 --- a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/iron_proxy.rs +++ b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/iron_proxy.rs @@ -9,7 +9,7 @@ use k8s_openapi::api::core::v1::{ SecurityContext, Service, ServicePort, ServiceSpec, Volume, VolumeMount, }; use k8s_openapi::api::networking::v1::{ - NetworkPolicy, NetworkPolicyEgressRule, NetworkPolicyIngressRule, NetworkPolicyPeer, + IPBlock, NetworkPolicy, NetworkPolicyEgressRule, NetworkPolicyIngressRule, NetworkPolicyPeer, NetworkPolicyPort, NetworkPolicySpec, }; use k8s_openapi::apimachinery::pkg::apis::meta::v1::{LabelSelector, ObjectMeta}; @@ -20,8 +20,8 @@ use serde_json::{Value, json}; use tokio::time::{Instant, sleep}; use crate::{ - AgentSandboxBackend, MANAGED_BY_LABEL, MANAGED_BY_VALUE, OtlpEgressTarget, SANDBOX_ID_LABEL, - is_not_found, map_kube_error, + API_SERVER_ENABLED_LABEL, AgentSandboxBackend, MANAGED_BY_LABEL, MANAGED_BY_VALUE, + OtlpEgressTarget, SANDBOX_ID_LABEL, is_not_found, map_kube_error, }; const IRON_PROXY_LABEL: &str = "centaur.ai/iron-proxy"; @@ -41,6 +41,7 @@ const PROXY_TLS_MODE: &str = "mitm"; const PROXY_TLS_CA_CERT_PATH: &str = "/etc/iron-proxy/ca.crt"; const PROXY_TLS_CA_KEY_PATH: &str = "/etc/iron-proxy/ca.key"; const PROXY_UPSTREAM_RESPONSE_HEADER_TIMEOUT: &str = "120s"; +const PROXY_UPSTREAM_DENY_CIDRS_ENV: &str = "IRON_PROXY_UPSTREAM_DENY_CIDRS"; const PROXY_LOG_LEVEL: &str = "info"; // iron-control multiplexes every Postgres upstream through a single listener, // routing by database name; the control plane owns each upstream DSN/role/ @@ -80,9 +81,11 @@ pub struct IronProxyConfig { pub ca_key_secret_name: String, pub env_from_secret_names: Vec, pub extra_env: BTreeMap, + pub upstream_deny_cidrs: Vec, pub op_connect_app_name: String, pub op_connect_port: u16, pub api_pod_labels: BTreeMap, + pub control_plane_pod_labels: BTreeMap, } impl IronProxyConfig { @@ -100,12 +103,17 @@ impl IronProxyConfig { ca_key_secret_name: ca_key_secret_name.into(), env_from_secret_names: Vec::new(), extra_env: BTreeMap::new(), + upstream_deny_cidrs: Vec::new(), op_connect_app_name: "onepassword-connect".to_owned(), op_connect_port: 8080, api_pod_labels: BTreeMap::from([( "app.kubernetes.io/component".to_owned(), "api".to_owned(), )]), + control_plane_pod_labels: BTreeMap::from([( + "app.kubernetes.io/component".to_owned(), + "console".to_owned(), + )]), } } } @@ -131,6 +139,8 @@ pub(crate) struct ResolvedIronProxy { // random per proxy pod. The claim barrier reads it back off the live pod // env, so it survives api-rs restarts and respects env overrides. management_api_key: String, + observability_enabled: bool, + api_server_enabled: bool, } /// The single Postgres listener the proxy multiplexes every upstream through. @@ -158,6 +168,11 @@ struct ProxySyncEnv { token: String, } +struct ControlPlaneEgressTarget { + peer: NetworkPolicyPeer, + port: u16, +} + impl AgentSandboxBackend { pub(crate) async fn resolve_iron_proxy( &self, @@ -189,6 +204,8 @@ impl AgentSandboxBackend { principal_id, pg, replace_placeholders, + spec.capabilities.observability_enabled, + spec.capabilities.api_server_enabled, ))) } @@ -261,11 +278,31 @@ impl AgentSandboxBackend { }; let pg = self.resolved_pg(); let replace_placeholders = self.effective_replace_placeholders(&principal_id).await?; + let observability_enabled = sandbox_observability_enabled(&sandbox, &self.config.container_name) + .unwrap_or_else(|| { + tracing::warn!( + sandbox_id = id.as_str(), + container_name = self.config.container_name.as_str(), + "sandbox observability capability env is missing or invalid; defaulting to enabled network policy" + ); + true + }); + let api_server_enabled = sandbox_api_server_enabled(&sandbox, &self.config.container_name) + .unwrap_or_else(|| { + tracing::warn!( + sandbox_id = id.as_str(), + container_name = self.config.container_name.as_str(), + "sandbox API server capability env is missing or invalid; defaulting to enabled network policy" + ); + true + }); Ok(Some(self.resolved_iron_proxy_for_principal( id, principal_id, pg, replace_placeholders, + observability_enabled, + api_server_enabled, ))) } @@ -275,6 +312,8 @@ impl AgentSandboxBackend { principal_id: String, pg: Option, replace_placeholders: BTreeMap, + observability_enabled: bool, + api_server_enabled: bool, ) -> ResolvedIronProxy { ResolvedIronProxy { proxy_host: iron_proxy_service_name(id), @@ -284,6 +323,8 @@ impl AgentSandboxBackend { pg, replace_placeholders, management_api_key: new_proxy_management_api_key(), + observability_enabled, + api_server_enabled, } } @@ -304,13 +345,18 @@ impl AgentSandboxBackend { ) .await .map_err(|err| map_kube_error("create iron-proxy service", err))?; - let control_port = url_port(&sync.control_url).unwrap_or(443); + let control_target = control_plane_egress_target( + &sync.control_url, + &self.config.namespace, + iron_proxy.control_plane_pod_labels.clone(), + ); for policy in build_iron_proxy_network_policies( id, resolved, iron_proxy, - control_port, + &control_target, self.config.otlp_egress.as_ref(), + resolved.observability_enabled, ) { self.network_policies() .create(&PostParams::default(), &policy) @@ -544,8 +590,36 @@ impl AgentSandboxBackend { let pg = self.resolved_pg_for_repair(sandbox.as_ref()); let principal_id = principal_id.to_owned(); let replace_placeholders = self.effective_replace_placeholders(&principal_id).await?; - let resolved = - self.resolved_iron_proxy_for_principal(id, principal_id, pg, replace_placeholders); + let observability_enabled = sandbox + .as_ref() + .and_then(|sandbox| sandbox_observability_enabled(sandbox, &self.config.container_name)) + .unwrap_or_else(|| { + tracing::warn!( + sandbox_id = id.as_str(), + container_name = self.config.container_name.as_str(), + "sandbox observability capability env is missing or invalid during proxy repair; defaulting to enabled network policy" + ); + true + }); + let api_server_enabled = sandbox + .as_ref() + .and_then(|sandbox| sandbox_api_server_enabled(sandbox, &self.config.container_name)) + .unwrap_or_else(|| { + tracing::warn!( + sandbox_id = id.as_str(), + container_name = self.config.container_name.as_str(), + "sandbox API server capability env is missing or invalid during proxy repair; defaulting to enabled network policy" + ); + true + }); + let resolved = self.resolved_iron_proxy_for_principal( + id, + principal_id, + pg, + replace_placeholders, + observability_enabled, + api_server_enabled, + ); self.create_iron_proxy_resources(id, Some(&resolved)) .await?; if let Some(sandbox) = sandbox @@ -1022,13 +1096,7 @@ pub(crate) fn apply_proxy_env(spec: &mut SandboxSpec, resolved: &ResolvedIronPro // collector; routing them through iron-proxy fails (plain-HTTP forwards // are rejected), so the endpoint host always bypasses the proxy. no_proxy_extra.extend(otlp_endpoint_hosts(spec)); - let api_host = env_value(spec, "CENTAUR_API_URL").and_then(host_from_url); - for (name, value) in proxy_env( - &resolved.proxy_host, - resolved.proxy_port, - api_host.as_deref(), - &no_proxy_extra, - ) { + for (name, value) in proxy_env(&resolved.proxy_host, resolved.proxy_port, &no_proxy_extra) { set_env(spec, &name, &value); } // Operator-granted replace placeholders: the sandbox sends the proxy_value @@ -1083,7 +1151,7 @@ fn build_iron_proxy_pod( Pod { metadata: object_meta_with_annotations( resolved.proxy_pod_name.clone(), - iron_proxy_labels(id), + iron_proxy_labels(id, resolved.api_server_enabled), annotations, ), spec: Some(PodSpec { @@ -1187,6 +1255,15 @@ fn iron_proxy_env_vars( ] { env.insert(name.to_owned(), env_var(name, &value)); } + if !iron_proxy.upstream_deny_cidrs.is_empty() { + env.insert( + PROXY_UPSTREAM_DENY_CIDRS_ENV.to_owned(), + env_var( + PROXY_UPSTREAM_DENY_CIDRS_ENV, + &iron_proxy.upstream_deny_cidrs.join(","), + ), + ); + } for (name, value) in &iron_proxy.extra_env { env.insert(name.clone(), env_var(name, value)); } @@ -1243,9 +1320,12 @@ fn build_iron_proxy_service(id: &SandboxId, resolved: &ResolvedIronProxy) -> Ser ports.push(service_port("pg", pg.port)); } Service { - metadata: object_meta(iron_proxy_service_name(id), iron_proxy_labels(id)), + metadata: object_meta( + iron_proxy_service_name(id), + iron_proxy_labels(id, resolved.api_server_enabled), + ), spec: Some(ServiceSpec { - selector: Some(iron_proxy_labels(id)), + selector: Some(iron_proxy_labels(id, resolved.api_server_enabled)), ports: Some(ports), ..Default::default() }), @@ -1257,30 +1337,18 @@ fn build_iron_proxy_network_policies( id: &SandboxId, resolved: &ResolvedIronProxy, iron_proxy: &IronProxyConfig, - control_port: u16, + control_target: &ControlPlaneEgressTarget, otlp_egress: Option<&OtlpEgressTarget>, + observability_enabled: bool, ) -> Vec { let sandbox_to_proxy_ports = sandbox_to_proxy_ports(resolved); - let mut sandbox_egress = vec![ + let sandbox_egress = vec![ egress_to( - vec![pod_peer(iron_proxy_labels(id))], + vec![pod_peer(iron_proxy_labels(id, resolved.api_server_enabled))], sandbox_to_proxy_ports.clone(), ), - egress_to( - vec![pod_peer(iron_proxy.api_pod_labels.clone())], - vec![network_port(8000), network_port(8080)], - ), dns_egress_rule(), ]; - if let Some(target) = otlp_egress { - // Direct harness OTLP export (codex usage/cost spans). The collector - // lives outside this namespace, so the sandbox bypasses iron-proxy for - // this one destination (the endpoint host also rides NO_PROXY). - sandbox_egress.push(egress_to( - vec![namespace_peer(&target.namespace)], - vec![network_port(target.port)], - )); - } vec![ NetworkPolicy { metadata: object_meta( @@ -1295,9 +1363,15 @@ fn build_iron_proxy_network_policies( }), }, NetworkPolicy { - metadata: object_meta(iron_proxy_policy_name(id), iron_proxy_labels(id)), + metadata: object_meta( + iron_proxy_policy_name(id), + iron_proxy_labels(id, resolved.api_server_enabled), + ), spec: Some(NetworkPolicySpec { - pod_selector: Some(label_selector(iron_proxy_labels(id))), + pod_selector: Some(label_selector(iron_proxy_labels( + id, + resolved.api_server_enabled, + ))), policy_types: Some(vec!["Ingress".to_owned(), "Egress".to_owned()]), ingress: Some(vec![ NetworkPolicyIngressRule { @@ -1311,7 +1385,12 @@ fn build_iron_proxy_network_policies( ports: Some(vec![network_port(PROXY_MANAGEMENT_PORT)]), }, ]), - egress: Some(proxy_egress_rules(iron_proxy, control_port)), + egress: Some(proxy_egress_rules( + iron_proxy, + control_target, + otlp_egress, + observability_enabled, + )), }), }, ] @@ -1325,25 +1404,37 @@ fn sandbox_to_proxy_ports(resolved: &ResolvedIronProxy) -> Vec, + observability_enabled: bool, ) -> Vec { // Upstream egress: 443/5432 for normal traffic, plus the iron-control port - // (deduped) so a sync-mode proxy can reach the control plane. - let mut upstream_ports = vec![network_port(443), network_port(5432)]; - if control_port != 443 && control_port != 5432 { - upstream_ports.push(network_port(control_port)); - } - let mut rules = vec![ - dns_egress_rule(), - egress_to( + // (deduped) so a sync-mode proxy can reach the control plane. Public + // upstreams are always constrained away from private/cluster CIDRs; any + // intra-cluster destination must be added as an explicit rule below. + let upstream_ports = vec![network_port(443), network_port(5432)]; + let mut rules = vec![dns_egress_rule()]; + rules.push(egress_to( + vec![control_target.peer.clone()], + vec![network_port(control_target.port)], + )); + rules.push(egress_to( + vec![all_namespaces_peer()], + vec![network_port(PG_LISTENER_PORT)], + )); + rules.push(egress_to(vec![public_ipv4_peer()], upstream_ports)); + if observability_enabled { + rules.push(egress_to( vec![pod_peer(iron_proxy.api_pod_labels.clone())], vec![network_port(8000), network_port(8080)], - ), - NetworkPolicyEgressRule { - ports: Some(upstream_ports), - ..Default::default() - }, - ]; + )); + if let Some(target) = otlp_egress { + rules.push(egress_to( + vec![namespace_peer(&target.namespace)], + vec![network_port(target.port)], + )); + } + } if matches!( iron_proxy.source_policy.kind, SourceKind::OnePasswordConnect @@ -1376,14 +1467,67 @@ fn namespace_peer(namespace: &str) -> NetworkPolicyPeer { } } +fn namespace_pod_peer(namespace: &str, labels: BTreeMap) -> NetworkPolicyPeer { + NetworkPolicyPeer { + namespace_selector: Some(label_selector(BTreeMap::from([( + "kubernetes.io/metadata.name".to_owned(), + namespace.to_owned(), + )]))), + pod_selector: Some(label_selector(labels)), + ..Default::default() + } +} + +fn all_namespaces_peer() -> NetworkPolicyPeer { + NetworkPolicyPeer { + namespace_selector: Some(LabelSelector::default()), + ..Default::default() + } +} + +fn public_ipv4_peer() -> NetworkPolicyPeer { + NetworkPolicyPeer { + ip_block: Some(IPBlock { + cidr: "0.0.0.0/0".to_owned(), + except: Some(vec![ + "0.0.0.0/8".to_owned(), + "10.0.0.0/8".to_owned(), + "100.64.0.0/10".to_owned(), + "127.0.0.0/8".to_owned(), + "169.254.0.0/16".to_owned(), + "172.16.0.0/12".to_owned(), + "192.0.0.0/24".to_owned(), + "192.0.2.0/24".to_owned(), + "192.168.0.0/16".to_owned(), + "198.18.0.0/15".to_owned(), + "198.51.100.0/24".to_owned(), + "203.0.113.0/24".to_owned(), + "224.0.0.0/4".to_owned(), + "240.0.0.0/4".to_owned(), + ]), + }), + ..Default::default() + } +} + +fn control_plane_egress_target( + control_url: &str, + default_namespace: &str, + control_plane_pod_labels: BTreeMap, +) -> ControlPlaneEgressTarget { + ControlPlaneEgressTarget { + peer: namespace_pod_peer(default_namespace, control_plane_pod_labels), + port: url_port(control_url).unwrap_or(443), + } +} + fn proxy_env( proxy_host: &str, proxy_port: u16, - api_host: Option<&str>, no_proxy_extra: &[String], ) -> BTreeMap { let proxy_url = format!("http://{proxy_host}:{proxy_port}"); - let no_proxy = no_proxy_value(proxy_host, api_host, no_proxy_extra); + let no_proxy = no_proxy_value(proxy_host, no_proxy_extra); BTreeMap::from([ ("FIREWALL_HOST".to_owned(), proxy_host.to_owned()), ("FIREWALL_PROXY_PORT".to_owned(), proxy_port.to_string()), @@ -1413,19 +1557,15 @@ fn proxy_env( ]) } -fn no_proxy_value(proxy_host: &str, api_host: Option<&str>, extra_values: &[String]) -> String { +fn no_proxy_value(proxy_host: &str, extra_values: &[String]) -> String { let mut hosts = BTreeSet::::from([ "localhost".to_owned(), "127.0.0.1".to_owned(), "::1".to_owned(), proxy_host.to_owned(), - "api".to_owned(), "victoriametrics".to_owned(), "victorialogs".to_owned(), ]); - if let Some(api_host) = api_host.filter(|value| !value.is_empty()) { - hosts.insert(api_host.to_owned()); - } for value in extra_values { hosts.extend( value @@ -1483,6 +1623,47 @@ fn pg_from_sandbox_env( pg_from_sandbox_dsn(dsn, listen, port) } +fn sandbox_observability_enabled( + sandbox: &crate::crd::Sandbox, + container_name: &str, +) -> Option { + sandbox_env_value( + sandbox, + "CENTAUR_SANDBOX_OBSERVABILITY_ENABLED", + container_name, + ) + .and_then(|value| value.parse().ok()) +} + +fn sandbox_api_server_enabled(sandbox: &crate::crd::Sandbox, container_name: &str) -> Option { + sandbox_env_value( + sandbox, + "CENTAUR_SANDBOX_API_SERVER_ENABLED", + container_name, + ) + .and_then(|value| value.parse().ok()) +} + +fn sandbox_env_value( + sandbox: &crate::crd::Sandbox, + name: &str, + fallback_container_name: &str, +) -> Option { + sandbox + .spec + .pod_template + .spec + .containers + .iter() + .find(|container| container.name == fallback_container_name) + .or_else(|| sandbox.spec.pod_template.spec.containers.first())? + .env + .as_ref()? + .iter() + .find(|env| env.name == name) + .and_then(|env| env.value.clone()) +} + fn pg_from_sandbox_dsn(dsn: &str, listen: &str, port: u16) -> Option { let rest = dsn .strip_prefix("postgresql://") @@ -1729,11 +1910,15 @@ fn sandbox_labels(id: &SandboxId) -> BTreeMap { ]) } -fn iron_proxy_labels(id: &SandboxId) -> BTreeMap { +fn iron_proxy_labels(id: &SandboxId, api_server_enabled: bool) -> BTreeMap { BTreeMap::from([ (MANAGED_BY_LABEL.to_owned(), MANAGED_BY_VALUE.to_owned()), (SANDBOX_ID_LABEL.to_owned(), id.as_str().to_owned()), (IRON_PROXY_LABEL.to_owned(), "true".to_owned()), + ( + API_SERVER_ENABLED_LABEL.to_owned(), + api_server_enabled.to_string(), + ), ]) } @@ -1758,9 +1943,46 @@ mod tests { pg: None, replace_placeholders: BTreeMap::new(), management_api_key: "test-management-key".to_owned(), + observability_enabled: true, + api_server_enabled: true, + } + } + + fn control_target() -> ControlPlaneEgressTarget { + ControlPlaneEgressTarget { + peer: namespace_pod_peer( + "centaur", + BTreeMap::from([( + "app.kubernetes.io/component".to_owned(), + "console".to_owned(), + )]), + ), + port: 3000, } } + fn peer_namespace(peer: &NetworkPolicyPeer) -> Option<&str> { + peer.namespace_selector + .as_ref()? + .match_labels + .as_ref()? + .get("kubernetes.io/metadata.name") + .map(String::as_str) + } + + fn peer_component(peer: &NetworkPolicyPeer) -> Option<&str> { + peer.pod_selector + .as_ref()? + .match_labels + .as_ref()? + .get("app.kubernetes.io/component") + .map(String::as_str) + } + + fn control_peer(target: &ControlPlaneEgressTarget) -> &NetworkPolicyPeer { + &target.peer + } + fn rule_allows_namespace_port( rule: &NetworkPolicyEgressRule, namespace: &str, @@ -1784,17 +2006,127 @@ mod tests { }) } + fn rule_allows_all_namespaces_port(rule: &NetworkPolicyEgressRule, port: u16) -> bool { + rule.to.as_ref().is_some_and(|peers| { + peers.iter().any(|peer| { + peer.namespace_selector + .as_ref() + .is_some_and(|selector| selector.match_labels.is_none()) + }) + }) && rule.ports.as_ref().is_some_and(|ports| { + ports + .iter() + .any(|policy_port| policy_port.port == Some(IntOrString::Int(i32::from(port)))) + }) + } + + fn rule_allows_public_port(rule: &NetworkPolicyEgressRule, port: u16) -> bool { + rule.to.as_ref().is_some_and(|peers| { + peers.iter().any(|peer| { + peer.ip_block + .as_ref() + .is_some_and(|block| block.cidr == "0.0.0.0/0") + }) + }) && rule.ports.as_ref().is_some_and(|ports| { + ports + .iter() + .any(|policy_port| policy_port.port == Some(IntOrString::Int(i32::from(port)))) + }) + } + + #[test] + fn control_plane_egress_target_uses_configured_namespace_and_labels() { + let target = control_plane_egress_target( + "http://prod-centaur-console:3000", + "centaur", + BTreeMap::from([( + "app.kubernetes.io/component".to_owned(), + "console".to_owned(), + )]), + ); + assert_eq!(target.port, 3000); + assert_eq!(peer_namespace(control_peer(&target)), Some("centaur")); + assert_eq!(peer_component(control_peer(&target)), Some("console")); + } + + #[test] + fn iron_proxy_labels_api_server_capability_when_enabled() { + let id = SandboxId::new("asbx-test"); + + assert_eq!( + iron_proxy_labels(&id, true) + .get(API_SERVER_ENABLED_LABEL) + .map(String::as_str), + Some("true") + ); + assert_eq!( + iron_proxy_labels(&id, false) + .get(API_SERVER_ENABLED_LABEL) + .map(String::as_str), + Some("false") + ); + } + + #[test] + fn iron_proxy_resources_carry_api_server_capability_label() { + let id = SandboxId::new("asbx-test"); + let iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); + let resolved = resolved(); + let sync = ProxySyncEnv { + proxy_id: "iprx_test".to_owned(), + control_url: "http://console:3000".to_owned(), + token: "proxy-token".to_owned(), + }; + + let pod = build_iron_proxy_pod(&id, &iron_proxy, &resolved, &sync); + assert_eq!( + pod.metadata + .labels + .as_ref() + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + + let service = build_iron_proxy_service(&id, &resolved); + assert_eq!( + service + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + assert_eq!( + service + .spec + .as_ref() + .and_then(|spec| spec.selector.as_ref()) + .and_then(|selector| selector.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + } + #[test] - fn sandbox_egress_policy_allows_otlp_collector_when_configured() { + fn sandbox_egress_policy_does_not_inline_otlp_collector_rule() { let id = SandboxId::new("asbx-test"); let iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); + let control_target = control_target(); let target = OtlpEgressTarget { namespace: "laminar".to_owned(), port: 8000, }; - let policies = - build_iron_proxy_network_policies(&id, &resolved(), &iron_proxy, 3000, Some(&target)); + let policies = build_iron_proxy_network_policies( + &id, + &resolved(), + &iron_proxy, + &control_target, + Some(&target), + true, + ); let sandbox_egress = policies[0] .spec .as_ref() @@ -1804,12 +2136,26 @@ mod tests { .unwrap() .clone(); assert!( - sandbox_egress + !sandbox_egress .iter() .any(|rule| rule_allows_namespace_port(rule, "laminar", 8000)) ); - - let policies = build_iron_proxy_network_policies(&id, &resolved(), &iron_proxy, 3000, None); + let proxy_egress = policies[1].spec.as_ref().unwrap().egress.as_ref().unwrap(); + assert!( + proxy_egress + .iter() + .any(|rule| rule_allows_namespace_port(rule, "laminar", 8000)) + ); + assert!(!proxy_egress.iter().any(|rule| rule.to.is_none())); + + let policies = build_iron_proxy_network_policies( + &id, + &resolved(), + &iron_proxy, + &control_target, + None, + true, + ); let sandbox_egress = policies[0] .spec .as_ref() @@ -1825,6 +2171,86 @@ mod tests { ); } + #[test] + fn restricted_sandbox_and_proxy_policies_block_internal_cluster_egress() { + let id = SandboxId::new("asbx-test"); + let iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); + let control_target = control_target(); + let target = OtlpEgressTarget { + namespace: "laminar".to_owned(), + port: 8000, + }; + + let policies = build_iron_proxy_network_policies( + &id, + &resolved(), + &iron_proxy, + &control_target, + Some(&target), + false, + ); + let sandbox_egress = policies[0].spec.as_ref().unwrap().egress.as_ref().unwrap(); + assert!( + !sandbox_egress + .iter() + .any(|rule| rule_allows_namespace_port(rule, "laminar", 8000)) + ); + assert!(!sandbox_egress.iter().any(|rule| { + rule.to.as_ref().is_some_and(|peers| { + peers.iter().any(|peer| { + peer.pod_selector.as_ref().is_some_and(|selector| { + selector.match_labels.as_ref() == Some(&iron_proxy.api_pod_labels) + }) + }) + }) + })); + + let proxy_egress = policies[1].spec.as_ref().unwrap().egress.as_ref().unwrap(); + assert!( + !proxy_egress + .iter() + .any(|rule| rule_allows_namespace_port(rule, "laminar", 8000)) + ); + assert!(!proxy_egress.iter().any(|rule| { + rule.to.as_ref().is_some_and(|peers| { + peers.iter().any(|peer| { + peer.pod_selector.as_ref().is_some_and(|selector| { + selector.match_labels.as_ref() == Some(&iron_proxy.api_pod_labels) + }) + }) + }) + })); + assert!(proxy_egress.iter().any(|rule| { + rule.to.as_ref().is_some_and(|peers| { + peers.iter().any(|peer| { + peer.ip_block.as_ref().is_some_and(|block| { + block.cidr == "0.0.0.0/0" + && block.except.as_ref().is_some_and(|except| { + except.iter().any(|cidr| cidr == "10.0.0.0/8") + && except.iter().any(|cidr| cidr == "172.16.0.0/12") + && except.iter().any(|cidr| cidr == "192.168.0.0/16") + }) + }) + }) + }) + })); + assert!( + proxy_egress + .iter() + .any(|rule| rule_allows_namespace_port(rule, "centaur", 3000)) + ); + assert!( + proxy_egress + .iter() + .any(|rule| rule_allows_all_namespaces_port(rule, 5432)) + ); + assert!( + !proxy_egress + .iter() + .any(|rule| rule_allows_public_port(rule, 3000)) + ); + } + #[test] fn managed_proxy_env_sets_response_header_timeout() { let iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); @@ -1843,6 +2269,33 @@ mod tests { assert_eq!(timeout, Some("120s")); } + #[test] + fn managed_proxy_env_sets_upstream_deny_cidrs() { + let mut iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); + iron_proxy.upstream_deny_cidrs = vec![ + "169.254.169.254/32".to_owned(), + "127.0.0.0/8".to_owned(), + "10.42.0.0/16".to_owned(), + "10.43.0.0/16".to_owned(), + ]; + let sync = ProxySyncEnv { + proxy_id: "proxy-id".to_owned(), + control_url: "http://iron-control".to_owned(), + token: "proxy-token".to_owned(), + }; + + let env = iron_proxy_env_vars(&iron_proxy, &resolved(), &sync); + let deny_cidrs = env + .iter() + .find(|var| var.name == PROXY_UPSTREAM_DENY_CIDRS_ENV) + .and_then(|var| var.value.as_deref()); + + assert_eq!( + deny_cidrs, + Some("169.254.169.254/32,127.0.0.0/8,10.42.0.0/16,10.43.0.0/16") + ); + } + #[test] fn pg_repair_reuses_credentials_from_existing_sandbox_dsn() { let pg = pg_from_sandbox_dsn( @@ -1869,7 +2322,15 @@ mod tests { let id = SandboxId::new("asbx-test"); let iron_proxy = IronProxyConfig::new("proxy:test", "ca-cert", "ca-key"); - let policies = build_iron_proxy_network_policies(&id, &resolved(), &iron_proxy, 3000, None); + let control_target = control_target(); + let policies = build_iron_proxy_network_policies( + &id, + &resolved(), + &iron_proxy, + &control_target, + None, + true, + ); let ingress = policies[1] .spec .as_ref() @@ -2124,6 +2585,32 @@ mod tests { assert_eq!(ack, ProxyAck::ManagementUnavailable); } + #[test] + fn apply_proxy_env_does_not_add_api_host_to_no_proxy() { + let mut spec = SandboxSpec::new("centaur-agent:latest") + .env("CENTAUR_API_URL", "http://api:8080") + .env("NO_PROXY", "custom.internal"); + + apply_proxy_env(&mut spec, &resolved()); + + for name in ["NO_PROXY", "no_proxy"] { + let value = spec + .env + .iter() + .find(|env| env.name == name) + .map(|env| env.value.clone()) + .unwrap(); + assert!( + !value.split(',').any(|host| host == "api"), + "{name} should not contain the API host: {value}" + ); + assert!( + value.split(',').any(|host| host == "custom.internal"), + "{name} should preserve explicit NO_PROXY extras: {value}" + ); + } + } + #[test] fn proxy_fallback_delay_subtracts_elapsed_probe_time() { assert_eq!( diff --git a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/lib.rs b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/lib.rs index 97f1a0701..1f7d6d73b 100644 --- a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/lib.rs +++ b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/lib.rs @@ -12,8 +12,8 @@ use std::time::{Duration, SystemTime, UNIX_EPOCH}; use async_trait::async_trait; use centaur_iron_control::IronControlClient; use centaur_sandbox_core::{ - MountKind, ObservedSandbox, SandboxBackend, SandboxError, SandboxHandle, SandboxId, SandboxIo, - SandboxResult, SandboxSpec, SandboxStatus, + MountKind, ObservedSandbox, RepoCacheAccess, SandboxBackend, SandboxError, SandboxHandle, + SandboxId, SandboxIo, SandboxResult, SandboxSpec, SandboxStatus, }; use k8s_openapi::api::core::v1::{PersistentVolumeClaim, Pod}; use kube::api::{ @@ -37,15 +37,15 @@ const BACKEND_NAME: &str = "agent-sandbox-k8s"; const DEFAULT_CONTAINER_NAME: &str = "agent"; const MANAGED_BY_LABEL: &str = "centaur.ai/managed-by"; const SANDBOX_ID_LABEL: &str = "centaur.ai/sandbox-id"; +const OBSERVABILITY_ENABLED_LABEL: &str = "centaur.ai/observability-enabled"; +const API_SERVER_ENABLED_LABEL: &str = "centaur.ai/api-server-enabled"; const MANAGED_BY_VALUE: &str = "api-rs"; // iron-control principal OID the sandbox's proxy binds to, stamped at create // so resume (which has only the sandbox id) can rebind without the spec or any // in-memory state. Survives pause and api-rs restarts. const IRON_CONTROL_PRINCIPAL_ANNOTATION: &str = "centaur.ai/iron-control-principal"; -// RFC 3339 instant stamped when the sandbox is paused for idleness and -// cleared on resume. The reaper uses it to stop sandboxes whose pause -// outlived the idle TTL, surviving api-rs restarts (the pause timer is -// otherwise in-memory only). +// RFC 3339 instant stamped when the sandbox is paused for idleness and cleared +// on resume. This keeps suspended status observable across api-rs restarts. const PAUSED_AT_ANNOTATION: &str = "centaur.ai/paused-at"; static NEXT_ID: AtomicU64 = AtomicU64::new(1); @@ -66,14 +66,13 @@ pub struct AgentSandboxConfig { /// git-clones the tools repo into the agent's `/app/tools`, and `TOOL_DIRS` /// is set so the agent's shim installer finds them. pub tools: Option, - /// Optional org/deployment overlay image copied into each sandbox before - /// the agent starts. The overlay can carry tools, workflows, prompts, and - /// harness config without rebuilding the base sandbox image. + /// Transitional org/deployment overlay image. Its contents are copied into + /// an emptyDir before the agent starts so staged rollouts can coexist with + /// the newer repo-cache overlay architecture and roll back safely. pub overlay_image: Option, - /// In-cluster OTLP collector (e.g. Laminar) the sandbox exports harness - /// traces to directly. The per-sandbox egress NetworkPolicy denies all - /// destinations except the proxy/control plane, so without this rule the - /// harness's usage/cost spans never leave the pod. + /// In-cluster OTLP collector (e.g. Laminar) used for observability-capable + /// sandboxes. Sandbox pod egress is granted by chart-level label policy; + /// the per-sandbox proxy uses this target for its own explicit egress. pub otlp_egress: Option, pub ready_timeout: Duration, } @@ -617,6 +616,16 @@ fn build_agent_sandbox( labels.extend(spec.labels.clone()); labels.insert(MANAGED_BY_LABEL.to_owned(), MANAGED_BY_VALUE.to_owned()); labels.insert(SANDBOX_ID_LABEL.to_owned(), id.as_str().to_owned()); + if spec.capabilities.observability_enabled { + labels.insert(OBSERVABILITY_ENABLED_LABEL.to_owned(), "true".to_owned()); + } + // Always project the capability. The transitional NetworkPolicy can then + // distinguish genuinely legacy pods (label absent) from new restricted + // pods (label explicitly false). + labels.insert( + API_SERVER_ENABLED_LABEL.to_owned(), + spec.capabilities.api_server_enabled.to_string(), + ); let mut pod_labels = labels.clone(); pod_labels.insert( @@ -650,13 +659,29 @@ fn build_agent_sandbox( .iter() .map(|env| (env.name.clone(), env.value.clone())) .collect(); - let repo_cache_tools = config + let repo_cache_enabled = spec.capabilities.repo_cache.enabled(); + let scoped_tools = config .tools .as_ref() - .filter(|_| spec.capabilities.repo_cache_enabled); - let baked_base_tools = config.tools.is_some() && !spec.capabilities.repo_cache_enabled; + .filter(|_| repo_cache_enabled) + .map(|tools| tools.scoped_for_repo_cache_access(&spec.capabilities.repo_cache)); + let repo_cache_tools = scoped_tools.as_ref().filter(|tools| tools.has_sources()); + let baked_base_tools = config.tools.is_some() && repo_cache_tools.is_none(); + // The legacy image carries private org overlays and predates capability + // scoping. Expose it only to full repo-cache principals; None/Public must + // not regain private tools, prompts, skills, or workflows through this + // transitional fallback. + let overlay_image = config + .overlay_image + .as_ref() + .filter(|_| matches!(spec.capabilities.repo_cache, RepoCacheAccess::All)); if repo_cache_tools.is_some() { + // Workflow-host specs can inherit the API deployment's transitional + // TOOLS_OVERLAY_PATH. Remove it, even when it arrived before this + // function, because entrypoint.sh appends it after TOOL_DIRS and would + // let stale image tools shadow the reviewed repo-cache bootstrap. + agent_env.retain(|(name, _)| name != "TOOLS_OVERLAY_PATH"); for (name, value) in tools::agent_env(repo_cache_tools) { upsert_env(&mut agent_env, &name, value); } @@ -665,17 +690,28 @@ fn build_agent_sandbox( upsert_env(&mut agent_env, &name, value); } } - if let Some(overlay_image) = &config.overlay_image { - upsert_env( + if let Some(overlay_image) = overlay_image { + insert_env_if_absent( &mut agent_env, - "CENTAUR_OVERLAY_DIR", + "CENTAUR_IMAGE_OVERLAY_DIR", overlay_image.mount_path.clone(), ); - upsert_env( + insert_env_if_absent( &mut agent_env, - "TOOLS_OVERLAY_PATH", - format!("{}/tools", overlay_image.mount_path.trim_end_matches('/')), + "CENTAUR_OVERLAY_DIR", + overlay_image.mount_path.clone(), ); + // Repo-cache tools are published into TOOL_DIRS and are the reviewed + // source of truth. Keep the image path only as a fallback when no repo + // tool source is available; otherwise entrypoint.sh would append the + // stale image tree after TOOL_DIRS and let it shadow repo-backed tools. + if repo_cache_tools.is_none() { + insert_env_if_absent( + &mut agent_env, + "TOOLS_OVERLAY_PATH", + format!("{}/tools", overlay_image.mount_path.trim_end_matches('/')), + ); + } } insert_optional( &mut container, @@ -709,7 +745,7 @@ fn build_agent_sandbox( volume_mounts.extend(tools::agent_volume_mounts_json(repo_cache_tools)); volumes.extend(tools::volumes_json(repo_cache_tools)); } - if let Some(overlay_image) = &config.overlay_image { + if let Some(overlay_image) = overlay_image { volume_mounts.push(json!({ "name": "overlay-root", "mountPath": overlay_image.mount_path, @@ -747,7 +783,7 @@ fn build_agent_sandbox( clone_proxy.as_ref(), )); } - if let Some(overlay_image) = &config.overlay_image { + if let Some(overlay_image) = overlay_image { init_containers.push(overlay_init_container_json(overlay_image)); } @@ -757,7 +793,7 @@ fn build_agent_sandbox( "automountServiceAccountToken": false, "enableServiceLinks": false, }); - if repo_cache_tools.is_some() || config.overlay_image.is_some() { + if repo_cache_tools.is_some() || overlay_image.is_some() { pod_spec["securityContext"] = tools::pod_security_context_json(); } insert_optional( @@ -851,6 +887,11 @@ fn mount_json(spec: &SandboxSpec) -> (Vec, Vec) { "mountPath": mount.target_path, "readOnly": mount.read_only, })); + if let Some(sub_path) = &mount.sub_path + && let Some(mount_obj) = mounts.last_mut().and_then(Value::as_object_mut) + { + mount_obj.insert("subPath".to_owned(), json!(sub_path)); + } volumes.push(match &mount.kind { MountKind::EmptyDir => json!({ "name": name, @@ -931,6 +972,15 @@ fn upsert_env(env: &mut Vec<(String, String)>, name: &str, value: String) { } } +/// Append an env default without replacing an operator-supplied value in +/// `spec.env`. Transitional image wiring is a fallback, while explicit +/// repo-backed paths are authoritative. +fn insert_env_if_absent(env: &mut Vec<(String, String)>, name: &str, value: String) { + if !env.iter().any(|(existing, _)| existing == name) { + env.push((name.to_owned(), value)); + } +} + fn next_sandbox_name() -> String { let millis = SystemTime::now() .duration_since(UNIX_EPOCH) @@ -954,7 +1004,7 @@ fn map_kube_error(operation: &str, err: Error) -> SandboxError { #[cfg(test)] mod tests { - use centaur_sandbox_core::{ResourceLimits, SandboxCapabilities, SandboxSpec}; + use centaur_sandbox_core::{RepoCacheAccess, ResourceLimits, SandboxCapabilities, SandboxSpec}; use k8s_openapi::api::core::v1::{PodCondition, PodStatus}; use super::*; @@ -1000,6 +1050,108 @@ mod tests { assert!(container.resources.as_ref().unwrap().limits.is_some()); } + #[test] + fn labels_observability_enabled_sandboxes_for_chart_policy() { + let spec = SandboxSpec::new("centaur-agent:latest").capabilities(SandboxCapabilities { + repo_cache: RepoCacheAccess::All, + observability_enabled: true, + api_server_enabled: true, + }); + let config = AgentSandboxConfig::new("centaur"); + + let sandbox = build_agent_sandbox(&SandboxId::new("asbx-test"), &spec, &config).unwrap(); + + assert_eq!( + sandbox + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(OBSERVABILITY_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + assert_eq!( + sandbox + .spec + .pod_template + .metadata + .as_ref() + .and_then(|metadata| metadata.labels.as_ref()) + .and_then(|labels| labels.get(OBSERVABILITY_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + assert_eq!( + sandbox + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + assert_eq!( + sandbox + .spec + .pod_template + .metadata + .as_ref() + .and_then(|metadata| metadata.labels.as_ref()) + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("true") + ); + } + + #[test] + fn labels_api_server_capability_false_for_restricted_sandboxes() { + let spec = SandboxSpec::new("centaur-agent:latest").capabilities(SandboxCapabilities { + repo_cache: RepoCacheAccess::All, + observability_enabled: false, + api_server_enabled: false, + }); + let config = AgentSandboxConfig::new("centaur"); + + let sandbox = build_agent_sandbox(&SandboxId::new("asbx-test"), &spec, &config).unwrap(); + + assert!( + sandbox + .metadata + .labels + .as_ref() + .is_none_or(|labels| !labels.contains_key(OBSERVABILITY_ENABLED_LABEL)) + ); + assert!( + sandbox + .spec + .pod_template + .metadata + .as_ref() + .and_then(|metadata| metadata.labels.as_ref()) + .is_none_or(|labels| !labels.contains_key(OBSERVABILITY_ENABLED_LABEL)) + ); + assert_eq!( + sandbox + .metadata + .labels + .as_ref() + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("false") + ); + assert_eq!( + sandbox + .spec + .pod_template + .metadata + .as_ref() + .and_then(|metadata| metadata.labels.as_ref()) + .and_then(|labels| labels.get(API_SERVER_ENABLED_LABEL)) + .map(String::as_str), + Some("false") + ); + } + #[test] fn tools_clone_rides_iron_proxy_when_enabled() { // apply_proxy_env runs before build_agent_sandbox in create(), so the @@ -1084,6 +1236,10 @@ mod tests { env.name == "CENTAUR_OVERLAY_DIR" && env.value.as_deref() == Some("/home/agent/overlay/org") })); + assert!(env.iter().any(|env| { + env.name == "CENTAUR_IMAGE_OVERLAY_DIR" + && env.value.as_deref() == Some("/home/agent/overlay/org") + })); assert!(env.iter().any(|env| { env.name == "TOOLS_OVERLAY_PATH" && env.value.as_deref() == Some("/home/agent/overlay/org/tools") @@ -1108,11 +1264,51 @@ mod tests { ); } + #[test] + fn repo_backed_overlay_env_wins_over_transitional_image_defaults() { + let spec = SandboxSpec::new("centaur-agent:latest") + .env( + "CENTAUR_OVERLAY_DIR", + "/home/agent/github/TipLink/fineas-centaur-overlay", + ) + .env("TOOLS_OVERLAY_PATH", "/app/overlay/org/tools"); + let config = AgentSandboxConfig::new("centaur") + .tools(ToolsConfig::new("paradigmxyz/centaur", "api:test")) + .overlay_image( + OverlayImageConfig::new("ghcr.io/example/overlay:sha-test") + .mount_path("/home/agent/overlay/org"), + ); + + let sandbox = build_agent_sandbox(&SandboxId::new("asbx-test"), &spec, &config).unwrap(); + let pod_spec = &sandbox.spec.pod_template.spec; + let env = pod_spec.containers[0].env.as_ref().unwrap(); + + assert_eq!( + env.iter() + .find(|entry| entry.name == "CENTAUR_OVERLAY_DIR") + .and_then(|entry| entry.value.as_deref()), + Some("/home/agent/github/TipLink/fineas-centaur-overlay") + ); + assert_eq!( + env.iter() + .find(|entry| entry.name == "TOOL_DIRS") + .and_then(|entry| entry.value.as_deref()), + Some("/app/tools") + ); + assert!(!env.iter().any(|entry| entry.name == "TOOLS_OVERLAY_PATH")); + assert!(pod_spec.init_containers.as_ref().is_some_and(|containers| { + containers + .iter() + .any(|container| container.name == "overlay-bootstrap") + })); + } + #[test] fn disabled_repo_cache_uses_baked_base_tools_without_bootstrap() { let spec = SandboxSpec::new("centaur-agent:latest").capabilities(SandboxCapabilities { - repo_cache_enabled: false, + repo_cache: RepoCacheAccess::None, observability_enabled: true, + api_server_enabled: true, }); let mut tools = ToolsConfig::new("paradigmxyz/centaur", "api:test"); tools.repo_cache_path = Some("/var/lib/centaur/repos".to_owned()); @@ -1166,6 +1362,48 @@ mod tests { ); } + #[test] + fn restricted_repo_cache_access_cannot_fall_back_to_private_overlay_image() { + for repo_cache in [RepoCacheAccess::None, RepoCacheAccess::Public] { + let spec = SandboxSpec::new("centaur-agent:latest").capabilities(SandboxCapabilities { + repo_cache, + observability_enabled: true, + api_server_enabled: true, + }); + let config = AgentSandboxConfig::new("centaur") + .tools(ToolsConfig::new("private-org/centaur", "api:test")) + .overlay_image( + OverlayImageConfig::new("ghcr.io/private-org/overlay:sha-test") + .mount_path("/home/agent/overlay/org"), + ); + + let sandbox = + build_agent_sandbox(&SandboxId::new("asbx-test"), &spec, &config).unwrap(); + let pod_spec = &sandbox.spec.pod_template.spec; + let container = &pod_spec.containers[0]; + let env = container.env.as_ref().unwrap(); + + assert!(!env.iter().any(|entry| { + entry.name == "CENTAUR_OVERLAY_DIR" + || entry.name == "CENTAUR_IMAGE_OVERLAY_DIR" + || entry.name == "TOOLS_OVERLAY_PATH" + })); + assert!( + container.volume_mounts.as_ref().is_none_or(|mounts| { + !mounts.iter().any(|mount| mount.name == "overlay-root") + }) + ); + assert!(pod_spec.init_containers.as_ref().is_none_or(|containers| { + !containers + .iter() + .any(|container| container.name == "overlay-bootstrap") + })); + assert!(pod_spec.volumes.as_ref().is_none_or(|volumes| { + !volumes.iter().any(|volume| volume.name == "overlay-root") + })); + } + } + #[test] fn maps_agent_sandbox_replicas_and_pod_readiness_to_status() { let ready_pod = pod_with_phase_and_ready("Running", true); diff --git a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/tools.rs b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/tools.rs index 3e5aaeba0..99b2815a0 100644 --- a/services/api-rs/crates/centaur-sandbox-agent-k8s/src/tools.rs +++ b/services/api-rs/crates/centaur-sandbox-agent-k8s/src/tools.rs @@ -22,6 +22,7 @@ //! republish from repo-cache or fetch the configured ref, then reinstall shims //! without restarting the pod. +use centaur_sandbox_core::RepoCacheAccess; use serde_json::{Value, json}; const AGENT_UID: i64 = 1001; @@ -43,6 +44,8 @@ const GITHUB_TOKEN_DIR: &str = "/tools-github-token"; const GITHUB_TOKEN_FILE: &str = "token"; const GITHUB_TOKEN_FILE_PATH: &str = "/tools-github-token/token"; const REPO_CACHE_VOLUME: &str = "tools-repo-cache"; +const PUBLIC_VISIBILITY: &str = "public"; +const PRIVATE_VISIBILITY: &str = "private"; /// Git source for the base tools tree. When set, every sandbox gets a /// `tools-bootstrap` init container that clones `repo` at `git_ref` and @@ -56,6 +59,8 @@ pub struct ToolsConfig { pub git_ref: Option, /// Subdirectory within the repo holding the tools (published to `/app/tools`). pub source_subdir: String, + /// Repo visibility. Public sandboxes only publish public sources. + pub visibility: String, /// Image the clone init container runs. It must include git and /// `install-tool-shims` (the default sandbox image does). pub image: String, @@ -71,8 +76,14 @@ pub struct ToolsConfig { /// Optional PVC backing for `repo_cache_path`. This lets Autopilot clusters use /// repoCache without hostPath volumes. pub repo_cache_pvc: Option, - /// Additional tool sources copied after the base tree. Later sources replace - /// earlier packages with the same tool name so overlays can override base tools. + /// Optional subdirectory within the repo-cache volume to expose at + /// `repo_cache_path`. Used for public-only repo-cache access. + pub repo_cache_sub_path: Option, + /// Whether running sandboxes should watch repo-cache checkouts and refresh + /// local tool shims when commits change. + pub auto_reload: bool, + /// Additional tool sources copied after the base tree. Duplicate tool names + /// are skipped by the copy helper. pub extra_sources: Vec, } @@ -82,6 +93,13 @@ pub struct ToolSource { pub repo: String, pub git_ref: Option, pub source_subdir: String, + pub visibility: String, +} + +impl ToolSource { + fn is_public(&self) -> bool { + self.visibility == PUBLIC_VISIBILITY + } } /// A Kubernetes Secret key holding a GitHub token, fed to `git` via `GIT_ASKPASS`. @@ -97,11 +115,14 @@ impl ToolsConfig { repo: repo.into(), git_ref: None, source_subdir: "tools".to_owned(), + visibility: PRIVATE_VISIBILITY.to_owned(), image: image.into(), image_pull_policy: None, github_token: None, repo_cache_path: None, repo_cache_pvc: None, + repo_cache_sub_path: None, + auto_reload: true, extra_sources: Vec::new(), } } @@ -111,10 +132,69 @@ impl ToolsConfig { repo: self.repo.clone(), git_ref: self.git_ref.clone(), source_subdir: self.source_subdir.clone(), + visibility: self.visibility.clone(), }]; sources.extend(self.extra_sources.clone()); sources } + + pub(crate) fn has_sources(&self) -> bool { + !self.repo.is_empty() + } + + pub fn scoped_for_repo_cache_access(&self, access: &RepoCacheAccess) -> Self { + let mut scoped = self.clone(); + scoped.repo_cache_sub_path = match access { + RepoCacheAccess::Public => Some(PUBLIC_VISIBILITY.to_owned()), + RepoCacheAccess::None | RepoCacheAccess::All => None, + }; + if matches!(access, RepoCacheAccess::Public) { + scoped.extra_sources.retain(ToolSource::is_public); + if scoped.visibility != PUBLIC_VISIBILITY { + if let Some(first_public) = scoped.extra_sources.first().cloned() { + scoped.repo = first_public.repo; + scoped.git_ref = first_public.git_ref; + scoped.source_subdir = first_public.source_subdir; + scoped.visibility = first_public.visibility; + scoped.extra_sources.remove(0); + } else { + scoped.repo.clear(); + } + } + } + scoped + } + + fn repo_cache_source_path(&self) -> Option { + let repo_cache_path = self.repo_cache_path.as_ref()?; + if self.repo_cache_pvc.is_some() { + return Some(repo_cache_path.clone()); + } + Some(match &self.repo_cache_sub_path { + Some(sub_path) => format!( + "{}/{}", + repo_cache_path.trim_end_matches('/'), + sub_path.trim_start_matches('/') + ), + None => repo_cache_path.clone(), + }) + } + + fn repo_cache_volume_mount(&self) -> Option { + let repo_cache_path = self.repo_cache_path.as_ref()?; + let mut mount = json!({ + "name": REPO_CACHE_VOLUME, + "mountPath": repo_cache_path, + "readOnly": true, + }); + if self.repo_cache_pvc.is_some() + && let Some(sub_path) = &self.repo_cache_sub_path + && let Some(obj) = mount.as_object_mut() + { + obj.insert("subPath".to_owned(), json!(sub_path)); + } + Some(mount) + } } pub(crate) fn security_context_json() -> Value { @@ -149,6 +229,12 @@ pub(crate) fn baked_base_tool_dirs() -> String { /// Agent env added for tools wiring. pub(crate) fn agent_env(tools: Option<&ToolsConfig>) -> Vec<(String, String)> { let mut env = vec![("TOOL_DIRS".to_owned(), agent_tool_dirs())]; + if let Some(tools) = tools { + env.push(( + "CENTAUR_TOOLS_AUTO_RELOAD".to_owned(), + tools.auto_reload.to_string(), + )); + } if tools .and_then(|tools| tools.github_token.as_ref()) .is_some() @@ -338,12 +424,8 @@ CENTAUR_TOOLS_METADATA" if let Some(proxy) = clone_proxy { volume_mounts.push(proxy.ca_volume_mount.clone()); } - if let Some(repo_cache_path) = &tools.repo_cache_path { - volume_mounts.push(json!({ - "name": REPO_CACHE_VOLUME, - "mountPath": repo_cache_path, - "readOnly": true, - })); + if let Some(mount) = tools.repo_cache_volume_mount() { + volume_mounts.push(mount); } let mut container = json!({ @@ -387,7 +469,7 @@ pub(crate) fn volumes_json(tools: Option<&ToolsConfig>) -> Vec { volumes.push(json!({ "name": REPO_CACHE_VOLUME, "hostPath": { - "path": repo_cache_path, + "path": tools.repo_cache_source_path().unwrap_or_else(|| repo_cache_path.clone()), "type": "DirectoryOrCreate", }, })); @@ -412,12 +494,8 @@ pub(crate) fn agent_volume_mounts_json(tools: Option<&ToolsConfig>) -> Vec bool { + !matches!(self, Self::None) + } + + pub fn as_str(&self) -> &'static str { + match self { + Self::None => "none", + Self::Public => "public", + Self::All => "all", + } + } +} + #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] pub struct SandboxCapabilities { - pub repo_cache_enabled: bool, + #[serde(default)] + pub repo_cache: RepoCacheAccess, pub observability_enabled: bool, + pub api_server_enabled: bool, } impl SandboxCapabilities { pub const fn default_enabled() -> Self { Self { - repo_cache_enabled: true, + repo_cache: RepoCacheAccess::All, observability_enabled: true, + api_server_enabled: true, } } - pub const fn is_default_enabled(&self) -> bool { - self.repo_cache_enabled && self.observability_enabled + pub fn is_default_enabled(&self) -> bool { + self.repo_cache.enabled() && self.observability_enabled && self.api_server_enabled } } @@ -127,6 +153,8 @@ pub struct Mount { pub kind: MountKind, pub target_path: String, pub read_only: bool, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub sub_path: Option, } impl Mount { @@ -135,6 +163,7 @@ impl Mount { kind, target_path: target_path.into(), read_only: false, + sub_path: None, } } @@ -142,6 +171,11 @@ impl Mount { self.read_only = true; self } + + pub fn sub_path(mut self, sub_path: impl Into) -> Self { + self.sub_path = Some(sub_path.into()); + self + } } #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] diff --git a/services/api-rs/crates/centaur-sandbox-manager/Cargo.toml b/services/api-rs/crates/centaur-sandbox-manager/Cargo.toml index 9849842b9..d184dd9f3 100644 --- a/services/api-rs/crates/centaur-sandbox-manager/Cargo.toml +++ b/services/api-rs/crates/centaur-sandbox-manager/Cargo.toml @@ -15,6 +15,9 @@ tracing.workspace = true [dev-dependencies] async-trait.workspace = true +centaur-session-core.workspace = true +serde_json.workspace = true +sqlx.workspace = true tokio = { version = "1", features = ["macros", "rt", "sync"] } [lints] diff --git a/services/api-rs/crates/centaur-sandbox-manager/src/reaper.rs b/services/api-rs/crates/centaur-sandbox-manager/src/reaper.rs index b058467af..80462b8bf 100644 --- a/services/api-rs/crates/centaur-sandbox-manager/src/reaper.rs +++ b/services/api-rs/crates/centaur-sandbox-manager/src/reaper.rs @@ -1,18 +1,18 @@ //! Background garbage collection for leaked sandboxes. //! -//! Sessions pause idle sandboxes (replicas to zero) but nothing stops them: -//! the pause timer lives in process memory and dies with the deploy, and -//! sandboxes whose sessions never go idle are retained forever. The reaper is -//! the restart-surviving backstop: it sweeps the backend's observed sandboxes -//! and stops any that outlived their welcome, releasing the sandbox, its -//! proxy resources, and its node pod slots. +//! Sessions pause idle sandboxes (replicas to zero), but paused sandboxes and +//! sandboxes whose sessions never go idle still need a restart-surviving +//! backstop. The reaper sweeps the backend's observed sandboxes and stops any +//! that exceed the configured max lifetime, releasing the sandbox, its proxy +//! resources, and its node pod slots. use std::{ sync::Arc, time::{Duration, SystemTime}, }; -use centaur_sandbox_core::{ObservedSandbox, SandboxResult, SandboxStatus}; +use centaur_sandbox_core::ObservedSandbox; +use centaur_sandbox_core::SandboxResult; use tokio::time::{MissedTickBehavior, interval}; use tracing::{info, warn}; @@ -22,9 +22,6 @@ use crate::SandboxManager; pub struct SandboxReaperConfig { /// How often to sweep. pub interval: Duration, - /// Stop sandboxes that have been suspended longer than this. `None` - /// disables the idle sweep. - pub idle_ttl: Option, /// Stop any sandbox older than this regardless of status. `None` disables /// the max-lifetime sweep. pub max_lifetime: Option, @@ -32,7 +29,7 @@ pub struct SandboxReaperConfig { impl SandboxReaperConfig { pub fn is_enabled(&self) -> bool { - self.idle_ttl.is_some() || self.max_lifetime.is_some() + self.max_lifetime.is_some() } } @@ -99,14 +96,6 @@ fn reap_reason( if observed.status.is_terminal() { return None; } - if let (Some(idle_ttl), Some(suspended_since)) = (config.idle_ttl, observed.suspended_since) - && matches!(observed.status, SandboxStatus::Suspended) - && now - .duration_since(suspended_since) - .is_ok_and(|age| age >= idle_ttl) - { - return Some("idle_ttl"); - } if let (Some(max_lifetime), Some(created_at)) = (config.max_lifetime, observed.created_at) && now .duration_since(created_at) @@ -121,73 +110,36 @@ fn reap_reason( mod tests { use super::*; - fn config(idle_ttl: Option, max_lifetime: Option) -> SandboxReaperConfig { + fn config(max_lifetime: Option) -> SandboxReaperConfig { SandboxReaperConfig { interval: Duration::from_secs(60), - idle_ttl, max_lifetime, } } - fn observed(status: SandboxStatus) -> ObservedSandbox { + fn observed(status: centaur_sandbox_core::SandboxStatus) -> ObservedSandbox { ObservedSandbox::new("sandbox-1", "fake", status) } #[test] - fn reaps_suspended_sandbox_past_idle_ttl() { + fn reaps_running_sandbox_past_max_lifetime() { let now = SystemTime::now(); - let sandbox = observed(SandboxStatus::Suspended) - .with_suspended_since(Some(now - Duration::from_secs(7200))); + let sandbox = observed(centaur_sandbox_core::SandboxStatus::Running) + .with_created_at(Some(now - Duration::from_secs(100_000))); - let reason = reap_reason( - &sandbox, - now, - &config(Some(Duration::from_secs(3600)), None), - ); + let reason = reap_reason(&sandbox, now, &config(Some(Duration::from_secs(86_400)))); - assert_eq!(reason, Some("idle_ttl")); + assert_eq!(reason, Some("max_lifetime")); } #[test] - fn keeps_suspended_sandbox_within_idle_ttl() { + fn reaps_suspended_sandbox_past_max_lifetime() { let now = SystemTime::now(); - let sandbox = observed(SandboxStatus::Suspended) + let sandbox = observed(centaur_sandbox_core::SandboxStatus::Suspended) + .with_created_at(Some(now - Duration::from_secs(100_000))) .with_suspended_since(Some(now - Duration::from_secs(60))); - let reason = reap_reason( - &sandbox, - now, - &config(Some(Duration::from_secs(3600)), None), - ); - - assert_eq!(reason, None); - } - - #[test] - fn keeps_suspended_sandbox_without_pause_timestamp() { - let now = SystemTime::now(); - let sandbox = observed(SandboxStatus::Suspended); - - let reason = reap_reason( - &sandbox, - now, - &config(Some(Duration::from_secs(3600)), None), - ); - - assert_eq!(reason, None); - } - - #[test] - fn reaps_running_sandbox_past_max_lifetime() { - let now = SystemTime::now(); - let sandbox = observed(SandboxStatus::Running) - .with_created_at(Some(now - Duration::from_secs(100_000))); - - let reason = reap_reason( - &sandbox, - now, - &config(None, Some(Duration::from_secs(86_400))), - ); + let reason = reap_reason(&sandbox, now, &config(Some(Duration::from_secs(86_400)))); assert_eq!(reason, Some("max_lifetime")); } @@ -195,14 +147,10 @@ mod tests { #[test] fn keeps_running_sandbox_within_max_lifetime() { let now = SystemTime::now(); - let sandbox = - observed(SandboxStatus::Running).with_created_at(Some(now - Duration::from_secs(60))); + let sandbox = observed(centaur_sandbox_core::SandboxStatus::Running) + .with_created_at(Some(now - Duration::from_secs(60))); - let reason = reap_reason( - &sandbox, - now, - &config(None, Some(Duration::from_secs(86_400))), - ); + let reason = reap_reason(&sandbox, now, &config(Some(Duration::from_secs(86_400)))); assert_eq!(reason, None); } @@ -210,17 +158,10 @@ mod tests { #[test] fn ignores_terminal_sandboxes() { let now = SystemTime::now(); - let sandbox = - observed(SandboxStatus::Gone).with_created_at(Some(now - Duration::from_secs(100_000))); - - let reason = reap_reason( - &sandbox, - now, - &config( - Some(Duration::from_secs(3600)), - Some(Duration::from_secs(86_400)), - ), - ); + let sandbox = observed(centaur_sandbox_core::SandboxStatus::Gone) + .with_created_at(Some(now - Duration::from_secs(100_000))); + + let reason = reap_reason(&sandbox, now, &config(Some(Duration::from_secs(86_400)))); assert_eq!(reason, None); } @@ -228,10 +169,10 @@ mod tests { #[test] fn disabled_config_reaps_nothing() { let now = SystemTime::now(); - let sandbox = observed(SandboxStatus::Suspended) + let sandbox = observed(centaur_sandbox_core::SandboxStatus::Suspended) .with_created_at(Some(now - Duration::from_secs(100_000))) .with_suspended_since(Some(now - Duration::from_secs(100_000))); - let config = config(None, None); + let config = config(None); assert!(!config.is_enabled()); assert_eq!(reap_reason(&sandbox, now, &config), None); diff --git a/services/api-rs/crates/centaur-sandbox-manager/src/warm_pool.rs b/services/api-rs/crates/centaur-sandbox-manager/src/warm_pool.rs index 8d1cf611a..dd8070e2a 100644 --- a/services/api-rs/crates/centaur-sandbox-manager/src/warm_pool.rs +++ b/services/api-rs/crates/centaur-sandbox-manager/src/warm_pool.rs @@ -1,19 +1,30 @@ -use std::{sync::Arc, time::Duration}; +use std::{ + sync::{ + Arc, + atomic::{AtomicBool, Ordering}, + }, + time::Duration, +}; use centaur_sandbox_core::{SandboxError, SandboxId, SandboxSpec, SandboxStatus}; use centaur_session_sqlx::{PgSessionStore, SessionStoreError}; use thiserror::Error; -use tokio::time::{MissedTickBehavior, interval}; -use tracing::warn; +use tokio::{ + sync::Mutex, + time::{MissedTickBehavior, interval}, +}; +use tracing::{debug, warn}; use crate::SandboxManager; pub type WarmSandboxSpecFactory = Arc SandboxSpec + Send + Sync>; +const STALE_EVICTING_WARM_SANDBOX_AGE: Duration = Duration::from_secs(300); pub struct WarmPoolConfig { pub target_size: usize, pub replenish_interval: Duration, pub bootstrap_iron_control_principal: Option, + pub max_running_sandboxes: Option, } pub struct WarmPoolManager { @@ -22,6 +33,8 @@ pub struct WarmPoolManager { spec_factory: WarmSandboxSpecFactory, workload_key: String, config: WarmPoolConfig, + paused: AtomicBool, + reconcile_lock: Mutex<()>, } impl WarmPoolManager { @@ -38,6 +51,8 @@ impl WarmPoolManager { spec_factory, workload_key: workload_key.into(), config, + paused: AtomicBool::new(false), + reconcile_lock: Mutex::new(()), } } @@ -45,6 +60,15 @@ impl WarmPoolManager { &self.workload_key } + /// Permanently pause this process's replenisher and wait for any in-flight + /// reconciliation to finish. Deployment drains use this before enumerating + /// sandboxes so the background loop cannot recreate a warm sandbox between + /// the drain and process shutdown. + pub async fn pause_and_wait(&self) { + self.paused.store(true, Ordering::SeqCst); + let _guard = self.reconcile_lock.lock().await; + } + pub fn spawn_replenisher(self: Arc) { tokio::spawn(async move { let mut tick = interval(self.config.replenish_interval); @@ -115,12 +139,21 @@ impl WarmPoolManager { } async fn replenish_once(&self) -> Result<(), WarmPoolError> { + let _guard = self.reconcile_lock.lock().await; + if self.paused.load(Ordering::SeqCst) { + return Ok(()); + } + self.prune_outdated_workload_ready_sandboxes().await?; + self.prune_stale_ready_sandboxes().await?; + self.prune_stale_evicting_sandboxes().await?; + let needed = self.config.target_size.saturating_sub( self.store .count_ready_warm_sandboxes(self.workload_key.as_str()) .await? .max(0) as usize, ); + let needed = needed.min(self.available_running_slots().await?); for _ in 0..needed { let mut spec = (self.spec_factory)(); @@ -140,6 +173,128 @@ impl WarmPoolManager { Ok(()) } + + async fn prune_outdated_workload_ready_sandboxes(&self) -> Result<(), WarmPoolError> { + for sandbox_id in self + .store + .reserve_ready_warm_sandboxes_for_workload_mismatch(self.workload_key.as_str()) + .await? + { + let id = SandboxId::new(sandbox_id.as_str()); + let result = match self.manager.status(&id).await { + Ok(status) if status_consumes_running_slot(&status) => { + match self.manager.stop(&id).await { + Ok(()) | Err(SandboxError::NotFound(_)) => { + "outdated ready warm sandbox stopped".to_owned() + } + Err(error) => { + let error_message = error.to_string(); + warn!(%sandbox_id, error = %error_message); + return Err(WarmPoolError::Sandbox(error)); + } + } + } + Ok(status) => format!("outdated ready warm sandbox was not running: {status:?}"), + Err(SandboxError::NotFound(_)) => { + "outdated ready warm sandbox was not found".to_owned() + } + Err(error) => { + let error_message = error.to_string(); + warn!(%sandbox_id, error = %error_message); + return Err(WarmPoolError::Sandbox(error)); + } + }; + warn!(%sandbox_id, reason = %result, "retiring outdated ready warm sandbox"); + self.store + .mark_warm_sandbox_failed(&sandbox_id, &result) + .await?; + } + Ok(()) + } + + async fn prune_stale_ready_sandboxes(&self) -> Result<(), WarmPoolError> { + for sandbox_id in self.store.list_ready_warm_sandbox_ids().await? { + let id = SandboxId::new(sandbox_id.as_str()); + let failure = match self.manager.status(&id).await { + Ok(SandboxStatus::Running) => continue, + Ok(status) => format!("ready warm sandbox was not running: {status:?}"), + Err(SandboxError::NotFound(_)) => "ready warm sandbox was not found".to_owned(), + Err(error) => { + let error_message = error.to_string(); + warn!(%sandbox_id, error = %error_message); + return Err(WarmPoolError::Sandbox(error)); + } + }; + warn!(%sandbox_id, error = %failure, "marking stale ready warm sandbox failed"); + if !self + .store + .mark_ready_warm_sandbox_failed_if_unclaimed(&sandbox_id, &failure) + .await? + { + debug!(%sandbox_id, "stale ready warm sandbox was claimed while its status was checked"); + } + } + Ok(()) + } + + async fn prune_stale_evicting_sandboxes(&self) -> Result<(), WarmPoolError> { + for sandbox_id in self + .store + .list_stale_evicting_warm_sandbox_ids(STALE_EVICTING_WARM_SANDBOX_AGE) + .await? + { + let id = SandboxId::new(sandbox_id.as_str()); + let failure = match self.manager.status(&id).await { + Ok(status) if status_consumes_running_slot(&status) => { + match self.manager.stop(&id).await { + Ok(()) | Err(SandboxError::NotFound(_)) => { + "stale evicting warm sandbox stopped".to_owned() + } + Err(error) => { + let error_message = error.to_string(); + warn!(%sandbox_id, error = %error_message); + return Err(WarmPoolError::Sandbox(error)); + } + } + } + Ok(status) => format!("stale evicting warm sandbox was not running: {status:?}"), + Err(SandboxError::NotFound(_)) => { + "stale evicting warm sandbox was not found".to_owned() + } + Err(error) => { + let error_message = error.to_string(); + warn!(%sandbox_id, error = %error_message); + return Err(WarmPoolError::Sandbox(error)); + } + }; + warn!(%sandbox_id, reason = %failure, "marking stale evicting warm sandbox failed"); + self.store + .mark_warm_sandbox_failed(&sandbox_id, &failure) + .await?; + } + Ok(()) + } + + async fn available_running_slots(&self) -> Result { + let Some(max_running) = self.config.max_running_sandboxes else { + return Ok(usize::MAX); + }; + let running = self + .manager + .list_observed() + .await? + .into_iter() + .filter(|observed| status_consumes_running_slot(&observed.status)) + .count(); + Ok(max_running.saturating_sub(running)) + } +} + +fn status_consumes_running_slot(status: &SandboxStatus) -> bool { + matches!( + status, + SandboxStatus::Created | SandboxStatus::Running | SandboxStatus::Unknown(_) + ) } #[derive(Debug, Error)] @@ -149,3 +304,451 @@ pub enum WarmPoolError { #[error(transparent)] Sandbox(#[from] SandboxError), } + +#[cfg(test)] +mod tests { + use std::{ + collections::BTreeMap, + sync::{Arc, Mutex}, + time::{Duration, SystemTime, UNIX_EPOCH}, + }; + + use async_trait::async_trait; + use centaur_sandbox_core::{ + ObservedSandbox, SandboxBackend, SandboxError, SandboxHandle, SandboxId, SandboxIo, + SandboxResult, SandboxSpec, SandboxStatus, + }; + use centaur_session_core::{HarnessType, ThreadKey}; + use serde_json::json; + use tokio::sync::OnceCell; + + use super::*; + + // Replenishment scans every warm-pool row, so DB-backed tests must not run + // fake backends against each other's rows. + static TEST_LOCK: tokio::sync::Mutex<()> = tokio::sync::Mutex::const_new(()); + + #[tokio::test] + async fn paused_pool_does_not_replenish_during_deployment_drain() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let backend = Arc::new(TestBackend::new(format!("paused-{}", unique_suffix()))); + let pool = WarmPoolManager::new( + Arc::new(SandboxManager::new(backend.clone())), + store, + Arc::new(|| SandboxSpec::new("image")), + format!("paused-workload-{}", unique_suffix()), + WarmPoolConfig { + target_size: 1, + replenish_interval: Duration::from_secs(1), + bootstrap_iron_control_principal: None, + max_running_sandboxes: None, + }, + ); + + pool.pause_and_wait().await; + pool.replenish_once().await.expect("paused replenish"); + + assert!(backend.created().is_empty()); + } + + #[tokio::test] + async fn replenisher_prunes_missing_ready_rows_before_counting() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let suffix = unique_suffix(); + let workload_key = format!("test-prune-{suffix}"); + let old_workload_key = format!("test-prune-old-{suffix}"); + let stale_sandbox = format!("stale-{suffix}"); + let old_stale_sandbox = format!("old-stale-{suffix}"); + let fresh_sandbox = format!("fresh-{suffix}"); + let claimed_thread = ThreadKey::parse(format!("test:warm-prune-{suffix}")) + .expect("parse claimed thread key"); + + store + .insert_ready_warm_sandbox(&stale_sandbox, &workload_key) + .await + .expect("insert stale warm sandbox row"); + store + .insert_ready_warm_sandbox(&old_stale_sandbox, &old_workload_key) + .await + .expect("insert stale warm sandbox row for old workload"); + assert_eq!( + store + .count_ready_warm_sandboxes(&workload_key) + .await + .expect("count ready warm sandboxes"), + 1 + ); + assert_eq!( + store + .count_ready_warm_sandboxes(&old_workload_key) + .await + .expect("count ready warm sandboxes for old workload"), + 1 + ); + + let backend = Arc::new(TestBackend::new(fresh_sandbox.clone())); + let pool = WarmPoolManager::new( + Arc::new(SandboxManager::new(backend.clone())), + store.clone(), + Arc::new(|| SandboxSpec::new("image")), + workload_key.clone(), + WarmPoolConfig { + target_size: 1, + replenish_interval: Duration::from_secs(60), + bootstrap_iron_control_principal: None, + max_running_sandboxes: None, + }, + ); + + pool.replenish_once().await.expect("replenish warm pool"); + + assert_eq!(backend.created(), vec![fresh_sandbox.clone()]); + assert_eq!( + store + .count_ready_warm_sandboxes(&workload_key) + .await + .expect("count ready warm sandboxes"), + 1 + ); + store + .create_or_get_session(&claimed_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create warm-pool claim session"); + assert_eq!( + store + .claim_ready_warm_sandbox(&workload_key, claimed_thread.as_str()) + .await + .expect("claim ready warm sandbox"), + Some(fresh_sandbox) + ); + assert_eq!( + store + .count_ready_warm_sandboxes(&old_workload_key) + .await + .expect("count ready warm sandboxes for old workload"), + 0 + ); + } + + #[tokio::test] + async fn replenisher_prunes_stale_evicting_rows() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let suffix = unique_suffix(); + let workload_key = format!("test-evicting-{suffix}"); + let stale_sandbox = format!("stale-evicting-{suffix}"); + + store + .insert_ready_warm_sandbox(&stale_sandbox, &workload_key) + .await + .expect("insert stale evicting warm sandbox row"); + sqlx::query( + r#" + update session_warm_sandboxes + set status = 'evicting', updated_at = now() - interval '10 minutes' + where sandbox_id = $1 + "#, + ) + .bind(&stale_sandbox) + .execute(store.pool()) + .await + .expect("make warm sandbox eviction stale"); + + let backend = Arc::new(TestBackend::new(format!("fresh-{suffix}"))); + backend.set_status(&stale_sandbox, SandboxStatus::Running); + let pool = WarmPoolManager::new( + Arc::new(SandboxManager::new(backend.clone())), + store.clone(), + Arc::new(|| SandboxSpec::new("image")), + workload_key.clone(), + WarmPoolConfig { + target_size: 0, + replenish_interval: Duration::from_secs(60), + bootstrap_iron_control_principal: None, + max_running_sandboxes: None, + }, + ); + + pool.replenish_once().await.expect("replenish warm pool"); + + assert_eq!( + backend + .status(&SandboxId::new(&stale_sandbox)) + .await + .unwrap(), + SandboxStatus::Stopped + ); + assert!( + !store + .list_referenced_sandbox_ids() + .await + .expect("list referenced sandboxes") + .contains(&stale_sandbox) + ); + } + + #[tokio::test] + async fn replenisher_stops_only_unclaimed_ready_rows_for_old_workloads() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let suffix = unique_suffix(); + let workload_key = format!("test-current-{suffix}"); + let old_workload_key = format!("test-old-{suffix}"); + let current_ready = format!("current-ready-{suffix}"); + let old_ready = format!("old-ready-{suffix}"); + let old_claimed = format!("old-claimed-{suffix}"); + let old_bound = format!("old-bound-{suffix}"); + let claimed_thread = ThreadKey::parse(format!("test:warm-claimed-{suffix}")) + .expect("parse claimed thread key"); + + for (sandbox_id, key) in [ + (¤t_ready, &workload_key), + (&old_claimed, &old_workload_key), + ] { + store + .insert_ready_warm_sandbox(sandbox_id, key) + .await + .expect("insert warm sandbox row"); + } + store + .create_or_get_session(&claimed_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create claimed session"); + assert_eq!( + store + .claim_ready_warm_sandbox(&old_workload_key, claimed_thread.as_str()) + .await + .expect("claim old workload sandbox"), + Some(old_claimed.clone()) + ); + store + .insert_ready_warm_sandbox(&old_ready, &old_workload_key) + .await + .expect("insert old ready warm sandbox row"); + store + .insert_ready_warm_sandbox(&old_bound, &old_workload_key) + .await + .expect("insert old bound warm sandbox row"); + let bound_thread = + ThreadKey::parse(format!("test:warm-bound-{suffix}")).expect("parse bound thread key"); + store + .create_or_get_session(&bound_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create bound session"); + store + .update_sandbox_id(&bound_thread, Some(&old_bound)) + .await + .expect("bind old sandbox to session"); + let bound_execution = store + .create_execution(&bound_thread, None, json!({})) + .await + .expect("create bound execution") + .execution + .execution_id; + store + .mark_execution_running(&bound_execution) + .await + .expect("mark bound execution running"); + + let backend = Arc::new(TestBackend::new(format!("unused-{suffix}"))); + for sandbox_id in [¤t_ready, &old_ready, &old_claimed, &old_bound] { + backend.set_status(sandbox_id, SandboxStatus::Running); + } + let pool = WarmPoolManager::new( + Arc::new(SandboxManager::new(backend.clone())), + store.clone(), + Arc::new(|| SandboxSpec::new("image")), + workload_key.clone(), + WarmPoolConfig { + target_size: 0, + replenish_interval: Duration::from_secs(60), + bootstrap_iron_control_principal: None, + max_running_sandboxes: None, + }, + ); + + pool.replenish_once().await.expect("reconcile warm pool"); + + assert_eq!( + backend.status(&SandboxId::new(&old_ready)).await.unwrap(), + SandboxStatus::Stopped + ); + for sandbox_id in [¤t_ready, &old_claimed, &old_bound] { + assert_eq!( + backend.status(&SandboxId::new(sandbox_id)).await.unwrap(), + SandboxStatus::Running, + "current or claimed sandbox must not be stopped" + ); + } + assert_eq!( + store + .count_ready_warm_sandboxes(&workload_key) + .await + .expect("count current ready rows"), + 1 + ); + let claimed_status = sqlx::query_scalar::<_, String>( + "select status from session_warm_sandboxes where sandbox_id = $1", + ) + .bind(&old_claimed) + .fetch_one(store.pool()) + .await + .expect("read claimed warm row"); + assert_eq!(claimed_status, "claimed"); + let bound_status = sqlx::query_scalar::<_, String>( + "select status from session_warm_sandboxes where sandbox_id = $1", + ) + .bind(&old_bound) + .fetch_one(store.pool()) + .await + .expect("read bound warm row"); + assert_eq!(bound_status, "ready"); + store + .fail_execution_if_active(&bound_execution, "test cleanup") + .await + .expect("terminalize bound execution"); + } + + async fn test_store() -> Option { + let Ok(url) = std::env::var("SESSION_RUNTIME_TEST_DATABASE_URL") else { + eprintln!("skipping: SESSION_RUNTIME_TEST_DATABASE_URL not set"); + return None; + }; + static MIGRATIONS: OnceCell<()> = OnceCell::const_new(); + MIGRATIONS + .get_or_init(|| async { + let store = PgSessionStore::connect(&url) + .await + .expect("connect test db"); + store.run_migrations().await.expect("run migrations"); + }) + .await; + Some( + PgSessionStore::connect(&url) + .await + .expect("connect test db after migrations"), + ) + } + + fn unique_suffix() -> String { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("system time should be after unix epoch") + .as_nanos() + .to_string() + } + + struct TestBackend { + create_id: String, + statuses: Mutex>, + created: Mutex>, + } + + impl TestBackend { + fn new(create_id: String) -> Self { + Self { + create_id, + statuses: Mutex::new(BTreeMap::new()), + created: Mutex::new(Vec::new()), + } + } + + fn created(&self) -> Vec { + self.created.lock().unwrap().clone() + } + + fn set_status(&self, sandbox_id: &str, status: SandboxStatus) { + self.statuses + .lock() + .unwrap() + .insert(sandbox_id.to_owned(), status); + } + } + + #[async_trait] + impl SandboxBackend for TestBackend { + fn name(&self) -> &'static str { + "test" + } + + async fn create(&self, _spec: SandboxSpec) -> SandboxResult { + self.statuses + .lock() + .unwrap() + .insert(self.create_id.clone(), SandboxStatus::Running); + self.created.lock().unwrap().push(self.create_id.clone()); + Ok(SandboxHandle::new( + SandboxId::new(self.create_id.clone()), + self.name(), + )) + } + + async fn open_io(&self, _id: &SandboxId) -> SandboxResult { + Err(SandboxError::Unsupported { + backend: self.name(), + operation: "open_io", + }) + } + + async fn status(&self, id: &SandboxId) -> SandboxResult { + self.statuses + .lock() + .unwrap() + .get(id.as_str()) + .cloned() + .ok_or_else(|| SandboxError::NotFound(id.as_str().to_owned())) + } + + async fn observe(&self, id: &SandboxId) -> SandboxResult { + Ok(ObservedSandbox::new( + id.clone(), + self.name(), + self.status(id).await?, + )) + } + + async fn list_observed(&self) -> SandboxResult> { + Ok(self + .statuses + .lock() + .unwrap() + .iter() + .map(|(id, status)| ObservedSandbox::new(id.clone(), self.name(), status.clone())) + .collect()) + } + + async fn stop(&self, id: &SandboxId) -> SandboxResult<()> { + self.statuses + .lock() + .unwrap() + .insert(id.as_str().to_owned(), SandboxStatus::Stopped); + Ok(()) + } + + async fn pause(&self, id: &SandboxId) -> SandboxResult<()> { + self.statuses + .lock() + .unwrap() + .insert(id.as_str().to_owned(), SandboxStatus::Suspended); + Ok(()) + } + + async fn resume(&self, id: &SandboxId) -> SandboxResult<()> { + self.statuses + .lock() + .unwrap() + .insert(id.as_str().to_owned(), SandboxStatus::Running); + Ok(()) + } + } +} diff --git a/services/api-rs/crates/centaur-session-cli/src/main.rs b/services/api-rs/crates/centaur-session-cli/src/main.rs index 75f0a935a..79c7b3988 100644 --- a/services/api-rs/crates/centaur-session-cli/src/main.rs +++ b/services/api-rs/crates/centaur-session-cli/src/main.rs @@ -1,4 +1,4 @@ -use std::str::FromStr; +use std::{env, str::FromStr}; use centaur_api_server::{ client::{CentaurClient, SseEvent as ApiSseEvent, SseEventStream}, @@ -79,7 +79,12 @@ async fn main() -> Result<()> { if generated_thread_key { eprintln!("thread_key={}", thread_key.as_str()); } - let client = CentaurClient::new(args.api_url.as_str()); + let api_key = env::var("CENTAUR_API_KEY") + .or_else(|_| env::var("CENTAUR_CONTROL_API_KEY")) + .wrap_err( + "CENTAUR_API_KEY or CENTAUR_CONTROL_API_KEY is required for authenticated session API access", + )?; + let client = CentaurClient::new(args.api_url.as_str()).with_bearer_token(api_key); if attach_mode { let events = client diff --git a/services/api-rs/crates/centaur-session-core/src/lib.rs b/services/api-rs/crates/centaur-session-core/src/lib.rs index a24a67d47..74a11a056 100644 --- a/services/api-rs/crates/centaur-session-core/src/lib.rs +++ b/services/api-rs/crates/centaur-session-core/src/lib.rs @@ -137,22 +137,67 @@ pub enum SessionStatus { Archived, } +#[derive(Clone, Debug, Default, Eq, PartialEq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum SandboxRepoCacheAccess { + None, + Public, + #[default] + All, +} + +impl SandboxRepoCacheAccess { + pub const fn enabled(&self) -> bool { + !matches!(self, Self::None) + } + + pub const fn as_str(&self) -> &'static str { + match self { + Self::None => "none", + Self::Public => "public", + Self::All => "all", + } + } + + pub fn parse(value: &str) -> Option { + match value.trim().to_ascii_lowercase().as_str() { + "none" => Some(Self::None), + "public" => Some(Self::Public), + "all" => Some(Self::All), + _ => None, + } + } + + pub const fn from_legacy_enabled(enabled: bool) -> Self { + if enabled { Self::All } else { Self::None } + } +} + #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] pub struct SandboxCapabilities { - pub repo_cache_enabled: bool, + #[serde(default)] + pub repo_cache: SandboxRepoCacheAccess, pub observability_enabled: bool, + pub api_server_enabled: bool, } impl SandboxCapabilities { pub const fn default_enabled() -> Self { Self { - repo_cache_enabled: true, + repo_cache: SandboxRepoCacheAccess::All, observability_enabled: true, + api_server_enabled: true, } } pub const fn is_default_enabled(&self) -> bool { - self.repo_cache_enabled && self.observability_enabled + matches!(self.repo_cache, SandboxRepoCacheAccess::All) + && self.observability_enabled + && self.api_server_enabled + } + + pub const fn repo_cache_enabled(&self) -> bool { + self.repo_cache.enabled() } } @@ -165,7 +210,15 @@ impl Default for SandboxCapabilities { #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] pub struct Session { pub thread_key: ThreadKey, + #[serde(skip_serializing_if = "Option::is_none")] + pub title: Option, pub sandbox_id: Option, + /// Digest binding the deployment boot-content generation to the sandbox + /// ID that received it. `None` is a legacy/untracked assignment; an older + /// rollback writer may also leave a stale `Some` after clearing/changing + /// `sandbox_id`, which fails ID-bound validation on the next forward turn. + #[serde(default)] + pub sandbox_content_revision: Option, /// Capabilities applied to the currently assigned sandbox. `None` means the /// sandbox predates capability tracking; callers may treat it as compatible /// only with the default-enabled profile. @@ -178,6 +231,10 @@ pub struct Session { /// iron-control principal OID this session's egress proxy binds to, /// captured at registration so a resumed session can recreate its sandbox. pub iron_control_principal: Option, + /// Last meaningful activity for the currently assigned sandbox. This is + /// the eviction signal for capacity pressure and intentionally separate + /// from `updated_at`, which also changes for metadata/status writes. + pub sandbox_last_active_at: Option, pub created_at: OffsetDateTime, pub updated_at: OffsetDateTime, } diff --git a/services/api-rs/crates/centaur-session-runtime/Cargo.toml b/services/api-rs/crates/centaur-session-runtime/Cargo.toml index 1b0ce2449..b12efc59a 100644 --- a/services/api-rs/crates/centaur-session-runtime/Cargo.toml +++ b/services/api-rs/crates/centaur-session-runtime/Cargo.toml @@ -14,6 +14,7 @@ centaur-session-sqlx.workspace = true centaur-telemetry.workspace = true dashmap.workspace = true futures-util.workspace = true +reqwest.workspace = true serde.workspace = true serde_json.workspace = true sha2.workspace = true @@ -25,6 +26,7 @@ uuid.workspace = true [dev-dependencies] async-trait.workspace = true +sqlx.workspace = true time.workspace = true tokio = { workspace = true, features = ["io-util", "macros", "rt-multi-thread", "sync", "time"] } uuid.workspace = true diff --git a/services/api-rs/crates/centaur-session-runtime/src/cleanup.rs b/services/api-rs/crates/centaur-session-runtime/src/cleanup.rs index 0dd724b1f..49eaeb60a 100644 --- a/services/api-rs/crates/centaur-session-runtime/src/cleanup.rs +++ b/services/api-rs/crates/centaur-session-runtime/src/cleanup.rs @@ -130,7 +130,7 @@ impl SessionSandboxCleanupWorker { &candidate.thread_key, &candidate.execution_id, &candidate.sandbox_id, - idle_backstop, + candidate.idle_timeout, ) .await { diff --git a/services/api-rs/crates/centaur-session-runtime/src/lib.rs b/services/api-rs/crates/centaur-session-runtime/src/lib.rs index 131d231a6..88b656bdc 100644 --- a/services/api-rs/crates/centaur-session-runtime/src/lib.rs +++ b/services/api-rs/crates/centaur-session-runtime/src/lib.rs @@ -1,15 +1,20 @@ mod cleanup; +mod title_generator; use std::{ - collections::{BTreeMap, HashMap, HashSet, VecDeque}, - sync::Arc, + collections::{BTreeMap, BTreeSet, HashMap, HashSet, VecDeque}, + future::Future, + sync::{ + Arc, + atomic::{AtomicBool, Ordering}, + }, time::{Duration, SystemTime}, }; use centaur_iron_control::SessionRegistrar; use centaur_sandbox_core::{ - Mount, SandboxBackend, SandboxCapabilities as BackendSandboxCapabilities, SandboxError, - SandboxId, SandboxIoGuard, SandboxRead, SandboxSpec, SandboxStatus, SandboxWrite, + Mount, RepoCacheAccess, SandboxBackend, SandboxCapabilities as BackendSandboxCapabilities, + SandboxError, SandboxId, SandboxIoGuard, SandboxRead, SandboxSpec, SandboxStatus, SandboxWrite, }; use centaur_sandbox_manager::{ SandboxManager, SandboxReaper, SandboxReaperConfig, WarmPoolConfig, WarmPoolError, @@ -17,42 +22,60 @@ use centaur_sandbox_manager::{ }; use centaur_session_core::{ ExecutionStatus, HarnessType, MessageRole, SandboxCapabilities as SessionSandboxCapabilities, - Session, SessionEvent, SessionExecution, SessionMessageInput, ThreadKey, + SandboxRepoCacheAccess as SessionRepoCacheAccess, Session, SessionEvent, SessionExecution, + SessionMessageInput, ThreadKey, }; use centaur_session_sqlx::{ - PgSessionStore, SessionEventListener, SessionStoreError, default_metadata, + PgSessionStore, ReleaseSessionResult, SandboxCapacityCandidate, SessionEventListener, + SessionStoreError, WorkflowOwnedSandbox, default_metadata, }; use centaur_telemetry::{ export_thread_trace_root_span, record_sandbox_warm_pool_claim, record_session_execution_finished, record_session_execution_started, record_session_failure, record_session_first_token_latency, set_span_parent_trace, }; -use dashmap::DashMap; -use futures_util::{SinkExt, Stream, StreamExt, stream}; +use dashmap::{DashMap, DashSet}; +use futures_util::{FutureExt, SinkExt, Stream, StreamExt, future::BoxFuture, stream}; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; use sha2::{Digest, Sha256}; use thiserror::Error; use tokio::{ io, - sync::Mutex, - time::{Instant, Interval, MissedTickBehavior, interval_at, sleep}, + sync::{Mutex, OwnedRwLockReadGuard, RwLock}, + time::{Instant, Interval, MissedTickBehavior, interval_at, sleep, timeout}, }; use tokio_util::codec::{FramedRead, FramedWrite, LinesCodec, LinesCodecError}; -use tracing::{Instrument, Span, error, info, info_span, warn}; +use tracing::{Instrument, Span, debug, error, info, info_span, warn}; +use uuid::Uuid; pub use cleanup::SessionSandboxCleanupConfig; +pub use title_generator::SessionTitleGenerationError; +use title_generator::{ + OpenAiSessionTitleGenerator, sanitize_session_title, session_title_source_from_parts, +}; pub const SESSION_OUTPUT_LINE_EVENT: &str = "session.output.line"; pub const SESSION_FIRST_TOKEN_EVENT: &str = "session.first_token"; - const EVENT_STREAM_SAFETY_POLL_INTERVAL: Duration = Duration::from_secs(30); const STEERING_STARTUP_RETRY_INTERVAL: Duration = Duration::from_millis(250); const STEERING_STARTUP_RETRY_TIMEOUT: Duration = Duration::from_secs(15); const SESSION_PIPE_MAX_REATTACH_ATTEMPTS: u32 = 3; const SESSION_PIPE_REATTACH_DELAY: Duration = Duration::from_millis(500); +const STDOUT_OWNER_LEASE: Duration = Duration::from_secs(45); +const STDOUT_OWNER_RENEW_INTERVAL: Duration = Duration::from_secs(10); +const EXECUTION_HANDOFF_POLL_INTERVAL: Duration = Duration::from_millis(500); +const EXECUTION_HANDOFF_DB_TIMEOUT: Duration = Duration::from_secs(5); +/// A live execution can briefly have no sandbox while it moves from queued +/// through warm-sandbox assignment. A periodic adoption scan must not fail a +/// young row it observes in that window. +const PRE_SANDBOX_ORPHAN_GRACE: Duration = Duration::from_secs(120); const COMPONENT_SESSION_RUNTIME: &str = "session_runtime"; const SANDBOX_REPOS_MOUNT_PATH: &str = "/home/agent/github"; +const PUBLIC_REPO_CACHE_SUBPATH: &str = "public"; +const CENTAUR_SKILL_DIRS_ENV: &str = "CENTAUR_SKILL_DIRS"; +const CENTAUR_PUBLIC_SKILL_DIRS_ENV: &str = "CENTAUR_PUBLIC_SKILL_DIRS"; +const SANDBOX_REPO_CACHE_LABEL: &str = "centaur.sandbox_repo_cache"; const OBSERVABILITY_TOOL_BLOCKLIST: &str = "vlogs,vmetrics,grafana,centaur_investigator,centaur-investigator"; @@ -63,6 +86,12 @@ type SessionInputSink = FramedWrite; type ExecutionSpanRegistry = Arc>>; type SessionPipeMap = Arc>; type SessionPipeOpenLocks = Arc>>>; +type ToolHostCallLocks = Arc>>>; +type SessionOperationLocks = Arc>>>; +type SessionTitleThreadSet = Arc>; +type SessionTitleGenerator = Arc< + dyn Fn(String) -> BoxFuture<'static, Result> + Send + Sync, +>; #[derive(Clone)] pub struct SessionRuntime { @@ -70,10 +99,37 @@ pub struct SessionRuntime { sandbox_runtime: SandboxRuntime, sandbox_pipes: SessionPipeMap, sandbox_pipe_open_locks: SessionPipeOpenLocks, + tool_host_call_locks: ToolHostCallLocks, + session_operation_locks: SessionOperationLocks, execution_spans: ExecutionSpanRegistry, iron_control: Option, warm_pool: Option>, personas: Option>, + session_title_generator: Option, + session_title_in_flight: SessionTitleThreadSet, + session_title_rerun_requested: SessionTitleThreadSet, + capacity: Option>, + stdout_owner_id: String, + /// Set once a shutdown handoff begins; fences new stdout-owner claims + /// so an execution cannot start on a control plane that is about to + /// exit and release its leases. + shutting_down: Arc, + /// Read-held across the entire sandbox ensure/assignment path. Drain takes + /// the write side after fencing new work, so no warm claim, resume, or + /// cold allocation can escape its inventory. + sandbox_allocation_gate: Arc>, +} + +#[derive(Clone, Copy, Debug)] +pub struct SandboxCapacityConfig { + pub max_running: usize, + pub hot_idle_grace: Duration, +} + +impl SandboxCapacityConfig { + pub fn is_enabled(&self) -> bool { + self.max_running > 0 + } } #[derive(Clone, Debug, Default, Serialize, Deserialize)] @@ -81,6 +137,7 @@ pub struct PersonaRegistry { personas: BTreeMap, default_persona_id: Option, overlay_chain: Vec, + public_source_roots: BTreeSet, } #[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)] @@ -135,9 +192,18 @@ impl PersonaRegistry { personas, default_persona_id, overlay_chain, + public_source_roots: BTreeSet::new(), }) } + pub fn with_public_source_roots( + mut self, + public_source_roots: impl IntoIterator, + ) -> Self { + self.public_source_roots = public_source_roots.into_iter().collect(); + self + } + pub fn summaries(&self) -> Vec { self.personas .values() @@ -155,16 +221,45 @@ impl PersonaRegistry { self.default_persona_id.as_deref() } + fn default_persona_id_for_access(&self, access: &SessionRepoCacheAccess) -> Option<&str> { + let default_persona_id = self.default_persona_id()?; + let persona = self.get(default_persona_id)?; + if self.persona_allowed_for_access(persona, access) { + Some(default_persona_id) + } else { + None + } + } + fn get(&self, persona_id: &str) -> Option<&PersonaDefinition> { self.personas.get(persona_id) } - fn context_for(&self, persona_id: &str, defaulted: bool) -> Result { + fn persona_allowed_for_access( + &self, + persona: &PersonaDefinition, + access: &SessionRepoCacheAccess, + ) -> bool { + !matches!(access, SessionRepoCacheAccess::Public) + || self.public_source_roots.contains(&persona.source_root) + } + + fn context_for_access( + &self, + persona_id: &str, + defaulted: bool, + access: &SessionRepoCacheAccess, + ) -> Result { let Some(persona) = self.get(persona_id) else { return Err(format!( "persona {persona_id:?} is not available in this deployment" )); }; + if !self.persona_allowed_for_access(persona, access) { + return Err(format!( + "persona {persona_id:?} is not available for public sandbox repo-cache access" + )); + } Ok(PersonaContext { persona_id: persona.id.clone(), source_root: persona.source_root.clone(), @@ -183,6 +278,10 @@ pub struct SandboxRuntime { spec_factory: SandboxSpecFactory, warm_spec_factory: Option, workload_key: Option, + /// Deployment-wide immutable boot-content identity shared by warm and + /// cold specs. Persisting it with assignments lets the first owned turn + /// after a rollout replace an otherwise reusable stale sandbox. + content_revision: Option, /// The harness warm sandboxes boot with. A warm claim is only valid for a /// session on the same harness; other sessions get a cold sandbox. warm_harness: Option, @@ -223,13 +322,14 @@ pub struct CreateOrGetSessionOutcome { pub harness_switched: bool, } -/// Result of [`SessionRuntime::release_thread`]. +/// Result of an owner-fenced canonical session release. #[derive(Clone, Debug)] pub struct ReleaseThreadOutcome { pub session: Session, pub release_id: Option, pub cancel_inflight: bool, pub sandbox_released: bool, + pub sandbox_missing: bool, pub sandbox_release_error: Option, pub execution_id: Option, pub execution_cancelled: bool, @@ -265,11 +365,59 @@ pub struct ExecuteSessionInput { pub max_duration_ms: Option, } +#[derive(Clone, Debug)] +pub struct InterruptExecutionOutcome { + pub interrupted: bool, + pub execution_id: Option, +} + +#[derive(Debug)] +pub struct ToolHostCallInput { + pub principal_id: String, + pub token_id: Option, + pub tool_name: String, + pub method: String, + pub arguments: Value, + pub timeout: Duration, +} + +#[derive(Debug)] +pub struct ToolHostCallOutput { + pub sandbox_id: String, + pub stdout: String, + pub stderr: String, + pub exit_status: Option, + pub timed_out: bool, +} + #[derive(Clone)] struct SessionPipe { stdin: Arc>, } +#[derive(Serialize)] +struct ToolHostRequest { + id: String, + tool: String, + method: String, + arguments: Value, + principal_id: String, + #[serde(skip_serializing_if = "Option::is_none")] + token_id: Option, + timeout_seconds: u64, +} + +#[derive(Deserialize)] +struct ToolHostResponse { + status: Option, + #[serde(default)] + stdout: String, + #[serde(default)] + stderr: String, + #[serde(default)] + timed_out: bool, +} + /// Shared handles threaded through background session tasks (stdout pump, /// terminal-output recording, max-duration failure, idle pause). #[derive(Clone)] @@ -278,6 +426,329 @@ struct RuntimeContext { manager: Arc, sandbox_pipes: SessionPipeMap, execution_spans: ExecutionSpanRegistry, + stdout_owner_id: String, +} + +struct SandboxCapacityController { + store: PgSessionStore, + manager: Arc, + sandbox_pipes: SessionPipeMap, + lock: Mutex<()>, + config: SandboxCapacityConfig, +} + +impl SandboxCapacityController { + fn new( + store: PgSessionStore, + manager: Arc, + sandbox_pipes: SessionPipeMap, + config: SandboxCapacityConfig, + ) -> Self { + Self { + store, + manager, + sandbox_pipes, + lock: Mutex::new(()), + config, + } + } + + async fn run_with_capacity( + &self, + protected_thread_key: &ThreadKey, + trigger_execution_id: &str, + operation: &'static str, + action: F, + ) -> Result + where + F: FnOnce() -> Fut, + Fut: Future>, + { + let _guard = self.lock.lock().await; + self.ensure_running_slot(protected_thread_key, trigger_execution_id, operation) + .await?; + action().await + } + + async fn ensure_running_slot( + &self, + protected_thread_key: &ThreadKey, + trigger_execution_id: &str, + operation: &'static str, + ) -> Result<(), SessionRuntimeError> { + let running = self.running_slot_count().await?; + if running < self.config.max_running { + return Ok(()); + } + + let mut slots_needed = running.saturating_sub(self.config.max_running) + 1; + let mut stopped_warm = 0usize; + let mut paused_idle = 0usize; + let mut stale_candidates_reconciled = 0usize; + + for sandbox_id in self + .store + .reserve_ready_warm_sandboxes_for_eviction(candidate_fetch_limit(slots_needed)) + .await? + { + if slots_needed == 0 { + break; + } + let id = SandboxId::new(sandbox_id.as_str()); + match self.manager.status(&id).await { + Ok(status) if status_consumes_running_slot(&status) => {} + Ok(_) | Err(SandboxError::NotFound(_)) => { + let _ = self + .store + .mark_warm_sandbox_failed( + sandbox_id.as_str(), + "not running during sandbox capacity admission", + ) + .await; + continue; + } + Err(error) => { + let failure = + format!("status failed during sandbox capacity admission: {error}"); + let _ = self + .store + .mark_warm_sandbox_failed(sandbox_id.as_str(), &failure) + .await; + return Err(SessionRuntimeError::Sandbox(error)); + } + } + + match self.manager.stop(&id).await { + Ok(()) | Err(SandboxError::NotFound(_)) => { + stopped_warm += 1; + slots_needed -= 1; + let _ = self + .store + .mark_warm_sandbox_failed( + sandbox_id.as_str(), + "stopped for sandbox capacity pressure", + ) + .await; + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "sandbox_capacity_warm_stopped", + sandbox_id, + trigger_thread_key = %protected_thread_key, + trigger_execution_id, + operation, + max_running = self.config.max_running, + "stopped warm sandbox for capacity" + ); + } + Err(error) => { + let failure = format!("stop failed during sandbox capacity admission: {error}"); + let _ = self + .store + .mark_warm_sandbox_failed(sandbox_id.as_str(), &failure) + .await; + return Err(SessionRuntimeError::Sandbox(error)); + } + } + } + + if slots_needed > 0 { + loop { + let candidates = self + .store + .list_sandbox_capacity_candidates( + Some(protected_thread_key), + self.config.hot_idle_grace, + candidate_fetch_limit(slots_needed), + ) + .await?; + if candidates.is_empty() { + break; + } + + let mut made_progress = false; + for candidate in candidates { + if slots_needed == 0 { + break; + } + match self + .pause_capacity_candidate( + &candidate, + protected_thread_key, + trigger_execution_id, + operation, + ) + .await? + { + CapacityCandidateAction::Paused => { + paused_idle += 1; + slots_needed -= 1; + made_progress = true; + } + CapacityCandidateAction::ReconciledStale => { + stale_candidates_reconciled += 1; + made_progress = true; + } + CapacityCandidateAction::Skipped => {} + } + } + + if slots_needed == 0 || !made_progress { + break; + } + } + } + + if slots_needed == 0 { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "sandbox_capacity_admitted", + trigger_thread_key = %protected_thread_key, + trigger_execution_id, + operation, + running_before = running, + max_running = self.config.max_running, + stopped_warm, + paused_idle, + stale_candidates_reconciled, + "admitted sandbox operation under capacity pressure" + ); + return Ok(()); + } + + Err(SessionRuntimeError::CapacityExceeded { + max_running: self.config.max_running, + running, + operation, + }) + } + + async fn pause_capacity_candidate( + &self, + candidate: &SandboxCapacityCandidate, + protected_thread_key: &ThreadKey, + trigger_execution_id: &str, + operation: &'static str, + ) -> Result { + let id = SandboxId::new(candidate.sandbox_id.as_str()); + match self.manager.status(&id).await { + Ok(SandboxStatus::Running | SandboxStatus::Created | SandboxStatus::Unknown(_)) => {} + Ok(SandboxStatus::Suspended) => { + return Ok(CapacityCandidateAction::Skipped); + } + Ok(SandboxStatus::Stopped | SandboxStatus::Gone) => { + return self.reconcile_stale_capacity_candidate(candidate).await; + } + Err(SandboxError::NotFound(_)) => { + return self.reconcile_stale_capacity_candidate(candidate).await; + } + Err(error) => return Err(SessionRuntimeError::Sandbox(error)), + } + + self.sandbox_pipes.remove(candidate.sandbox_id.as_str()); + match self.manager.pause(&id).await { + Ok(()) => { + self.store + .append_event( + &candidate.thread_key, + candidate.latest_execution_id.as_deref(), + "session.sandbox_paused", + json!({ + "thread_key": candidate.thread_key.as_str(), + "sandbox_id": candidate.sandbox_id.as_str(), + "reason": "capacity_pressure", + "trigger_thread_key": protected_thread_key.as_str(), + "trigger_execution_id": trigger_execution_id, + "operation": operation, + "last_active_at": candidate.last_active_at, + "hot_idle_grace_ms": duration_millis_u64(self.config.hot_idle_grace), + "max_running": self.config.max_running, + }), + ) + .await?; + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "sandbox_capacity_idle_paused", + thread_key = %candidate.thread_key, + sandbox_id = %candidate.sandbox_id, + trigger_thread_key = %protected_thread_key, + trigger_execution_id, + operation, + last_active_at = %candidate.last_active_at, + max_running = self.config.max_running, + "paused idle sandbox for capacity" + ); + Ok(CapacityCandidateAction::Paused) + } + Err(error) => { + self.store + .append_event( + &candidate.thread_key, + candidate.latest_execution_id.as_deref(), + "session.sandbox_pause_failed", + json!({ + "thread_key": candidate.thread_key.as_str(), + "sandbox_id": candidate.sandbox_id.as_str(), + "reason": "capacity_pressure", + "trigger_thread_key": protected_thread_key.as_str(), + "trigger_execution_id": trigger_execution_id, + "operation": operation, + "error": error.to_string(), + }), + ) + .await?; + Err(SessionRuntimeError::Sandbox(error)) + } + } + } + + async fn reconcile_stale_capacity_candidate( + &self, + candidate: &SandboxCapacityCandidate, + ) -> Result { + let cleared = self + .store + .clear_sandbox_id_if_matches(&candidate.thread_key, candidate.sandbox_id.as_str()) + .await?; + if cleared { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "sandbox_capacity_stale_reconciled", + thread_key = %candidate.thread_key, + sandbox_id = %candidate.sandbox_id, + "cleared stale sandbox assignment during capacity admission" + ); + Ok(CapacityCandidateAction::ReconciledStale) + } else { + Ok(CapacityCandidateAction::Skipped) + } + } + + async fn running_slot_count(&self) -> Result { + Ok(self + .manager + .list_observed() + .await? + .into_iter() + .filter(|observed| status_consumes_running_slot(&observed.status)) + .count()) + } +} + +enum CapacityCandidateAction { + Paused, + ReconciledStale, + Skipped, +} + +fn candidate_fetch_limit(slots_needed: usize) -> i64 { + slots_needed.saturating_mul(4).clamp(16, 1000) as i64 +} + +fn status_consumes_running_slot(status: &SandboxStatus) -> bool { + matches!( + status, + SandboxStatus::Created | SandboxStatus::Running | SandboxStatus::Unknown(_) + ) } struct EventStreamState { @@ -314,6 +785,25 @@ struct EnsureSessionSandboxRequest<'a> { execution_id: &'a str, } +#[derive(Clone, Debug, Eq, PartialEq)] +enum SandboxBootMode { + Harness, + ToolHost { principal_id: String }, +} + +impl SandboxBootMode { + fn as_str(&self) -> &'static str { + match self { + Self::Harness => "harness", + Self::ToolHost { .. } => "tool_host", + } + } + + fn uses_warm_pool(&self) -> bool { + matches!(self, Self::Harness) + } +} + struct PersonaResolution { persona_id: Option, context: Option, @@ -327,11 +817,64 @@ impl SessionRuntime { sandbox_runtime, sandbox_pipes: Arc::new(DashMap::new()), sandbox_pipe_open_locks: Arc::new(DashMap::new()), + tool_host_call_locks: Arc::new(DashMap::new()), + session_operation_locks: Arc::new(DashMap::new()), execution_spans: Arc::new(Mutex::new(HashMap::new())), iron_control: None, warm_pool: None, personas: None, + session_title_generator: None, + session_title_in_flight: Arc::new(DashSet::new()), + session_title_rerun_requested: Arc::new(DashSet::new()), + capacity: None, + stdout_owner_id: format!("api-rs-{}", uuid::Uuid::new_v4().simple()), + shutting_down: Arc::new(AtomicBool::new(false)), + sandbox_allocation_gate: Arc::new(RwLock::new(())), + } + } + + pub fn with_session_title_generator(mut self, generator: F) -> Self + where + F: Fn(String) -> Fut + Send + Sync + 'static, + Fut: Future> + Send + 'static, + { + self.session_title_generator = Some(Arc::new(move |source| generator(source).boxed())); + self + } + + pub fn with_openai_session_title_generator_from_env(mut self) -> Self { + let Some(generator) = OpenAiSessionTitleGenerator::from_env() else { + return self; + }; + self.session_title_generator = Some(Arc::new(move |source| { + let generator = generator.clone(); + async move { generator.generate(source).await }.boxed() + })); + self + } + + /// Acquire the same irreversible shutdown fence used by session sandbox + /// allocation. Callers that allocate auxiliary sandboxes (for example the + /// Python workflow host) must hold this permit across the allocation so a + /// deployment drain cannot miss a sandbox created outside the normal + /// session ensure path. + pub async fn acquire_sandbox_allocation_permit( + &self, + ) -> Result, SessionRuntimeError> { + if self.shutting_down.load(Ordering::SeqCst) { + return Err(SessionRuntimeError::ShuttingDown); + } + let guard = self.sandbox_allocation_gate.clone().read_owned().await; + if self.shutting_down.load(Ordering::SeqCst) { + return Err(SessionRuntimeError::ShuttingDown); } + Ok(guard) + } + + /// Return the process-wide sandbox manager handle. Auxiliary runtimes must + /// use this handle so the session drain inventories their sandboxes too. + pub fn sandbox_runtime_handle(&self) -> SandboxRuntime { + self.sandbox_runtime.clone() } pub fn with_personas(mut self, personas: PersonaRegistry) -> Self { @@ -346,14 +889,29 @@ impl SessionRuntime { .unwrap_or_default() } + pub async fn session_title( + &self, + thread_key: &ThreadKey, + ) -> Result, SessionRuntimeError> { + Ok(self.store.get_session_title(thread_key).await?) + } + + pub async fn get_session( + &self, + thread_key: &ThreadKey, + ) -> Result { + Ok(self.store.get_session(thread_key).await?) + } + fn resolve_persona_for_create( &self, requested_persona_id: Option<&str>, + capabilities: &SessionSandboxCapabilities, ) -> Result { let requested = requested_persona_id.and_then(clean_persona_id); - let selected = requested.or_else(|| self.default_persona_id()); + let selected = requested.or_else(|| self.default_persona_id_for_access(capabilities)); let defaulted = requested.is_none() && selected.is_some(); - let context = self.resolve_persona_context(selected, defaulted)?; + let context = self.resolve_persona_context(selected, defaulted, capabilities)?; Ok(PersonaResolution { persona_id: selected.map(str::to_owned), context, @@ -365,14 +923,16 @@ impl SessionRuntime { &self, persona_id: Option<&str>, _harness_type: &HarnessType, + capabilities: &SessionSandboxCapabilities, ) -> Result, SessionRuntimeError> { - self.resolve_persona_context(persona_id.and_then(clean_persona_id), false) + self.resolve_persona_context(persona_id.and_then(clean_persona_id), false, capabilities) } fn resolve_persona_context( &self, persona_id: Option<&str>, defaulted: bool, + capabilities: &SessionSandboxCapabilities, ) -> Result, SessionRuntimeError> { let Some(persona_id) = persona_id else { return Ok(None); @@ -383,7 +943,7 @@ impl SessionRuntime { ))); }; registry - .context_for(persona_id, defaulted) + .context_for_access(persona_id, defaulted, &capabilities.repo_cache) .map(Some) .map_err(SessionRuntimeError::BadRequest) } @@ -394,52 +954,420 @@ impl SessionRuntime { .and_then(|personas| personas.default_persona_id()) } + fn default_persona_id_for_access( + &self, + capabilities: &SessionSandboxCapabilities, + ) -> Option<&str> { + self.personas + .as_ref() + .and_then(|personas| personas.default_persona_id_for_access(&capabilities.repo_cache)) + } + fn context(&self) -> RuntimeContext { RuntimeContext { store: self.store.clone(), manager: self.sandbox_runtime.manager.clone(), sandbox_pipes: self.sandbox_pipes.clone(), execution_spans: self.execution_spans.clone(), + stdout_owner_id: self.stdout_owner_id.clone(), } } - /// Attach an iron-control registrar so each new session upserts its - /// principal and assigns the configured roles. - pub fn with_iron_control(mut self, registrar: SessionRegistrar) -> Self { - self.iron_control = Some(registrar); - self - } - - pub fn with_warm_pool(mut self, config: WarmPoolConfig) -> Self { - if config.target_size == 0 { - return self; + pub async fn run_tool_host_call( + &self, + input: ToolHostCallInput, + ) -> Result { + let principal_id = input.principal_id.trim().to_owned(); + let tool_name = input.tool_name.trim().to_owned(); + let method = input.method.trim().to_owned(); + if principal_id.is_empty() { + return Err(SessionRuntimeError::BadRequest( + "tool host principal_id is required".to_owned(), + )); + } + if tool_name.is_empty() { + return Err(SessionRuntimeError::BadRequest( + "tool host tool_name is required".to_owned(), + )); + } + if method.is_empty() { + return Err(SessionRuntimeError::BadRequest( + "tool host method is required".to_owned(), + )); + } + if input.timeout.is_zero() { + return Err(SessionRuntimeError::BadRequest( + "tool host timeout must be non-zero".to_owned(), + )); } - let (Some(spec_factory), Some(workload_key)) = ( - self.sandbox_runtime.warm_spec_factory.clone(), - self.sandbox_runtime.workload_key.clone(), - ) else { - warn!( - target_size = config.target_size, - "session sandbox warm pool requested for runtime without a warm sandbox spec" - ); - return self; + let thread_key = tool_host_thread_key(&principal_id)?; + let input = ToolHostCallInput { + principal_id, + tool_name, + method, + ..input }; + let call_lock = self.tool_host_call_lock(&thread_key); + let result = { + let _call_guard = call_lock.lock().await; + self.locked_tool_host_call(&thread_key, input).await + }; + // Drop our clone so an idle entry is only referenced by the map, then + // evict it; remove_if holds the shard lock, so no concurrent caller + // can clone the entry between the count check and the removal. + drop(call_lock); + self.tool_host_call_locks + .remove_if(thread_key.as_str(), |_, lock| Arc::strong_count(lock) == 1); + result + } - let pool = Arc::new(WarmPoolManager::new( - self.sandbox_runtime.manager.clone(), - self.store.clone(), - spec_factory, - workload_key, - config, - )); + fn tool_host_call_lock(&self, thread_key: &ThreadKey) -> Arc> { + self.tool_host_call_locks + .entry(thread_key.as_str().to_owned()) + .or_insert_with(|| Arc::new(Mutex::new(()))) + .clone() + } + + fn session_operation_lock(&self, thread_key: &ThreadKey) -> Arc> { + self.session_operation_locks + .entry(thread_key.as_str().to_owned()) + .or_insert_with(|| Arc::new(Mutex::new(()))) + .clone() + } + + fn release_session_operation_lock(&self, thread_key: &ThreadKey, lock: Arc>) { + drop(lock); + self.session_operation_locks + .remove_if(thread_key.as_str(), |_, lock| Arc::strong_count(lock) == 1); + } + + async fn locked_tool_host_call( + &self, + thread_key: &ThreadKey, + input: ToolHostCallInput, + ) -> Result { + let ToolHostCallInput { + principal_id, + token_id, + tool_name, + method, + arguments, + timeout, + } = input; + self.create_or_get_tool_host_session(thread_key, &principal_id) + .await?; + + let request_id = format!("mcp-call-{}", Uuid::new_v4().simple()); + let request = ToolHostRequest { + id: request_id.clone(), + tool: tool_name.clone(), + method: method.clone(), + arguments, + principal_id, + token_id, + timeout_seconds: timeout.as_secs().max(1), + }; + let input_line = serde_json::to_string(&request).map_err(|error| { + SessionRuntimeError::Sandbox(SandboxError::io_source("encode tool host request", error)) + })?; + let response_timeout = timeout.saturating_add(Duration::from_secs(5)); + let execution = self + .execute_session( + thread_key, + ExecuteSessionInput { + idempotency_key: Some(request_id.clone()), + metadata: Some(json!({ + "mcp_tool_host_call": true, + "request_id": request_id, + "tool": tool_name, + "method": method, + "timeout_ms": duration_millis_u64(timeout), + })), + input_lines: vec![input_line], + idle_timeout_ms: None, + max_duration_ms: Some(duration_millis_u64(response_timeout)), + }, + ) + .await?; + self.wait_for_tool_host_call(thread_key, &execution.execution_id, response_timeout) + .await + } + + async fn create_or_get_tool_host_session( + &self, + thread_key: &ThreadKey, + principal_id: &str, + ) -> Result<(), SessionRuntimeError> { + let harness = self + .sandbox_runtime + .warm_harness + .clone() + .unwrap_or(HarnessType::Codex); + let metadata = tool_host_session_metadata(principal_id); + let session = self + .store + .create_or_get_session(thread_key, &harness, None, metadata) + .await?; + if self.iron_control.is_some() + && session.iron_control_principal.as_deref() != Some(principal_id) + { + self.store + .set_iron_control_principal(thread_key, Some(principal_id)) + .await?; + } + Ok(()) + } + + async fn wait_for_tool_host_call( + &self, + thread_key: &ThreadKey, + execution_id: &str, + response_timeout: Duration, + ) -> Result { + let events = self + .stream_events(thread_key, 0, Some(execution_id)) + .await?; + futures_util::pin_mut!(events); + match timeout(response_timeout, async { + while let Some(event) = events.next().await { + let event = event?; + match event.event_type.as_str() { + "session.execution_completed" => { + return self.tool_host_completed_output(thread_key, &event).await; + } + "session.execution_failed" => { + return self.tool_host_failed_output(thread_key, &event).await; + } + _ => {} + } + } + Err(SessionRuntimeError::Sandbox(SandboxError::io( + "session event stream ended before tool host call completed", + ))) + }) + .await + { + Ok(output) => output, + // Best-effort sandbox id: a store error must not replace the + // timeout result with an internal error. + Err(_) => Ok(ToolHostCallOutput { + sandbox_id: self + .current_sandbox_id(thread_key) + .await + .unwrap_or_default(), + stdout: String::new(), + stderr: format!( + "tool host call timed out after {} ms", + response_timeout.as_millis() + ), + exit_status: None, + timed_out: true, + }), + } + } + + async fn tool_host_completed_output( + &self, + thread_key: &ThreadKey, + event: &SessionEvent, + ) -> Result { + let sandbox_id = self.current_sandbox_id(thread_key).await?; + let Some(result_text) = event.payload.get("result_text").and_then(Value::as_str) else { + return Ok(ToolHostCallOutput { + sandbox_id, + stdout: String::new(), + stderr: String::new(), + exit_status: Some(0), + timed_out: false, + }); + }; + let response = serde_json::from_str::(result_text).map_err(|error| { + SessionRuntimeError::Sandbox(SandboxError::io_source( + "decode tool host response", + error, + )) + })?; + Ok(ToolHostCallOutput { + sandbox_id, + stdout: response.stdout, + stderr: response.stderr, + exit_status: response.status, + timed_out: response.timed_out, + }) + } + + async fn tool_host_failed_output( + &self, + thread_key: &ThreadKey, + event: &SessionEvent, + ) -> Result { + let error = event + .payload + .get("error") + .and_then(Value::as_str) + .unwrap_or("tool host execution failed") + .to_owned(); + let timed_out = event + .payload + .get("reason") + .and_then(Value::as_str) + .is_some_and(|reason| reason == "max_duration_exceeded"); + Ok(ToolHostCallOutput { + sandbox_id: self.current_sandbox_id(thread_key).await?, + stdout: String::new(), + stderr: error, + exit_status: None, + timed_out, + }) + } + + async fn current_sandbox_id( + &self, + thread_key: &ThreadKey, + ) -> Result { + Ok(self + .store + .get_session(thread_key) + .await? + .sandbox_id + .unwrap_or_default()) + } + + async fn claim_stdout_owner(&self, execution_id: &str) -> Result<(), SessionRuntimeError> { + if self.shutting_down.load(Ordering::SeqCst) { + return Err(SessionRuntimeError::ShuttingDown); + } + let claimed = self + .store + .claim_stdout_owner(execution_id, &self.stdout_owner_id, STDOUT_OWNER_LEASE) + .await?; + if !claimed { + return Err(SessionRuntimeError::BadRequest(format!( + "execution {execution_id} stdout is owned by another control plane process" + ))); + } + spawn_stdout_owner_renewer(self.context(), execution_id.to_owned()); + Ok(()) + } + + async fn claim_expired_stdout_owner( + &self, + execution_id: &str, + ) -> Result { + let claimed = self + .store + .claim_expired_stdout_owner(execution_id, &self.stdout_owner_id, STDOUT_OWNER_LEASE) + .await?; + if claimed { + spawn_stdout_owner_renewer(self.context(), execution_id.to_owned()); + } + Ok(claimed) + } + + /// Attach an iron-control registrar so each new session upserts its + /// principal and assigns it the configured roles. + pub fn with_iron_control(mut self, registrar: SessionRegistrar) -> Self { + self.iron_control = Some(registrar); + self + } + + /// Register the shared unauthenticated MCP tool-host principal when + /// iron-control is enabled, so proxy-backed tool calls can resolve an + /// effective config without minting per-user credentials in this layer. + pub async fn register_mcp_tool_host_principal( + &self, + principal_id: &str, + ) -> Result { + let principal_id = principal_id.trim(); + if principal_id.is_empty() { + return Err(SessionRuntimeError::BadRequest( + "mcp tool host principal_id is required".to_owned(), + )); + } + if principal_id.contains(':') { + return Err(SessionRuntimeError::BadRequest( + "mcp tool host principal_id must not contain ':'".to_owned(), + )); + } + let thread_key = tool_host_thread_key(principal_id)?; + if let Some(registrar) = &self.iron_control { + // Serialize with run_tool_host_call so concurrent registrations + // for the same principal cannot interleave with session setup. + let call_lock = self.tool_host_call_lock(&thread_key); + let _call_guard = call_lock.lock().await; + let metadata = tool_host_session_metadata(principal_id); + let principal = registrar + .register_session(thread_key.as_str(), Some(&metadata)) + .await?; + return Ok(principal.id); + } + Ok(principal_id.to_owned()) + } + + pub fn with_warm_pool(mut self, config: WarmPoolConfig) -> Self { + if config.target_size == 0 { + return self; + } + + let (Some(spec_factory), Some(workload_key)) = ( + self.sandbox_runtime.warm_spec_factory.clone(), + self.sandbox_runtime.workload_key.clone(), + ) else { + warn!( + target_size = config.target_size, + "session sandbox warm pool requested for runtime without a warm sandbox spec" + ); + return self; + }; + + let pool = Arc::new(WarmPoolManager::new( + self.sandbox_runtime.manager.clone(), + self.store.clone(), + spec_factory, + workload_key, + config, + )); pool.clone().spawn_replenisher(); self.warm_pool = Some(pool); self } - /// Spawn the background reaper that stops sandboxes whose idle pause or - /// total lifetime expired. No-op when both TTLs are disabled. + pub fn with_sandbox_capacity(mut self, config: SandboxCapacityConfig) -> Self { + if !config.is_enabled() { + return self; + } + self.capacity = Some(Arc::new(SandboxCapacityController::new( + self.store.clone(), + self.sandbox_runtime.manager.clone(), + self.sandbox_pipes.clone(), + config, + ))); + self + } + + async fn run_with_running_capacity( + &self, + thread_key: &ThreadKey, + execution_id: &str, + operation: &'static str, + action: F, + ) -> Result + where + F: FnOnce() -> Fut, + Fut: Future>, + { + if let Some(capacity) = self.capacity.as_ref() { + capacity + .run_with_capacity(thread_key, execution_id, operation, action) + .await + } else { + action().await + } + } + + /// Spawn the background reaper that stops sandboxes whose total lifetime + /// expired. No-op when max-lifetime reaping is disabled. pub fn with_sandbox_reaper(self, config: SandboxReaperConfig) -> Self { if !config.is_enabled() { return self; @@ -466,6 +1394,56 @@ impl SessionRuntime { persona_id: Option<&str>, metadata: Option, on_harness_conflict: HarnessConflictPolicy, + ) -> Result { + self.create_or_get_session_inner( + thread_key, + harness_type, + persona_id, + metadata, + on_harness_conflict, + None, + ) + .await + } + + /// Create a session bound to an already-authenticated Console principal. + /// This deliberately bypasses thread-derived principal registration so a + /// scoped child session inherits the caller's exact sandbox capabilities + /// instead of receiving the deployment's default roles. + pub async fn create_or_get_session_for_principal( + &self, + thread_key: &ThreadKey, + harness_type: &HarnessType, + persona_id: Option<&str>, + metadata: Option, + on_harness_conflict: HarnessConflictPolicy, + principal_id: &str, + ) -> Result { + let principal_id = principal_id.trim(); + if principal_id.is_empty() { + return Err(SessionRuntimeError::BadRequest( + "principal-bound session requires a principal id".to_owned(), + )); + } + self.create_or_get_session_inner( + thread_key, + harness_type, + persona_id, + metadata, + on_harness_conflict, + Some(principal_id), + ) + .await + } + + async fn create_or_get_session_inner( + &self, + thread_key: &ThreadKey, + harness_type: &HarnessType, + persona_id: Option<&str>, + metadata: Option, + on_harness_conflict: HarnessConflictPolicy, + principal_override: Option<&str>, ) -> Result { let span = info_span!( "centaur.api_rs.session.create_or_get", @@ -493,33 +1471,78 @@ impl SessionRuntime { "creating or loading session" ); let mut harness_switched = false; - let persona_resolution = self.resolve_persona_for_create(persona_id)?; let mut session_metadata = default_metadata(metadata); + let (registered_principal, desired_capabilities) = + match (principal_override, self.iron_control.as_ref()) { + (Some(principal_id), Some(registrar)) => { + let principal = registrar.get_principal(principal_id).await?; + let desired_capabilities = sandbox_capabilities_from_principal(&principal); + (Some(principal), desired_capabilities) + } + (Some(_), None) => { + return Err(SessionRuntimeError::BadRequest( + "principal-bound sessions require Console registration".to_owned(), + )); + } + (None, Some(registrar)) => { + let principal = registrar + .register_session(thread_key.as_str(), Some(&session_metadata)) + .await?; + let desired_capabilities = sandbox_capabilities_from_principal(&principal); + (Some(principal), desired_capabilities) + } + (None, None) => (None, SessionSandboxCapabilities::default_enabled()), + }; + let persona_resolution = + self.resolve_persona_for_create(persona_id, &desired_capabilities)?; if let Some(context) = persona_resolution.context.as_ref() { add_persona_metadata(&mut session_metadata, context); } - let session = match self - .store - .create_or_get_session( - thread_key, - harness_type, - persona_resolution.persona_id.as_deref(), - session_metadata.clone(), - ) - .await - { + let create_result = if let Some(principal_id) = principal_override { + self.store + .create_or_get_session_for_principal( + thread_key, + harness_type, + persona_resolution.persona_id.as_deref(), + session_metadata.clone(), + principal_id, + ) + .await + } else { + self.store + .create_or_get_session( + thread_key, + harness_type, + persona_resolution.persona_id.as_deref(), + session_metadata.clone(), + ) + .await + }; + let session = match create_result { Ok(session) => session, Err(SessionStoreError::PersonaConflict { existing, .. }) if persona_id.is_none() && persona_resolution.defaulted => { - self.store - .create_or_get_session( - thread_key, - harness_type, - existing.as_deref(), - default_metadata(None), - ) - .await? + if let Some(principal_id) = principal_override { + self.store + .create_or_get_session_for_principal( + thread_key, + harness_type, + existing.as_deref(), + default_metadata(None), + principal_id, + ) + .await? + } else { + self.store + .create_or_get_session( + thread_key, + harness_type, + existing.as_deref(), + default_metadata(None), + ) + .await? + } } Err(SessionStoreError::HarnessConflict { existing, .. }) if on_harness_conflict == HarnessConflictPolicy::Restart => @@ -532,9 +1555,11 @@ impl SessionRuntime { } Err(error) => return Err(error.into()), }; - if let Some(context) = - self.resolve_stored_persona(session.persona_id.as_deref(), harness_type)? - { + if let Some(context) = self.resolve_stored_persona( + session.persona_id.as_deref(), + harness_type, + &desired_capabilities, + )? { self.store .append_event( thread_key, @@ -548,20 +1573,20 @@ impl SessionRuntime { ) .await?; } - if let Some(registrar) = &self.iron_control { - // iron-control is the source of truth for the session's egress - // proxy: without a registered principal the proxy has no identity - // to bind to, so a registration failure must fail session creation - // rather than silently boot a sandbox with a non-functional proxy. - let principal = registrar - .register_session(thread_key.as_str(), Some(&session_metadata)) - .await?; + if let Some(principal) = registered_principal { // Persist the principal OID on the session row so a resumed session // can recreate its sandbox after a restart without re-deriving it. - let session = self - .store - .set_iron_control_principal(thread_key, Some(&principal.id)) - .await?; + let session = if principal_override.is_some() { + debug_assert_eq!( + session.iron_control_principal.as_deref(), + Some(principal.id.as_str()) + ); + session + } else { + self.store + .set_iron_control_principal(thread_key, Some(&principal.id)) + .await? + }; info!( component = COMPONENT_SESSION_RUNTIME, event = "session_create_or_get_completed", @@ -695,6 +1720,15 @@ impl SessionRuntime { "appending session messages" ); let message_ids = self.store.append_messages(thread_key, messages).await?; + if let Err(error) = self.store.touch_session_sandbox_activity(thread_key).await { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_sandbox_activity_touch_failed", + thread_key = %thread_key, + %error, + "failed to touch sandbox activity after message append" + ); + } info!( component = COMPONENT_SESSION_RUNTIME, event = "session_messages_append_completed", @@ -724,9 +1758,45 @@ impl SessionRuntime { }; self.forward_messages_to_active_execution(thread_key, messages, &message_ids) .await; + self.spawn_session_title_generation(thread_key); Ok(message_ids) } + fn spawn_session_title_generation(&self, thread_key: &ThreadKey) { + let Some(generator) = self.session_title_generator.clone() else { + return; + }; + if !self.session_title_in_flight.insert(thread_key.clone()) { + self.session_title_rerun_requested + .insert(thread_key.clone()); + return; + } + let store = self.store.clone(); + let in_flight = self.session_title_in_flight.clone(); + let rerun_requested = self.session_title_rerun_requested.clone(); + let thread_key = thread_key.clone(); + tokio::spawn(async move { + // Appends skipped while generation is in flight request one more pass, + // which lets low-signal wakeups defer to a later substantive message. + loop { + rerun_requested.remove(&thread_key); + maybe_generate_session_title(store.clone(), generator.clone(), thread_key.clone()) + .await; + if rerun_requested.remove(&thread_key).is_some() { + continue; + } + + in_flight.remove(&thread_key); + if rerun_requested.remove(&thread_key).is_some() + && in_flight.insert(thread_key.clone()) + { + continue; + } + break; + } + }); + } + /// Stop every non-terminal sandbox the backend currently owns. /// /// Intended for a clean control-plane shutdown (e.g. before a deploy): @@ -734,6 +1804,14 @@ impl SessionRuntime { /// rest, and the [`DrainReport`] records which were stopped and which /// failed so the caller can surface partial failure. pub async fn drain(&self) -> Result { + // This fence is irreversible for the process lifetime. New executions + // fail before claiming stdout; allocators that passed the first check + // must still cross the write barrier below and recheck the fence. + self.shutting_down.store(true, Ordering::SeqCst); + if let Some(warm_pool) = &self.warm_pool { + warm_pool.pause_and_wait().await; + } + let _allocation_guard = self.sandbox_allocation_gate.write().await; let observed = self.sandbox_runtime.manager.list_observed().await?; let mut report = DrainReport::default(); for sandbox in observed { @@ -766,105 +1844,272 @@ impl SessionRuntime { } } } + let failed_ids = report + .failed + .iter() + .map(|failure| failure.sandbox_id.clone()) + .collect::>(); + for sandbox in self.sandbox_runtime.manager.list_observed().await? { + if sandbox.status.is_terminal() || failed_ids.contains(sandbox.id.as_str()) { + continue; + } + report.failed.push(DrainFailure { + sandbox_id: sandbox.id.as_str().to_owned(), + error: format!( + "sandbox remained non-terminal after drain: {:?}", + sandbox.status + ), + }); + } Ok(report) } - pub async fn stop_workflow_owned_sandboxes( + pub async fn release_thread( + &self, + thread_key: &ThreadKey, + release_id: Option<&str>, + expected_sandbox_id: Option<&str>, + cancel_inflight: bool, + ) -> Result { + let operation_lock = self.session_operation_lock(thread_key); + let result = { + let _guard = operation_lock.lock().await; + self.release_thread_locked(thread_key, release_id, expected_sandbox_id, cancel_inflight) + .await + }; + self.release_session_operation_lock(thread_key, operation_lock); + result + } + + async fn release_thread_locked( + &self, + thread_key: &ThreadKey, + release_id: Option<&str>, + expected_sandbox_id: Option<&str>, + cancel_inflight: bool, + ) -> Result { + let snapshot = self.store.get_session(thread_key).await?; + if expected_sandbox_id.is_some_and(|value| value.trim().is_empty()) { + return Err(SessionRuntimeError::BadRequest( + "expected_sandbox_id must not be empty".to_owned(), + )); + } + let expected_sandbox_id = expected_sandbox_id.map(str::trim); + if let Some(expected_sandbox_id) = expected_sandbox_id + && snapshot.sandbox_id.as_deref() != Some(expected_sandbox_id) + { + return Err(SessionRuntimeError::BadRequest(format!( + "session sandbox does not match caller fence (expected {expected_sandbox_id:?}, current {:?})", + snapshot.sandbox_id + ))); + } + let sandbox_id = expected_sandbox_id + .map(ToOwned::to_owned) + .or_else(|| snapshot.sandbox_id.clone()); + let cancellation_reason = + release_error_message(release_id, thread_key, sandbox_id.as_deref()); + let (session, cancelled_execution) = match self + .store + .release_session_if_sandbox_matches( + thread_key, + sandbox_id.as_deref(), + cancel_inflight, + &cancellation_reason, + ) + .await? + { + ReleaseSessionResult::Released { + session, + cancelled_execution, + } => (*session, cancelled_execution), + ReleaseSessionResult::ActiveExecution(execution) => { + return Err(SessionRuntimeError::BadRequest(format!( + "thread has active execution {}; retry with cancel_inflight=true", + execution.execution_id + ))); + } + ReleaseSessionResult::SandboxMismatch { current_sandbox_id } => { + return Err(SessionRuntimeError::BadRequest(format!( + "session sandbox changed during release (expected {sandbox_id:?}, current {current_sandbox_id:?}); retry" + ))); + } + }; + + let execution_id = cancelled_execution + .as_ref() + .map(|execution| execution.execution_id.clone()); + if let Some(execution_id) = execution_id.as_deref() { + self.execution_spans.lock().await.remove(execution_id); + } + if let Some(sandbox_id) = sandbox_id.as_deref() { + self.sandbox_pipes.remove(sandbox_id); + } + + let mut sandbox_released = false; + let mut sandbox_missing = false; + let mut sandbox_release_error = None; + if let Some(sandbox_id) = sandbox_id.as_deref() { + match self + .sandbox_runtime + .manager + .stop(&SandboxId::new(sandbox_id)) + .await + { + Ok(()) => sandbox_released = true, + Err(SandboxError::NotFound(_)) => { + sandbox_released = true; + sandbox_missing = true; + } + Err(error) => { + warn!(%thread_key, %sandbox_id, %error, "failed to stop released sandbox"); + sandbox_release_error = Some(error.to_string()); + } + } + } + + if let Some(execution) = cancelled_execution.as_ref() + && let Err(error) = self + .store + .append_event( + thread_key, + Some(&execution.execution_id), + "session.execution_cancelled", + json!({ + "execution_id": execution.execution_id, + "thread_key": thread_key.as_str(), + "sandbox_id": sandbox_id.as_deref(), + "release_id": release_id, + "reason": "thread_released", + }), + ) + .await + { + warn!(%thread_key, %error, "failed to record release cancellation event"); + } + if let Err(error) = self + .store + .append_event( + thread_key, + execution_id.as_deref(), + "session.released", + json!({ + "thread_key": thread_key.as_str(), + "release_id": release_id, + "cancel_inflight": cancel_inflight, + "sandbox_id": sandbox_id.as_deref(), + "sandbox_released": sandbox_released, + "sandbox_release_error": sandbox_release_error.as_deref(), + "execution_id": execution_id.as_deref(), + "execution_cancelled": cancelled_execution.is_some(), + }), + ) + .await + { + warn!(%thread_key, %error, "failed to record session release event"); + } + + Ok(ReleaseThreadOutcome { + session, + release_id: release_id.map(ToOwned::to_owned), + cancel_inflight, + sandbox_released, + sandbox_missing, + sandbox_release_error, + execution_id, + execution_cancelled: cancelled_execution.is_some(), + }) + } + + pub async fn stop_workflow_owned_sandboxes( &self, workflow_run_id: &str, reason: &str, ) -> Result { - let sandboxes = self + let sessions = self .store .list_workflow_owned_sandboxes(workflow_run_id) .await?; + self.stop_workflow_owned_sessions(sessions, workflow_run_id, reason) + .await + } + + async fn stop_workflow_owned_sessions( + &self, + sessions: Vec, + workflow_run_id: &str, + reason: &str, + ) -> Result { let mut report = WorkflowSandboxCleanupReport::default(); - for sandbox in sandboxes { - let sandbox_id = sandbox.sandbox_id; - let thread_key = sandbox.thread_key; - self.sandbox_pipes.remove(&sandbox_id); - let id = SandboxId::new(sandbox_id.clone()); - let mut missing = false; - match self.sandbox_runtime.manager.stop(&id).await { - Ok(()) => report.stopped.push(sandbox_id.clone()), - Err(SandboxError::NotFound(_)) => { - missing = true; - report.missing.push(sandbox_id.clone()); - } + for session in sessions { + let sandbox_id = session.sandbox_id; + let thread_key = session.thread_key; + let release_id = format!("workflow:run:{workflow_run_id}:{reason}"); + let outcome = match self + .release_thread(&thread_key, Some(&release_id), sandbox_id.as_deref(), true) + .await + { + Ok(outcome) => outcome, Err(error) => { - let error = error.to_string(); + let id = sandbox_id + .clone() + .unwrap_or_else(|| format!("thread:{}", thread_key.as_str())); warn!( thread_key = %thread_key, - sandbox_id, + sandbox_id = sandbox_id.as_deref(), workflow_run_id, reason, %error, - "failed to stop workflow-owned sandbox" + "failed to release workflow-owned session" ); report.failed.push(DrainFailure { - sandbox_id: sandbox_id.clone(), - error: error.clone(), + sandbox_id: id, + error: error.to_string(), }); - if let Err(event_error) = self - .store - .append_event( - &thread_key, - None, - "session.workflow_sandbox_stop_failed", - json!({ - "thread_key": thread_key.as_str(), - "sandbox_id": sandbox_id, - "workflow_run_id": workflow_run_id, - "reason": reason, - "error": error, - }), - ) - .await - { - warn!( - thread_key = %thread_key, - sandbox_id, - workflow_run_id, - %event_error, - "failed to append workflow sandbox stop failure event" - ); - } continue; } + }; + if let Some(sandbox_id) = sandbox_id.as_deref() { + if outcome.sandbox_missing { + report.missing.push(sandbox_id.to_owned()); + } else if outcome.sandbox_released { + report.stopped.push(sandbox_id.to_owned()); + } + if let Some(error) = outcome.sandbox_release_error.as_deref() { + report.failed.push(DrainFailure { + sandbox_id: sandbox_id.to_owned(), + error: error.to_owned(), + }); + } + if let Err(error) = self + .store + .mark_warm_sandbox_failed(sandbox_id, "workflow-owned sandbox stopped") + .await + { + warn!( + thread_key = %thread_key, + sandbox_id, + workflow_run_id, + %error, + "failed to mark workflow-owned warm sandbox failed" + ); + } } - - if let Err(error) = self - .store - .mark_warm_sandbox_failed(&sandbox_id, "workflow-owned sandbox stopped") - .await - { - warn!( - thread_key = %thread_key, - sandbox_id, - workflow_run_id, - %error, - "failed to mark workflow-owned warm sandbox failed" - ); - } - - let cleared = self - .store - .clear_sandbox_id_if_matches(&thread_key, &sandbox_id) - .await?; if let Err(error) = self .store .append_event( &thread_key, - None, + outcome.execution_id.as_deref(), "session.workflow_sandbox_stopped", json!({ "thread_key": thread_key.as_str(), "sandbox_id": sandbox_id, "workflow_run_id": workflow_run_id, "reason": reason, - "missing": missing, - "cleared": cleared, + "missing": outcome.sandbox_missing, + "cleared": outcome.session.sandbox_id.is_none(), + "execution_id": outcome.execution_id, + "execution_cancelled": outcome.execution_cancelled, }), ) .await @@ -882,116 +2127,21 @@ impl SessionRuntime { Ok(report) } - pub async fn release_thread( + pub async fn execute_session( &self, thread_key: &ThreadKey, - release_id: Option<&str>, - cancel_inflight: bool, - ) -> Result { - let session = self.store.get_session(thread_key).await?; - let sandbox_id = session.sandbox_id.clone(); - let mut execution_id = None; - let mut execution_cancelled = false; - - if let Some(active_execution) = self.store.active_execution_for_thread(thread_key).await? { - execution_id = Some(active_execution.execution_id.clone()); - if !cancel_inflight { - return Err(SessionRuntimeError::BadRequest(format!( - "thread {} has active execution {}; pass cancel_inflight=true to release it", - thread_key.as_str(), - active_execution.execution_id - ))); - } - if matches!( - active_execution.status, - ExecutionStatus::Queued | ExecutionStatus::Running - ) { - let cancel_error = Self::release_error_message( - release_id, - thread_key, - sandbox_id.as_deref(), - active_execution.execution_id.as_str(), - ); - if let Some(cancelled) = self - .store - .cancel_execution_if_active(&active_execution.execution_id, &cancel_error) - .await? - { - execution_cancelled = true; - self.execution_spans - .lock() - .await - .remove(&cancelled.execution_id); - self.store - .append_event( - thread_key, - Some(&cancelled.execution_id), - "session.execution_cancelled", - json!({ - "execution_id": cancelled.execution_id.as_str(), - "thread_key": thread_key.as_str(), - "sandbox_id": sandbox_id.as_deref(), - "release_id": release_id, - "reason": "thread_released", - }), - ) - .await?; - } - } - } - - let mut sandbox_released = false; - let mut sandbox_release_error = None; - if let Some(ref sandbox_id) = sandbox_id { - self.sandbox_pipes.remove(sandbox_id); - let id = SandboxId::new(sandbox_id.clone()); - match self.sandbox_runtime.manager.stop(&id).await { - Ok(()) | Err(SandboxError::NotFound(_)) => { - sandbox_released = true; - } - Err(error) => { - warn!( - thread_key = %thread_key, - sandbox_id = %sandbox_id, - %error, - "failed to release sandbox" - ); - sandbox_release_error = Some(error.to_string()); - } - } - } - - let session = self.store.release_session(thread_key).await?; - self.store - .append_event( - thread_key, - execution_id.as_deref(), - "session.released", - json!({ - "thread_key": thread_key.as_str(), - "release_id": release_id, - "cancel_inflight": cancel_inflight, - "sandbox_id": sandbox_id.as_deref(), - "sandbox_released": sandbox_released, - "sandbox_release_error": sandbox_release_error.as_deref(), - "execution_id": execution_id.as_deref(), - "execution_cancelled": execution_cancelled, - }), - ) - .await?; - - Ok(ReleaseThreadOutcome { - session, - release_id: release_id.map(ToOwned::to_owned), - cancel_inflight, - sandbox_released, - sandbox_release_error, - execution_id, - execution_cancelled, - }) + input: ExecuteSessionInput, + ) -> Result { + let operation_lock = self.session_operation_lock(thread_key); + let result = { + let _guard = operation_lock.lock().await; + self.execute_session_locked(thread_key, input).await + }; + self.release_session_operation_lock(thread_key, operation_lock); + result } - pub async fn execute_session( + async fn execute_session_locked( &self, thread_key: &ThreadKey, input: ExecuteSessionInput, @@ -1084,6 +2234,11 @@ impl SessionRuntime { ); return Ok(execution); } + if let Err(error) = self.claim_stdout_owner(&execution.execution_id).await { + self.record_execution_failure(thread_key, &execution.execution_id, &error) + .await; + return Err(error); + } let execution_trace_span = info_span!( "centaur.api_rs.session.execution", component = COMPONENT_SESSION_RUNTIME, @@ -1225,6 +2380,19 @@ impl SessionRuntime { ) { self.execution_spans.lock().await.remove(execution_id); let error_message = error.to_string(); + let execution = match self + .store + .fail_execution_if_active_and_stdout_owner( + execution_id, + &self.stdout_owner_id, + &error_message, + ) + .await + { + Ok(Some(execution)) => execution, + Ok(None) => return, + Err(_) => return, + }; let _ = self .store .append_event( @@ -1238,44 +2406,18 @@ impl SessionRuntime { }), ) .await; - if let Ok(execution) = self - .store - .fail_execution(execution_id, &error_message) - .await - { - record_finished_execution_metric( - &self.store, - thread_key, - &execution, - "failed", - Some(runtime_error_failure_class(error)), - ) - .await; - } + record_finished_execution_metric( + &self.store, + thread_key, + &execution, + "failed", + Some(runtime_error_failure_class(error)), + ) + .await; } - fn release_error_message( - release_id: Option<&str>, - thread_key: &ThreadKey, - sandbox_id: Option<&str>, - execution_id: &str, - ) -> String { - match release_id { - Some(release_id) => format!( - "thread released (release_id={release_id}, thread_key={}, sandbox_id={:?}, execution_id={execution_id})", - thread_key.as_str(), - sandbox_id, - ), - None => format!( - "thread released (thread_key={}, sandbox_id={:?}, execution_id={execution_id})", - thread_key.as_str(), - sandbox_id, - ), - } - } - - async fn forward_messages_to_active_execution( - &self, + async fn forward_messages_to_active_execution( + &self, thread_key: &ThreadKey, messages: &[SessionMessageInput], message_ids: &[String], @@ -1351,6 +2493,63 @@ impl SessionRuntime { } } + pub async fn interrupt_active_execution( + &self, + thread_key: &ThreadKey, + reason: &str, + ) -> Result { + let Some(execution) = self.store.active_execution_for_thread(thread_key).await? else { + return Ok(InterruptExecutionOutcome { + interrupted: false, + execution_id: None, + }); + }; + + let execution_span = self + .execution_spans + .lock() + .await + .get(&execution.execution_id) + .cloned(); + let trace = SessionTraceContext::new(thread_key, execution_span.as_ref()); + let input_lines = input_lines_with_session_context( + thread_key, + &trace, + &[interrupt_input_line(thread_key, reason)], + ); + + let pipe = self + .wait_for_active_steering_pipe(thread_key, &execution.execution_id) + .await + .map_err(SessionRuntimeError::BadRequest)?; + write_input_lines( + &pipe, + &input_lines, + thread_key, + &execution.execution_id, + None, + ) + .await?; + + self.store + .append_event( + thread_key, + Some(&execution.execution_id), + "session.interrupt_delivered", + json!({ + "execution_id": execution.execution_id, + "thread_key": thread_key.as_str(), + "reason": reason, + }), + ) + .await?; + + Ok(InterruptExecutionOutcome { + interrupted: true, + execution_id: Some(execution.execution_id), + }) + } + async fn wait_for_active_steering_pipe( &self, thread_key: &ThreadKey, @@ -1469,6 +2668,7 @@ impl SessionRuntime { &self, request: EnsureSessionSandboxRequest<'_>, ) -> Result { + let _allocation_guard = self.acquire_sandbox_allocation_permit().await?; let EnsureSessionSandboxRequest { thread_key, harness_type, @@ -1479,6 +2679,7 @@ impl SessionRuntime { desired_capabilities, execution_id, } = request; + let boot_mode = sandbox_boot_mode_for_thread(thread_key, iron_control_principal); let span = info_span!( "centaur.api_rs.sandbox.ensure", component = COMPONENT_SESSION_RUNTIME, @@ -1492,45 +2693,149 @@ impl SessionRuntime { existing_sandbox_id = existing_sandbox_id.unwrap_or(""), iron_control_principal_present = iron_control_principal.is_some(), persona_id = persona_id.unwrap_or(""), - sandbox_repo_cache_enabled = desired_capabilities.repo_cache_enabled, + sandbox_boot_mode = boot_mode.as_str(), + sandbox_repo_cache_access = desired_capabilities.repo_cache.as_str(), + sandbox_repo_cache_enabled = desired_capabilities.repo_cache_enabled(), sandbox_observability_enabled = desired_capabilities.observability_enabled, + sandbox_api_server_enabled = desired_capabilities.api_server_enabled, ); let ensure_started = Instant::now(); let result = async { - let persona_context = self.resolve_stored_persona(persona_id, harness_type)?; + let mut assignment_fence = existing_sandbox_id.map(ToOwned::to_owned); + let persona_context = + self.resolve_stored_persona(persona_id, harness_type, desired_capabilities)?; if let Some(sandbox_id) = existing_sandbox_id { let id = SandboxId::new(sandbox_id); - if !sandbox_capabilities_match(existing_sandbox_capabilities, desired_capabilities) - { + let persisted_session = self.store.get_session(thread_key).await?; + if persisted_session.sandbox_id.as_deref() != Some(sandbox_id) { + return Err(SessionRuntimeError::BadRequest(format!( + "session sandbox assignment changed while execution {execution_id} was starting" + ))); + } + let previous_content_revision = + persisted_session.sandbox_content_revision.clone(); + let desired_content_revision = self + .sandbox_runtime + .content_revision + .as_ref() + .map(|generation| sandbox_assignment_content_revision(generation, sandbox_id)); + let capabilities_mismatch = !sandbox_capabilities_match( + existing_sandbox_capabilities, + desired_capabilities, + ); + let content_revision_mismatch = desired_content_revision + .as_ref() + .is_some_and(|desired| previous_content_revision.as_ref() != Some(desired)); + if capabilities_mismatch || content_revision_mismatch { + if self + .store + .clear_sandbox_from_active_execution( + thread_key, + execution_id, + &self.stdout_owner_id, + sandbox_id, + ) + .await? + .is_none() + { + return Err(SessionRuntimeError::BadRequest(format!( + "execution {execution_id} is no longer authorized to replace sandbox {sandbox_id}" + ))); + } + assignment_fence = None; self.sandbox_pipes.remove(sandbox_id); match self.sandbox_runtime.manager.stop(&id).await { Ok(()) | Err(SandboxError::NotFound(_)) => {} - Err(error) => return Err(SessionRuntimeError::Sandbox(error)), + Err(error) => { + let previous_capabilities = existing_sandbox_capabilities + .cloned() + .unwrap_or_else(SessionSandboxCapabilities::default_enabled); + if let Err(restore_error) = self + .store + .assign_sandbox_to_active_execution( + thread_key, + execution_id, + &self.stdout_owner_id, + None, + sandbox_id, + previous_content_revision.as_deref(), + &previous_capabilities, + ) + .await + { + warn!( + %thread_key, + execution_id, + sandbox_id, + %restore_error, + "failed to restore sandbox assignment after replacement stop failed" + ); + } + return Err(SessionRuntimeError::Sandbox(error)); + } } - self.store.update_sandbox_id(thread_key, None).await?; - self.store - .append_event( - thread_key, - Some(execution_id), - "session.sandbox_capabilities_replaced", - json!({ - "execution_id": execution_id, - "thread_key": thread_key.as_str(), - "sandbox_id": sandbox_id, - "previous_capabilities": existing_sandbox_capabilities, - "desired_capabilities": desired_capabilities, - }), + if let Err(error) = self + .store + .mark_warm_sandbox_failed( + sandbox_id, + "assigned sandbox replaced for deployment compatibility", ) - .await?; + .await + { + warn!( + %thread_key, + execution_id, + sandbox_id, + %error, + "failed to retire replaced sandbox from warm-pool bookkeeping" + ); + } + if capabilities_mismatch { + self.store + .append_event( + thread_key, + Some(execution_id), + "session.sandbox_capabilities_replaced", + json!({ + "execution_id": execution_id, + "thread_key": thread_key.as_str(), + "sandbox_id": sandbox_id, + "previous_capabilities": existing_sandbox_capabilities, + "desired_capabilities": desired_capabilities, + }), + ) + .await?; + } + if content_revision_mismatch { + self.store + .append_event( + thread_key, + Some(execution_id), + "session.sandbox_content_replaced", + json!({ + "execution_id": execution_id, + "thread_key": thread_key.as_str(), + "sandbox_id": sandbox_id, + "previous_content_revision": previous_content_revision, + "desired_content_revision": desired_content_revision, + "deployment_content_revision": self.sandbox_runtime.content_revision.as_deref(), + }), + ) + .await?; + } info!( component = COMPONENT_SESSION_RUNTIME, - event = "sandbox_ensure_capabilities_replaced", + event = "sandbox_ensure_assignment_replaced", thread_key = %thread_key, execution_id, sandbox_id, - sandbox_repo_cache_enabled = desired_capabilities.repo_cache_enabled, + capabilities_mismatch, + content_revision_mismatch, + sandbox_repo_cache_access = desired_capabilities.repo_cache.as_str(), + sandbox_repo_cache_enabled = desired_capabilities.repo_cache_enabled(), sandbox_observability_enabled = desired_capabilities.observability_enabled, - "replacing existing sandbox whose capabilities do not match" + sandbox_api_server_enabled = desired_capabilities.api_server_enabled, + "replacing existing sandbox whose deployment contract does not match" ); } else { match self.sandbox_runtime.manager.status(&id).await { @@ -1542,6 +2847,16 @@ impl SessionRuntime { .ensure_iron_control_proxy_resources(&id, principal_id) .await?; } + self.commit_execution_sandbox_assignment( + thread_key, + execution_id, + assignment_fence.as_deref(), + sandbox_id, + desired_capabilities, + false, + false, + ) + .await?; span.record("centaur.sandbox_id", sandbox_id); span.record("sandbox_id", sandbox_id); let ready_duration = ensure_started.elapsed(); @@ -1570,8 +2885,33 @@ impl SessionRuntime { } ExistingSandboxAction::ResumeOrReplace => { self.sandbox_pipes.remove(sandbox_id); - match self.sandbox_runtime.manager.resume(&id).await { + let resume_id = id.clone(); + match self + .run_with_running_capacity( + thread_key, + execution_id, + "resume", + || async { + self.sandbox_runtime + .manager + .resume(&resume_id) + .await + .map_err(SessionRuntimeError::Sandbox) + }, + ) + .await + { Ok(()) => { + self.commit_execution_sandbox_assignment( + thread_key, + execution_id, + assignment_fence.as_deref(), + sandbox_id, + desired_capabilities, + false, + false, + ) + .await?; span.record("centaur.sandbox_id", sandbox_id); span.record("sandbox_id", sandbox_id); let ready_duration = ensure_started.elapsed(); @@ -1610,7 +2950,7 @@ impl SessionRuntime { ); return Ok(sandbox_id.to_owned()); } - Err(error) => { + Err(SessionRuntimeError::Sandbox(error)) => { warn!( component = COMPONENT_SESSION_RUNTIME, event = "sandbox_ensure_resume_failed", @@ -1634,6 +2974,7 @@ impl SessionRuntime { ) .await?; } + Err(error) => return Err(error), } } ExistingSandboxAction::Replace => { @@ -1684,7 +3025,8 @@ impl SessionRuntime { .warm_pool .as_ref() .filter(|_| { - warm_harness_matches + boot_mode.uses_warm_pool() + && warm_harness_matches && warm_persona_matches && desired_capabilities.is_default_enabled() }) @@ -1698,13 +3040,16 @@ impl SessionRuntime { span.record("centaur.sandbox_id", sandbox_id.as_str()); span.record("sandbox_id", sandbox_id.as_str()); let ready_duration = ensure_started.elapsed(); - self.store - .update_sandbox_assignment( - thread_key, - sandbox_id.as_str(), - desired_capabilities, - ) - .await?; + self.commit_execution_sandbox_assignment( + thread_key, + execution_id, + assignment_fence.as_deref(), + sandbox_id.as_str(), + desired_capabilities, + true, + true, + ) + .await?; self.store .append_event( thread_key, @@ -1759,16 +3104,32 @@ impl SessionRuntime { if let Some(principal) = iron_control_principal { spec.iron_control_principal = Some(principal.to_owned()); } + apply_sandbox_boot_mode(&mut spec, &boot_mode); apply_sandbox_capabilities(&mut spec, desired_capabilities); let create_started = Instant::now(); - let handle = self.sandbox_runtime.manager.create_running(spec).await?; + let handle = self + .run_with_running_capacity(thread_key, execution_id, "cold_create", || async { + self.sandbox_runtime + .manager + .create_running(spec) + .await + .map_err(SessionRuntimeError::Sandbox) + }) + .await?; let startup_duration = create_started.elapsed(); let ready_duration = ensure_started.elapsed(); span.record("centaur.sandbox_id", handle.id.as_str()); span.record("sandbox_id", handle.id.as_str()); - self.store - .update_sandbox_assignment(thread_key, handle.id.as_str(), desired_capabilities) - .await?; + self.commit_execution_sandbox_assignment( + thread_key, + execution_id, + assignment_fence.as_deref(), + handle.id.as_str(), + desired_capabilities, + true, + false, + ) + .await?; self.record_sandbox_ready(SandboxReadyObservation { thread_key, execution_id, @@ -1810,6 +3171,80 @@ impl SessionRuntime { result } + #[allow(clippy::too_many_arguments)] + async fn commit_execution_sandbox_assignment( + &self, + thread_key: &ThreadKey, + execution_id: &str, + expected_sandbox_id: Option<&str>, + sandbox_id: &str, + desired_capabilities: &SessionSandboxCapabilities, + newly_allocated: bool, + claimed_from_warm_pool: bool, + ) -> Result<(), SessionRuntimeError> { + let content_revision = self + .sandbox_runtime + .content_revision + .as_ref() + .map(|generation| sandbox_assignment_content_revision(generation, sandbox_id)); + if self + .store + .assign_sandbox_to_active_execution( + thread_key, + execution_id, + &self.stdout_owner_id, + expected_sandbox_id, + sandbox_id, + content_revision.as_deref(), + desired_capabilities, + ) + .await? + .is_some() + { + return Ok(()); + } + + if newly_allocated { + self.sandbox_pipes.remove(sandbox_id); + match self + .sandbox_runtime + .manager + .stop(&SandboxId::new(sandbox_id)) + .await + { + Ok(()) | Err(SandboxError::NotFound(_)) => {} + Err(error) => warn!( + %thread_key, + execution_id, + sandbox_id, + %error, + "failed to stop sandbox after execution assignment fence rejected it" + ), + } + if claimed_from_warm_pool + && let Err(error) = self + .store + .mark_warm_sandbox_failed( + sandbox_id, + "execution ended before warm sandbox assignment committed", + ) + .await + { + warn!( + %thread_key, + execution_id, + sandbox_id, + %error, + "failed to retire rejected warm sandbox assignment" + ); + } + } + + Err(SessionRuntimeError::BadRequest(format!( + "execution {execution_id} is no longer active or its sandbox assignment fence changed" + ))) + } + async fn resolve_sandbox_capabilities( &self, iron_control_principal: Option<&str>, @@ -1821,10 +3256,7 @@ impl SessionRuntime { return Ok(SessionSandboxCapabilities::default_enabled()); }; let principal = registrar.get_principal(principal_id).await?; - Ok(SessionSandboxCapabilities { - repo_cache_enabled: principal.sandbox_repo_cache_enabled, - observability_enabled: principal.sandbox_observability_enabled, - }) + Ok(sandbox_capabilities_from_principal(&principal)) } async fn record_sandbox_ready(&self, observation: SandboxReadyObservation<'_>) { @@ -1841,6 +3273,22 @@ impl SessionRuntime { let startup_duration_ms = startup_duration.map(duration_millis_u64).unwrap_or(0); let sandbox_started_for_request = startup_duration.is_some(); + if let Err(error) = self + .store + .touch_sandbox_activity(thread_key, sandbox_id) + .await + { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_sandbox_activity_touch_failed", + thread_key = %thread_key, + execution_id, + sandbox_id, + %error, + "failed to touch sandbox activity after sandbox ready" + ); + } + if let Err(error) = self .store .append_event( @@ -2033,7 +3481,52 @@ impl SessionRuntime { /// and re-arm the remaining max-duration deadline. /// 3. The sandbox is gone: record the failure honestly. pub async fn adopt_orphaned_executions(&self) { - let executions = match self.store.list_active_executions().await { + // A one-shot scan has no later tick to revisit skipped rows, so + // queued orphans are failed immediately regardless of age — the + // pre-rescan startup behavior. + self.run_orphan_adoption_scan(&mut OrphanAdoptionState::default(), None) + .await; + } + + /// Re-run the orphan adoption scan every `interval` for the lifetime of + /// the process (the first scan runs immediately). A startup-only scan + /// misses executions orphaned after it ran — most commonly the previous + /// pod of a rolling deploy reaching its termination grace period + /// mid-turn after the new pod already scanned — and those stay wedged + /// until the next deploy. + pub fn spawn_orphan_adoption(&self, interval: Duration) -> tokio::task::JoinHandle<()> { + let runtime = self.clone(); + tokio::spawn(async move { + let mut state = OrphanAdoptionState::default(); + let mut ticker = interval_at(Instant::now(), interval); + ticker.set_missed_tick_behavior(MissedTickBehavior::Delay); + loop { + ticker.tick().await; + if runtime.shutting_down.load(Ordering::SeqCst) { + return; + } + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + } + }) + } + + /// One pass over all active executions. `pre_sandbox_grace` is the + /// minimum age before a row awaiting sandbox assignment is treated as + /// orphaned; `None` is only correct when no re-scan will follow. + async fn run_orphan_adoption_scan( + &self, + state: &mut OrphanAdoptionState, + pre_sandbox_grace: Option, + ) { + // Serialize the complete select/claim/attach pass with shutdown. A + // pre-shutdown scan may finish, but handoff then observes its lease; + // no scan can claim a new lease after handoff takes the write side. + let Ok(_allocation_guard) = self.acquire_sandbox_allocation_permit().await else { + return; + }; + let executions = match self.store.list_active_executions_with_ownership().await { Ok(executions) => executions, Err(error) => { warn!( @@ -2046,37 +3539,135 @@ impl SessionRuntime { } }; if executions.is_empty() { + state.deferred.clear(); return; } + let mut adopted = 0_usize; + let mut failed = 0_usize; + let mut skipped = 0_usize; + let mut own = 0_usize; + let mut deferred = HashSet::new(); + for candidate in executions { + let execution_id = candidate.execution.execution_id.clone(); + // Advisory fast path: a live lease means the execution has an + // active pump somewhere. Skip our own executions silently and + // defer peers' without touching the session row or the sandbox + // backend — the conditional claim below stays the sole authority + // on ownership. + if candidate.stdout_owner_lease_active { + if candidate.stdout_owner_id.as_deref() == Some(self.stdout_owner_id.as_str()) { + own += 1; + continue; + } + if !state.deferred.contains(&execution_id) { + self.record_adoption_deferral(&candidate.execution).await; + } + deferred.insert(execution_id); + continue; + } + let record_deferral = !state.deferred.contains(&execution_id); + match self + .adopt_orphaned_execution(&candidate.execution, record_deferral, pre_sandbox_grace) + .await + { + Ok(OrphanAdoption::Adopted) => adopted += 1, + Ok(OrphanAdoption::Failed) => failed += 1, + Ok(OrphanAdoption::Skipped) => skipped += 1, + Ok(OrphanAdoption::Deferred) => { + deferred.insert(execution_id); + } + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_failed", + thread_key = %candidate.execution.thread_key, + execution_id = %candidate.execution.execution_id, + %error, + "failed to adopt orphaned execution; will retry on the next scan" + ); + // Keep the dedup entry across transient errors so a + // recovered deferral is not re-recorded. + if state.deferred.contains(&execution_id) { + deferred.insert(execution_id); + } + } + } + } + state.deferred = deferred; + if adopted > 0 || failed > 0 { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_scan", + adopted, + failed, + deferred = state.deferred.len(), + skipped, + own, + "adopted executions orphaned by a previous control plane process" + ); + } else { + debug!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_scan", + adopted, + failed, + deferred = state.deferred.len(), + skipped, + own, + "orphan adoption scan found nothing adoptable" + ); + } + } + + async fn record_adoption_deferral(&self, execution: &SessionExecution) { info!( component = COMPONENT_SESSION_RUNTIME, - event = "execution_adoption_scan", - orphan_count = executions.len(), - "adopting executions orphaned by a previous control plane process" + event = "execution_adoption_deferred", + thread_key = %execution.thread_key, + execution_id = %execution.execution_id, + "active stdout owner lease still exists; deferring adoption" ); - for execution in executions { - if let Err(error) = self.adopt_orphaned_execution(&execution).await { - warn!( - component = COMPONENT_SESSION_RUNTIME, - event = "execution_adoption_failed", - thread_key = %execution.thread_key, - execution_id = %execution.execution_id, - %error, - "failed to adopt orphaned execution; will retry on next startup" - ); - } - } + let _ = self + .store + .append_event( + &execution.thread_key, + Some(&execution.execution_id), + "session.execution_adoption_deferred", + json!({ "reason": "stdout_owner_lease_active" }), + ) + .await; } async fn adopt_orphaned_execution( &self, execution: &SessionExecution, - ) -> Result<(), SessionRuntimeError> { + record_deferral: bool, + pre_sandbox_grace: Option, + ) -> Result { let thread_key = &execution.thread_key; let execution_id = execution.execution_id.as_str(); if execution.status == ExecutionStatus::Queued { // Input is only written after an execution is marked running, so // a queued orphan never reached the harness: nothing can come. + // On a periodic scan, young queued rows are skipped instead of + // failed: they are most likely a live execute_session observed + // mid-transition, and a later tick revisits them. + if let Some(grace) = pre_sandbox_grace { + let age = SystemTime::now() + .duration_since(SystemTime::from(execution.created_at)) + .unwrap_or_default(); + if age < grace { + debug!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_skipped", + thread_key = %thread_key, + execution_id, + age_ms = duration_millis_u64(age), + "skipping young queued execution; a live execute may still claim it" + ); + return Ok(OrphanAdoption::Skipped); + } + } self.fail_orphaned_execution( thread_key, execution_id, @@ -2084,10 +3675,25 @@ impl SessionRuntime { "orphaned before input was sent", ) .await; - return Ok(()); + return Ok(OrphanAdoption::Failed); } let session = self.store.get_session(thread_key).await?; let Some(sandbox_id) = session.sandbox_id.as_deref() else { + let running_since = execution.started_at.unwrap_or(execution.created_at); + let running_age = SystemTime::now() + .duration_since(SystemTime::from(running_since)) + .unwrap_or_default(); + if pre_sandbox_grace.is_some_and(|grace| running_age < grace) { + debug!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_skipped", + thread_key = %thread_key, + execution_id, + age_ms = duration_millis_u64(running_age), + "skipping young running execution awaiting sandbox assignment" + ); + return Ok(OrphanAdoption::Skipped); + } self.fail_orphaned_execution( thread_key, execution_id, @@ -2095,7 +3701,7 @@ impl SessionRuntime { "orphaned with no sandbox assigned", ) .await; - return Ok(()); + return Ok(OrphanAdoption::Failed); }; let id = SandboxId::new(sandbox_id); let status = match self.sandbox_runtime.manager.status(&id).await { @@ -2113,7 +3719,25 @@ impl SessionRuntime { &format!("sandbox no longer accepts io (status {status:?})"), ) .await; - return Ok(()); + return Ok(OrphanAdoption::Failed); + } + if !self.claim_expired_stdout_owner(execution_id).await? { + // Deferrals repeat on every periodic scan while another control + // plane pumps the execution; only the first observation is worth + // an info log and a durable event. + if record_deferral { + self.record_adoption_deferral(execution).await; + } else { + debug!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_adoption_deferred", + thread_key = %thread_key, + execution_id, + sandbox_id, + "active stdout owner lease still exists; deferring adoption" + ); + } + return Ok(OrphanAdoption::Deferred); } // The turn may have finished while no control plane was attached. An @@ -2168,13 +3792,19 @@ impl SessionRuntime { terminal, ) .await?; - return Ok(()); + return Ok(OrphanAdoption::Adopted); } // No terminal in the recorded output: treat the turn as still in // flight. Re-attach the stdout pump and re-arm the remaining // max-duration budget so an adopted-but-silent turn stays bounded. - self.ensure_session_pipe(thread_key, sandbox_id).await?; + if let Err(error) = self.ensure_session_pipe(thread_key, sandbox_id).await { + let _ = self + .store + .release_stdout_owner(execution_id, &self.stdout_owner_id) + .await; + return Err(error); + } info!( component = COMPONENT_SESSION_RUNTIME, event = "execution_adopted", @@ -2205,7 +3835,7 @@ impl SessionRuntime { idle_timeout_from_execution(execution), ); } - Ok(()) + Ok(OrphanAdoption::Adopted) } async fn fail_orphaned_execution( @@ -2215,6 +3845,10 @@ impl SessionRuntime { sandbox_id: &str, detail: &str, ) { + let _ = self + .store + .claim_stdout_owner(execution_id, &self.stdout_owner_id, STDOUT_OWNER_LEASE) + .await; let error = format!("execution orphaned by control plane restart; {detail}"); if let Err(record_error) = record_terminal_output( &self.context(), @@ -2235,6 +3869,218 @@ impl SessionRuntime { ); } } + + /// Hands off this control plane's in-flight executions before process + /// exit. Waits up to `timeout` for owned executions to finish naturally + /// (their stdout pumps keep running until the process exits), then + /// releases the remaining stdout-owner leases so another control + /// plane's adoption scan can claim the executions right away instead of + /// waiting out the lease TTL. Turn output produced after the release is + /// not lost: adoption replays it from the sandbox backend's recorded + /// output. + pub async fn handoff_owned_executions(&self, timeout: Duration) { + // Fence new stdout-owner claims first: an execution accepted after + // this point would otherwise claim a lease that outlives the + // process, stranding it until the lease TTL expires. + self.shutting_down.store(true, Ordering::SeqCst); + // Wait for any allocation or orphan-adoption pass that started before + // the fence. Holding the write side makes the following lease count a + // closed-world inventory. + let _allocation_guard = self.sandbox_allocation_gate.write().await; + let deadline = Instant::now() + .checked_add(timeout) + .unwrap_or_else(|| Instant::now() + Duration::from_secs(3600)); + loop { + let count = tokio::time::timeout( + EXECUTION_HANDOFF_DB_TIMEOUT, + self.store + .count_executions_with_stdout_owner(&self.stdout_owner_id), + ) + .await; + let Ok(count) = count else { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_count_timeout", + "timed out counting in-flight executions; releasing leases now" + ); + break; + }; + match count { + Ok(0) => { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_idle", + "no in-flight executions to hand off at shutdown" + ); + return; + } + Ok(in_flight) => { + if Instant::now() >= deadline { + break; + } + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_waiting", + in_flight, + "waiting for in-flight executions to finish before shutdown" + ); + } + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_count_failed", + %error, + "failed to count in-flight executions; releasing leases now" + ); + break; + } + } + sleep(EXECUTION_HANDOFF_POLL_INTERVAL).await; + } + let released = tokio::time::timeout( + EXECUTION_HANDOFF_DB_TIMEOUT, + self.store + .release_stdout_owned_executions(&self.stdout_owner_id), + ) + .await; + let Ok(released) = released else { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_release_timeout", + "timed out releasing stdout-owner leases; peers must wait for lease expiry" + ); + return; + }; + match released { + Ok(released) => { + for execution in &released { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_released", + thread_key = %execution.thread_key, + execution_id = %execution.execution_id, + "released stdout-owner lease at shutdown for adoption by a peer" + ); + let _ = self + .store + .append_event( + &execution.thread_key, + Some(&execution.execution_id), + "session.stdout_owner_released", + json!({ + "execution_id": execution.execution_id, + "reason": "control_plane_shutdown", + }), + ) + .await; + } + if released.is_empty() { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_idle", + "in-flight executions finished during the shutdown drain" + ); + } + } + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "execution_handoff_release_failed", + %error, + "failed to release stdout-owner leases at shutdown" + ); + } + } + } +} + +/// Outcome of one orphan-adoption attempt. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum OrphanAdoption { + /// Terminal output was recovered or a live pump was re-attached. + Adopted, + /// Another control plane still holds the stdout-owner lease. + Deferred, + /// The execution was failed as unrecoverable. + Failed, + /// Too young to judge (freshly queued); revisit on a later scan. + Skipped, +} + +/// Scan state carried across periodic orphan-adoption ticks. +#[derive(Debug, Default)] +struct OrphanAdoptionState { + /// Executions whose deferral was already recorded, so long-lived leases + /// do not produce a `session.execution_adoption_deferred` event on every + /// tick. + deferred: HashSet, +} + +async fn maybe_generate_session_title( + store: PgSessionStore, + generator: SessionTitleGenerator, + thread_key: ThreadKey, +) { + let parts = match store.title_generation_candidate(&thread_key).await { + Ok(Some(parts)) => parts, + Ok(None) => return, + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_title_candidate_failed", + thread_key = %thread_key, + %error, + "failed to load session title candidate" + ); + return; + } + }; + let Some(source) = session_title_source_from_parts(&parts) else { + return; + }; + let raw_title = match generator(source).await { + Ok(title) => title, + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_title_generation_failed", + thread_key = %thread_key, + %error, + "failed to generate session title" + ); + return; + } + }; + let Some(title) = sanitize_session_title(&raw_title) else { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_title_generation_empty", + thread_key = %thread_key, + "session title generation returned an empty title" + ); + return; + }; + match store.set_session_title_if_empty(&thread_key, &title).await { + Ok(true) => { + info!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_title_set", + thread_key = %thread_key, + title, + "session title set" + ); + } + Ok(false) => {} + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_title_set_failed", + thread_key = %thread_key, + %error, + "failed to set session title" + ); + } + } } impl SandboxRuntime { @@ -2268,6 +4114,12 @@ impl SandboxRuntime { workload: SandboxWorkloadMode, ) -> Self { let warm_harness = workload.default_harness(); + // One deployment-wide generation is derived from the default warm + // spec and stamped on every assignment, including cold non-default + // harness/persona sandboxes. The warm spec includes the base image, + // mounts, complete environment (model/config included), and the chart's + // full boot-content manifest fingerprint. + let content_revision = workload.deployment_content_revision(); let warm_workload = workload.clone(); let mut runtime = Self::backend_with_warm_spec_factory( backend, @@ -2277,6 +4129,7 @@ impl SandboxRuntime { move || warm_workload.warm_spec(), ); runtime.warm_harness = warm_harness; + runtime.content_revision = Some(content_revision); runtime } @@ -2292,6 +4145,7 @@ impl SandboxRuntime { spec_factory: Arc::new(spec_factory), warm_spec_factory: None, workload_key: None, + content_revision: None, warm_harness: None, } } @@ -2315,6 +4169,7 @@ impl SandboxRuntime { spec_factory: Arc::new(spec_factory), warm_spec_factory: Some(warm_spec_factory), workload_key: Some(workload_key), + content_revision: None, warm_harness: None, } } @@ -2355,6 +4210,10 @@ impl SandboxWorkloadMode { } } + fn deployment_content_revision(&self) -> String { + sandbox_spec_key(&self.warm_spec()) + } + fn spec( &self, thread_key: &ThreadKey, @@ -2426,6 +4285,15 @@ fn sandbox_spec_key(spec: &SandboxSpec) -> String { format!("sandbox-spec-sha256:{digest:x}") } +fn sandbox_assignment_content_revision(generation: &str, sandbox_id: &str) -> String { + let mut digest = Sha256::new(); + digest.update(b"centaur-sandbox-assignment-v1\0"); + digest.update(generation.as_bytes()); + digest.update(b"\0"); + digest.update(sandbox_id.as_bytes()); + format!("sandbox-assignment-sha256:{:x}", digest.finalize()) +} + fn mock_app_server_script() -> &'static str { r#"while IFS= read -r line; do model="$(printf '%s\n' "$line" | sed -n 's/.*"model":"\([^"]*\)".*/\1/p')" @@ -2486,6 +4354,20 @@ fn session_event_stream( if let Some(event) = state.pending.pop_front() { state.after_event_id = event.event_id; state.emitted_count += 1; + // Execution-scoped streams are per-turn: after the + // execution's terminal event nothing else will ever + // arrive, so complete the response instead of parking + // forever. Abandoned client connections otherwise pin + // this stream's dedicated LISTEN connection until the + // TCP peer is proven dead (the 2026-07-06 incident + // exhausted both the Slackbot fetch pool and staging + // Postgres this way). The 30s safety tick makes this + // robust even when the notify is missed. + if state.execution_id.is_some() + && is_terminal_execution_event(&event.event_type) + { + state.done = true; + } return Some((Ok(event), state)); } if state.done { @@ -2541,6 +4423,15 @@ fn session_event_stream( ) } +/// Terminal event types for a single execution: once one of these is emitted +/// on an execution-scoped stream, the stream has nothing left to deliver. +fn is_terminal_execution_event(event_type: &str) -> bool { + matches!( + event_type, + "session.execution_completed" | "session.execution_failed" | "session.execution_cancelled" + ) +} + /// How a stdout pump pass ended once the attach stream closed. enum StdoutPumpEnd { /// The stream closed with no execution in flight, or the execution was @@ -2925,6 +4816,7 @@ async fn run_stdout_pump( "session stdout pump started" ); let mut output_state = StdoutPumpState::default(); + let mut lost_stdout_ownership = HashSet::new(); let mut line_count = 0_u64; while let Some(line) = stdout.next().await { let line = match line { @@ -2953,6 +4845,9 @@ async fn run_stdout_pump( else { continue; }; + if lost_stdout_ownership.contains(&output_execution_id) { + continue; + } let first_token_execution = active_execution .as_ref() .filter(|execution| { @@ -2975,10 +4870,24 @@ async fn run_stdout_pump( sandbox_id, &output_execution_id, ); - let output_event = - append_output_line(&ctx.store, &thread_key, Some(&output_execution_id), &line) - .instrument(output_span.clone()) - .await?; + let Some(output_event) = + append_output_line(&ctx, &thread_key, &output_execution_id, &line) + .instrument(output_span.clone()) + .await? + else { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_stdout_owner_lost", + thread_key = %thread_key, + execution_id = %output_execution_id, + sandbox_id, + stdout_owner_id = %ctx.stdout_owner_id, + "stdout pump no longer owns execution output; suppressing further rows" + ); + lost_stdout_ownership.insert(output_execution_id.clone()); + output_state.forget(&output_execution_id); + continue; + }; if let Some(execution) = first_token_execution { record_first_token_observation( &ctx, @@ -3729,6 +5638,9 @@ enum TerminalOutput { reason: &'static str, result_text: Option, }, + Cancelled { + reason: &'static str, + }, Failed { error: String, }, @@ -3747,7 +5659,10 @@ async fn record_terminal_output( reason, result_text, } => { - let Some(execution) = ctx.store.complete_execution_if_active(execution_id).await? + let Some(execution) = ctx + .store + .complete_execution_if_active_and_stdout_owner(execution_id, &ctx.stdout_owner_id) + .await? else { return Ok(()); }; @@ -3771,11 +5686,41 @@ async fn record_terminal_output( .await?; (execution, "completed") } + TerminalOutput::Cancelled { reason } => { + let Some(execution) = ctx + .store + .cancel_execution_if_active_and_stdout_owner( + execution_id, + &ctx.stdout_owner_id, + reason, + ) + .await? + else { + return Ok(()); + }; + ctx.store + .append_event( + thread_key, + Some(execution_id), + "session.execution_cancelled", + json!({ + "execution_id": execution_id, + "thread_key": thread_key.as_str(), + "reason": reason, + }), + ) + .await?; + (execution, "cancelled") + } TerminalOutput::Failed { error } => { failure_class = Some(terminal_failure_class(&error)); let Some(execution) = ctx .store - .fail_execution_if_active(execution_id, &error) + .fail_execution_if_active_and_stdout_owner( + execution_id, + &ctx.stdout_owner_id, + &error, + ) .await? else { return Ok(()); @@ -3796,6 +5741,21 @@ async fn record_terminal_output( } }; ctx.execution_spans.lock().await.remove(execution_id); + if let Err(error) = ctx + .store + .touch_sandbox_activity(thread_key, sandbox_id) + .await + { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_sandbox_activity_touch_failed", + thread_key = %thread_key, + execution_id, + sandbox_id, + %error, + "failed to touch sandbox activity after terminal output" + ); + } record_finished_execution_metric( &ctx.store, thread_key, @@ -3839,6 +5799,33 @@ fn spawn_max_duration_failure( }); } +fn spawn_stdout_owner_renewer(ctx: RuntimeContext, execution_id: String) { + tokio::spawn(async move { + loop { + sleep(STDOUT_OWNER_RENEW_INTERVAL).await; + match ctx + .store + .renew_stdout_owner(&execution_id, &ctx.stdout_owner_id, STDOUT_OWNER_LEASE) + .await + { + Ok(true) => {} + Ok(false) => break, + Err(error) => { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_stdout_owner_renew_failed", + execution_id, + stdout_owner_id = %ctx.stdout_owner_id, + %error, + "failed to renew stdout owner lease" + ); + break; + } + } + } + }); +} + async fn record_max_duration_failure( ctx: &RuntimeContext, thread_key: &ThreadKey, @@ -3850,12 +5837,22 @@ async fn record_max_duration_failure( let error = format!("execution exceeded max_duration_ms={max_duration_ms}"); let Some(execution) = ctx .store - .fail_execution_if_active(execution_id, &error) + .fail_execution_if_active_and_stdout_owner(execution_id, &ctx.stdout_owner_id, &error) .await? else { return Ok(()); }; ctx.execution_spans.lock().await.remove(execution_id); + if let Err(error) = ctx.store.touch_session_sandbox_activity(thread_key).await { + warn!( + component = COMPONENT_SESSION_RUNTIME, + event = "session_sandbox_activity_touch_failed", + thread_key = %thread_key, + execution_id, + %error, + "failed to touch sandbox activity after max duration" + ); + } ctx.store .append_event( thread_key, @@ -4034,6 +6031,23 @@ fn duration_millis_u64(duration: Duration) -> u64 { duration.as_millis().min(u128::from(u64::MAX)) as u64 } +fn release_error_message( + release_id: Option<&str>, + thread_key: &ThreadKey, + sandbox_id: Option<&str>, +) -> String { + match release_id { + Some(release_id) => format!( + "thread released (release_id={release_id}, thread_key={}, sandbox_id={sandbox_id:?})", + thread_key.as_str() + ), + None => format!( + "thread released (thread_key={}, sandbox_id={sandbox_id:?})", + thread_key.as_str() + ), + } +} + fn clean_persona_id(value: &str) -> Option<&str> { let value = value.trim(); if value.is_empty() { None } else { Some(value) } @@ -4058,30 +6072,116 @@ fn sandbox_capabilities_match( ) } +fn sandbox_repo_cache_access_from_principal( + principal: ¢aur_iron_control::Principal, +) -> SessionRepoCacheAccess { + match principal + .labels + .get(SANDBOX_REPO_CACHE_LABEL) + .map(|value| value.trim().to_ascii_lowercase()) + { + Some(value) if value == "all" => SessionRepoCacheAccess::All, + Some(value) if value == "public" => SessionRepoCacheAccess::Public, + Some(_) => SessionRepoCacheAccess::None, + None => SessionRepoCacheAccess::None, + } +} + +fn sandbox_capabilities_from_principal( + principal: ¢aur_iron_control::Principal, +) -> SessionSandboxCapabilities { + SessionSandboxCapabilities { + repo_cache: sandbox_repo_cache_access_from_principal(principal), + observability_enabled: principal.sandbox_observability_enabled, + api_server_enabled: principal.sandbox_api_server_enabled, + } +} + fn apply_sandbox_capabilities(spec: &mut SandboxSpec, capabilities: &SessionSandboxCapabilities) { spec.capabilities = BackendSandboxCapabilities { - repo_cache_enabled: capabilities.repo_cache_enabled, + repo_cache: match capabilities.repo_cache { + SessionRepoCacheAccess::None => RepoCacheAccess::None, + SessionRepoCacheAccess::Public => RepoCacheAccess::Public, + SessionRepoCacheAccess::All => RepoCacheAccess::All, + }, observability_enabled: capabilities.observability_enabled, + api_server_enabled: capabilities.api_server_enabled, }; upsert_spec_env( spec, "CENTAUR_SANDBOX_REPO_CACHE_ENABLED", - capabilities.repo_cache_enabled.to_string(), + capabilities.repo_cache_enabled().to_string(), + ); + upsert_spec_env( + spec, + "CENTAUR_SANDBOX_REPO_CACHE_ACCESS", + capabilities.repo_cache.as_str().to_owned(), ); upsert_spec_env( spec, "CENTAUR_SANDBOX_OBSERVABILITY_ENABLED", capabilities.observability_enabled.to_string(), ); - if !capabilities.repo_cache_enabled { - spec.mounts - .retain(|mount| mount.target_path != SANDBOX_REPOS_MOUNT_PATH); + upsert_spec_env( + spec, + "CENTAUR_SANDBOX_API_SERVER_ENABLED", + capabilities.api_server_enabled.to_string(), + ); + match capabilities.repo_cache { + SessionRepoCacheAccess::None => { + spec.mounts + .retain(|mount| mount.target_path != SANDBOX_REPOS_MOUNT_PATH); + remove_spec_env(spec, CENTAUR_SKILL_DIRS_ENV); + } + SessionRepoCacheAccess::Public => { + scope_repo_cache_mounts_to_public(spec); + scope_skill_dirs_to_public(spec); + } + SessionRepoCacheAccess::All => { + remove_spec_env(spec, CENTAUR_PUBLIC_SKILL_DIRS_ENV); + } } + remove_spec_env(spec, CENTAUR_PUBLIC_SKILL_DIRS_ENV); if !capabilities.observability_enabled { append_spec_env_csv(spec, "TOOL_BLOCKLIST", OBSERVABILITY_TOOL_BLOCKLIST); } } +fn scope_repo_cache_mounts_to_public(spec: &mut SandboxSpec) { + for mount in spec + .mounts + .iter_mut() + .filter(|mount| mount.target_path == SANDBOX_REPOS_MOUNT_PATH) + { + match &mut mount.kind { + centaur_sandbox_core::MountKind::Bind { source_path } => { + *source_path = format!( + "{}/{}", + source_path.trim_end_matches('/'), + PUBLIC_REPO_CACHE_SUBPATH + ); + } + centaur_sandbox_core::MountKind::NamedVolume(_) => { + mount.sub_path = Some(PUBLIC_REPO_CACHE_SUBPATH.to_owned()); + } + centaur_sandbox_core::MountKind::EmptyDir => {} + } + } +} + +fn scope_skill_dirs_to_public(spec: &mut SandboxSpec) { + let public_skill_dirs = spec + .env + .iter() + .find(|env| env.name == CENTAUR_PUBLIC_SKILL_DIRS_ENV) + .map(|env| env.value.trim().to_owned()) + .filter(|value| !value.is_empty()); + match public_skill_dirs { + Some(public_skill_dirs) => upsert_spec_env(spec, CENTAUR_SKILL_DIRS_ENV, public_skill_dirs), + None => remove_spec_env(spec, CENTAUR_SKILL_DIRS_ENV), + } +} + fn append_spec_env_csv(spec: &mut SandboxSpec, name: &str, values: &str) { let existing = spec .env @@ -4174,6 +6274,7 @@ fn execution_duration(execution: &SessionExecution) -> Option { fn runtime_error_failure_class(error: &SessionRuntimeError) -> &'static str { match error { SessionRuntimeError::BadRequest(_) => "bad_request", + SessionRuntimeError::ShuttingDown => "shutting_down", SessionRuntimeError::Store(_) => "store", SessionRuntimeError::Sandbox(SandboxError::NotFound(_)) => "sandbox_not_found", SessionRuntimeError::Sandbox(SandboxError::Unsupported { .. }) => "sandbox_unsupported", @@ -4183,6 +6284,7 @@ fn runtime_error_failure_class(error: &SessionRuntimeError) -> &'static str { SessionRuntimeError::Sandbox(SandboxError::InvalidSpec(_)) => "sandbox_invalid_spec", SessionRuntimeError::IronControl(_) => "iron_control", SessionRuntimeError::WarmPool(_) => "warm_pool", + SessionRuntimeError::CapacityExceeded { .. } => "capacity", } } @@ -4275,6 +6377,11 @@ fn completed_turn_terminal_output(value: &Value, prior_final_answer_text: &str) prior_final_answer_text, ) } + Some("interrupted") if prior_final_answer_text.trim().is_empty() => { + TerminalOutput::Cancelled { + reason: "turn_interrupted", + } + } Some(_status) if !prior_final_answer_text.trim().is_empty() => { completed_terminal_output_with_fallback( value, @@ -4691,17 +6798,33 @@ fn steering_input_line( .ok() } -async fn append_output_line( - store: &PgSessionStore, +fn interrupt_input_line(thread_key: &ThreadKey, reason: &str) -> String { + serde_json::to_string(&json!({ + "type": "interrupt", + "thread_key": thread_key.as_str(), + "trace_metadata": { + "source": "session.interrupt_active_execution", + "action": "interrupt_active_execution", + "reason": reason, + }, + })) + .expect("interrupt input line serializes") +} + +async fn append_output_line( + ctx: &RuntimeContext, thread_key: &ThreadKey, - execution_id: Option<&str>, + execution_id: &str, line: &str, -) -> Result { +) -> Result, SessionRuntimeError> { let safe_line = redact_sensitive_text(line); - let event = store - .append_event( + let event = ctx + .store + .append_event_if_stdout_owner( thread_key, execution_id, + &ctx.stdout_owner_id, + STDOUT_OWNER_LEASE, SESSION_OUTPUT_LINE_EVENT, Value::String(safe_line), ) @@ -4926,6 +7049,66 @@ fn nonzero_duration_millis(value: u64) -> Result Ok(Duration::from_millis(value)) } +fn tool_host_thread_key(principal_id: &str) -> Result { + ThreadKey::parse(format!("mcp:{principal_id}")) + .map_err(|error| SessionRuntimeError::BadRequest(error.to_string())) +} + +/// Session/principal metadata recorded for observability; runtime behavior +/// derives from the `mcp:` thread-key prefix, not from these fields. +fn tool_host_session_metadata(principal_id: &str) -> Value { + json!({ + "mcp_tool_host": true, + "mcp_principal_id": principal_id, + }) +} + +fn sandbox_boot_mode_for_thread( + thread_key: &ThreadKey, + iron_control_principal: Option<&str>, +) -> SandboxBootMode { + let Some(thread_principal_id) = thread_key.as_str().strip_prefix("mcp:") else { + return SandboxBootMode::Harness; + }; + let principal_id = iron_control_principal + .unwrap_or(thread_principal_id) + .to_owned(); + SandboxBootMode::ToolHost { principal_id } +} + +fn apply_sandbox_boot_mode(spec: &mut SandboxSpec, boot_mode: &SandboxBootMode) { + let SandboxBootMode::ToolHost { principal_id } = boot_mode else { + return; + }; + spec.labels + .insert("centaur.ai/component".to_owned(), "tool-host".to_owned()); + spec.labels + .insert("centaur.ai/workload".to_owned(), "mcp-tool-host".to_owned()); + if !principal_id.trim().is_empty() { + spec.iron_control_principal = Some(principal_id.to_owned()); + upsert_spec_env(spec, "CENTAUR_MCP_PRINCIPAL_ID", principal_id.to_owned()); + } + configure_tool_host_command(spec); +} + +fn configure_tool_host_command(spec: &mut SandboxSpec) { + if should_preserve_entrypoint_for_tool_host(spec) { + spec.command = Some(vec!["/entrypoint.sh".to_owned()]); + spec.args = vec!["centaur-tool-host".to_owned()]; + } else { + spec.command = Some(vec!["centaur-tool-host".to_owned()]); + spec.args.clear(); + } +} + +fn should_preserve_entrypoint_for_tool_host(spec: &SandboxSpec) -> bool { + spec.command + .as_ref() + .and_then(|command| command.first()) + .is_some_and(|program| program == "/entrypoint.sh") + || spec.args.first().is_some_and(|arg| arg == "harness-server") +} + fn execution_metadata( metadata: Option, idle_timeout_ms: Option, @@ -4985,6 +7168,8 @@ fn terminal_output_from_lines(lines: &[String]) -> Option { pub enum SessionRuntimeError { #[error("{0}")] BadRequest(String), + #[error("control plane is shutting down")] + ShuttingDown, #[error(transparent)] Store(#[from] SessionStoreError), #[error(transparent)] @@ -4993,6 +7178,14 @@ pub enum SessionRuntimeError { IronControl(#[from] centaur_iron_control::IronControlError), #[error(transparent)] WarmPool(#[from] WarmPoolError), + #[error( + "sandbox running capacity exceeded during {operation}: running={running}, max_running={max_running}" + )] + CapacityExceeded { + max_running: usize, + running: usize, + operation: &'static str, + }, } #[cfg(test)] @@ -5003,6 +7196,170 @@ mod tests { use serde_json::json; use time::OffsetDateTime; + #[test] + fn sandbox_repo_cache_label_controls_access() { + assert_eq!( + sandbox_repo_cache_access_from_principal(&test_principal( + std::collections::BTreeMap::new() + )), + SessionRepoCacheAccess::None + ); + for value in ["none", "private", "bogus"] { + assert_eq!( + sandbox_repo_cache_access_from_principal(&test_principal( + std::collections::BTreeMap::from([( + SANDBOX_REPO_CACHE_LABEL.to_owned(), + value.to_owned(), + )]) + )), + SessionRepoCacheAccess::None + ); + } + assert_eq!( + sandbox_repo_cache_access_from_principal(&test_principal( + std::collections::BTreeMap::from([( + SANDBOX_REPO_CACHE_LABEL.to_owned(), + "public".to_owned(), + )]) + )), + SessionRepoCacheAccess::Public + ); + assert_eq!( + sandbox_repo_cache_access_from_principal(&test_principal( + std::collections::BTreeMap::from([( + SANDBOX_REPO_CACHE_LABEL.to_owned(), + "all".to_owned(), + )]) + )), + SessionRepoCacheAccess::All + ); + } + + #[test] + fn public_repo_cache_scopes_bind_mount_to_public_projection() { + let mut spec = SandboxSpec::new("mock").mount(Mount::new( + MountKind::Bind { + source_path: "/var/lib/centaur/repos".to_owned(), + }, + SANDBOX_REPOS_MOUNT_PATH, + )); + let capabilities = SessionSandboxCapabilities { + repo_cache: SessionRepoCacheAccess::Public, + observability_enabled: true, + api_server_enabled: true, + }; + + apply_sandbox_capabilities(&mut spec, &capabilities); + + assert_eq!(spec.capabilities.repo_cache, RepoCacheAccess::Public); + assert_eq!( + env_value(&spec, "CENTAUR_SANDBOX_REPO_CACHE_ACCESS"), + Some("public") + ); + assert_eq!( + spec.mounts[0].kind, + MountKind::Bind { + source_path: "/var/lib/centaur/repos/public".to_owned(), + } + ); + assert_eq!(spec.mounts[0].sub_path, None); + } + + #[test] + fn public_repo_cache_scopes_named_volume_to_public_subpath() { + let mut spec = SandboxSpec::new("mock").mount(Mount::new( + MountKind::NamedVolume("centaur-repo-cache".to_owned()), + SANDBOX_REPOS_MOUNT_PATH, + )); + let capabilities = SessionSandboxCapabilities { + repo_cache: SessionRepoCacheAccess::Public, + observability_enabled: true, + api_server_enabled: true, + }; + + apply_sandbox_capabilities(&mut spec, &capabilities); + + assert_eq!( + spec.mounts[0].kind, + MountKind::NamedVolume("centaur-repo-cache".to_owned()) + ); + assert_eq!(spec.mounts[0].sub_path.as_deref(), Some("public")); + } + + #[test] + fn public_repo_cache_scopes_skill_dirs_to_public_dirs() { + let mut spec = SandboxSpec::new("mock") + .env( + CENTAUR_SKILL_DIRS_ENV, + "/home/agent/github/acme/private/.agents/skills:\ + /home/agent/github/acme/public/.agents/skills", + ) + .env( + CENTAUR_PUBLIC_SKILL_DIRS_ENV, + "/home/agent/github/acme/public/.agents/skills", + ); + let capabilities = SessionSandboxCapabilities { + repo_cache: SessionRepoCacheAccess::Public, + observability_enabled: true, + api_server_enabled: true, + }; + + apply_sandbox_capabilities(&mut spec, &capabilities); + + assert_eq!( + env_value(&spec, CENTAUR_SKILL_DIRS_ENV), + Some("/home/agent/github/acme/public/.agents/skills") + ); + assert_eq!(env_value(&spec, CENTAUR_PUBLIC_SKILL_DIRS_ENV), None); + } + + #[test] + fn disabled_repo_cache_removes_repo_mount() { + let mut spec = SandboxSpec::new("mock") + .mount(Mount::new( + MountKind::Bind { + source_path: "/var/lib/centaur/repos".to_owned(), + }, + SANDBOX_REPOS_MOUNT_PATH, + )) + .mount(Mount::new(MountKind::EmptyDir, "/workspace")) + .env( + CENTAUR_SKILL_DIRS_ENV, + "/home/agent/github/acme/private/.agents/skills", + ) + .env( + CENTAUR_PUBLIC_SKILL_DIRS_ENV, + "/home/agent/github/acme/public/.agents/skills", + ); + let capabilities = SessionSandboxCapabilities { + repo_cache: SessionRepoCacheAccess::None, + observability_enabled: true, + api_server_enabled: true, + }; + + apply_sandbox_capabilities(&mut spec, &capabilities); + + assert_eq!(spec.capabilities.repo_cache, RepoCacheAccess::None); + assert_eq!(spec.mounts.len(), 1); + assert_eq!(spec.mounts[0].target_path, "/workspace"); + assert_eq!(env_value(&spec, CENTAUR_SKILL_DIRS_ENV), None); + assert_eq!(env_value(&spec, CENTAUR_PUBLIC_SKILL_DIRS_ENV), None); + } + + fn test_principal( + labels: std::collections::BTreeMap, + ) -> centaur_iron_control::Principal { + centaur_iron_control::Principal { + id: "prn_test".to_owned(), + namespace: "default".to_owned(), + foreign_id: Some("slack-channel-t-c".to_owned()), + name: "Test".to_owned(), + labels, + sandbox_observability_enabled: true, + sandbox_api_server_enabled: true, + } + } + #[test] fn persona_registry_validates_default_and_summarizes_without_prompt() { let registry = PersonaRegistry::new( @@ -5032,6 +7389,75 @@ mod tests { assert!(PersonaRegistry::new(Vec::new(), Some("missing".to_owned()), Vec::new()).is_err()); } + #[test] + fn persona_registry_limits_public_access_to_public_source_roots() { + let registry = PersonaRegistry::new( + [ + PersonaDefinition { + id: "private".to_owned(), + source_root: "/repo/private/tools".to_owned(), + source_path: "/repo/private/tools/personas/private".to_owned(), + source_ref: None, + prompt_hash: "sha256:private".to_owned(), + prompt: "private prompt".to_owned(), + }, + PersonaDefinition { + id: "public".to_owned(), + source_root: "/repo/public/tools".to_owned(), + source_path: "/repo/public/tools/personas/public".to_owned(), + source_ref: None, + prompt_hash: "sha256:public".to_owned(), + prompt: "public prompt".to_owned(), + }, + ], + Some("private".to_owned()), + vec![ + "/repo/private/tools".to_owned(), + "/repo/public/tools".to_owned(), + ], + ) + .unwrap() + .with_public_source_roots(["/repo/public/tools".to_owned()]); + + assert_eq!( + registry.default_persona_id_for_access(&SessionRepoCacheAccess::All), + Some("private") + ); + assert_eq!( + registry.default_persona_id_for_access(&SessionRepoCacheAccess::Public), + None + ); + assert!( + registry + .context_for_access("private", false, &SessionRepoCacheAccess::Public) + .is_err() + ); + assert_eq!( + registry + .context_for_access("public", false, &SessionRepoCacheAccess::Public) + .unwrap() + .persona_id, + "public" + ); + } + + #[test] + fn tool_host_command_preserves_sandbox_entrypoint_for_tool_setup() { + let thread_key = ThreadKey::parse("mcp:test").unwrap(); + let workload = SandboxWorkloadMode::codex_app_server( + "centaur-agent:latest", + [("TOOL_DIRS".to_owned(), "/app/tools".to_owned())], + HarnessType::Codex, + ); + let mut spec = workload.spec(&thread_key, &HarnessType::Codex, None); + + configure_tool_host_command(&mut spec); + + assert_eq!(spec.command, Some(vec!["/entrypoint.sh".to_owned()])); + assert_eq!(spec.args, vec!["centaur-tool-host"]); + assert_eq!(env_value(&spec, "TOOL_DIRS"), Some("/app/tools")); + } + #[test] fn turn_completed_without_answer_text_is_terminal() { let event = json!({ @@ -5103,7 +7529,7 @@ mod tests { } #[test] - fn interrupted_turn_completed_without_answer_is_failure() { + fn interrupted_turn_completed_without_answer_is_cancelled() { let event = json!({ "type": "turn.completed", "turn": {"id": "turn-1", "status": "interrupted"}, @@ -5111,8 +7537,8 @@ mod tests { assert_eq!( terminal_output(&event, ""), - Some(TerminalOutput::Failed { - error: "turn completed with status interrupted before final answer".to_owned() + Some(TerminalOutput::Cancelled { + reason: "turn_interrupted" }) ); } @@ -5744,6 +8170,54 @@ mod tests { ); } + #[test] + fn warm_workload_key_changes_with_bootstrap_fingerprint() { + let first = SandboxWorkloadMode::codex_app_server( + "centaur-agent:reviewed", + [( + "CENTAUR_SANDBOX_BOOTSTRAP_FINGERPRINT".to_owned(), + "ref-one".to_owned(), + )], + HarnessType::Codex, + ); + let second = SandboxWorkloadMode::codex_app_server( + "centaur-agent:reviewed", + [( + "CENTAUR_SANDBOX_BOOTSTRAP_FINGERPRINT".to_owned(), + "ref-two".to_owned(), + )], + HarnessType::Codex, + ); + + assert_ne!( + sandbox_spec_key(&first.warm_spec()), + sandbox_spec_key(&second.warm_spec()) + ); + assert_ne!( + first.deployment_content_revision(), + second.deployment_content_revision() + ); + } + + #[test] + fn deployment_content_revision_changes_with_default_model_env() { + let first = SandboxWorkloadMode::codex_app_server( + "centaur-agent:reviewed", + [("CODEX_MODEL".to_owned(), "gpt-5.5".to_owned())], + HarnessType::Codex, + ); + let second = SandboxWorkloadMode::codex_app_server( + "centaur-agent:reviewed", + [("CODEX_MODEL".to_owned(), "gpt-5.6".to_owned())], + HarnessType::Codex, + ); + + assert_ne!( + first.deployment_content_revision(), + second.deployment_content_revision() + ); + } + #[test] fn codex_workload_pins_harness_via_container_args() { let workload = SandboxWorkloadMode::codex_app_server( @@ -5915,13 +8389,16 @@ mod tests { let now = OffsetDateTime::now_utc(); Session { thread_key, + title: None, sandbox_id: Some(sandbox_id.to_owned()), + sandbox_content_revision: None, sandbox_capabilities: None, harness_type: HarnessType::Codex, harness_thread_id: None, persona_id: None, status: SessionStatus::Idle, iron_control_principal: None, + sandbox_last_active_at: Some(now), created_at: now, updated_at: now, } @@ -5974,12 +8451,15 @@ mod tests { #[cfg(test)] mod adoption_tests { use std::{ - collections::BTreeSet, + collections::{BTreeMap, BTreeSet}, sync::atomic::{AtomicBool, AtomicUsize, Ordering}, }; + use centaur_iron_control::IronControlClient; use centaur_sandbox_core::{ObservedSandbox, SandboxHandle, SandboxIo, SandboxResult}; - use tokio::io::{AsyncWriteExt, DuplexStream}; + use centaur_session_core::SessionStatus; + use tokio::io::{AsyncReadExt, AsyncWriteExt, DuplexStream}; + use tokio::sync::{Notify, OnceCell}; use super::*; @@ -5993,12 +8473,20 @@ mod adoption_tests { recorded_output: std::sync::Mutex>, open_count: AtomicUsize, status: std::sync::Mutex, + observed_statuses: std::sync::Mutex>, create_id: String, created_specs: std::sync::Mutex>, resume_fails: AtomicBool, + stop_fails: AtomicBool, stopped: std::sync::Mutex>, proxy_ensures: std::sync::Mutex>, missing_on_stop: std::sync::Mutex>, + block_create: AtomicBool, + create_started: Notify, + continue_create: Notify, + block_status: AtomicBool, + status_started: Notify, + continue_status: Notify, } impl MockBackend { @@ -6008,12 +8496,20 @@ mod adoption_tests { recorded_output: std::sync::Mutex::new(recorded_output), open_count: AtomicUsize::new(0), status: std::sync::Mutex::new(status), + observed_statuses: std::sync::Mutex::new(BTreeMap::new()), create_id: "mock-sbx".to_owned(), created_specs: std::sync::Mutex::new(Vec::new()), resume_fails: AtomicBool::new(false), + stop_fails: AtomicBool::new(false), stopped: std::sync::Mutex::new(Vec::new()), proxy_ensures: std::sync::Mutex::new(Vec::new()), missing_on_stop: std::sync::Mutex::new(BTreeSet::new()), + block_create: AtomicBool::new(false), + create_started: Notify::new(), + continue_create: Notify::new(), + block_status: AtomicBool::new(false), + status_started: Notify::new(), + continue_status: Notify::new(), } } @@ -6033,10 +8529,29 @@ mod adoption_tests { *self.status.lock().unwrap() = status; } + fn set_observed_status(&self, sandbox_id: &str, status: SandboxStatus) { + self.observed_statuses + .lock() + .unwrap() + .insert(sandbox_id.to_owned(), status); + } + + fn status_of(&self, sandbox_id: &str) -> Option { + self.observed_statuses + .lock() + .unwrap() + .get(sandbox_id) + .cloned() + } + fn fail_resume(&self) { self.resume_fails.store(true, Ordering::SeqCst); } + fn fail_stop(&self) { + self.stop_fails.store(true, Ordering::SeqCst); + } + fn mark_stop_missing(&self, sandbox_id: &str) { self.missing_on_stop .lock() @@ -6055,6 +8570,30 @@ mod adoption_tests { fn created_specs(&self) -> Vec { self.created_specs.lock().unwrap().clone() } + + fn block_next_create(&self) { + self.block_create.store(true, Ordering::SeqCst); + } + + async fn wait_for_create_start(&self) { + self.create_started.notified().await; + } + + fn continue_create(&self) { + self.continue_create.notify_one(); + } + + fn block_next_status(&self) { + self.block_status.store(true, Ordering::SeqCst); + } + + async fn wait_for_status_start(&self) { + self.status_started.notified().await; + } + + fn continue_status(&self) { + self.continue_status.notify_one(); + } } #[async_trait::async_trait] @@ -6064,7 +8603,12 @@ mod adoption_tests { } async fn create(&self, spec: SandboxSpec) -> SandboxResult { + if self.block_create.swap(false, Ordering::SeqCst) { + self.create_started.notify_one(); + self.continue_create.notified().await; + } self.created_specs.lock().unwrap().push(spec); + self.set_observed_status(&self.create_id, SandboxStatus::Running); Ok(SandboxHandle::new( SandboxId::new(self.create_id.clone()), "mock", @@ -6089,6 +8633,13 @@ mod adoption_tests { } async fn status(&self, _id: &SandboxId) -> SandboxResult { + if self.block_status.swap(false, Ordering::SeqCst) { + self.status_started.notify_one(); + self.continue_status.notified().await; + } + if let Some(status) = self.status_of(_id.as_str()) { + return Ok(status); + } Ok(self.status.lock().unwrap().clone()) } @@ -6098,14 +8649,24 @@ mod adoption_tests { } async fn list_observed(&self) -> SandboxResult> { - Ok(Vec::new()) + Ok(self + .observed_statuses + .lock() + .unwrap() + .iter() + .map(|(id, status)| ObservedSandbox::new(id.as_str(), "mock", status.clone())) + .collect()) } async fn stop(&self, id: &SandboxId) -> SandboxResult<()> { + if self.stop_fails.swap(false, Ordering::SeqCst) { + return Err(SandboxError::io("injected stop failure")); + } if self.missing_on_stop.lock().unwrap().contains(id.as_str()) { return Err(SandboxError::NotFound(id.as_str().to_owned())); } self.stopped.lock().unwrap().push(id.as_str().to_owned()); + self.set_observed_status(id.as_str(), SandboxStatus::Stopped); Ok(()) } @@ -6122,6 +8683,7 @@ mod adoption_tests { } async fn pause(&self, _id: &SandboxId) -> SandboxResult<()> { + self.set_observed_status(_id.as_str(), SandboxStatus::Suspended); Ok(()) } @@ -6129,6 +8691,7 @@ mod adoption_tests { if self.resume_fails.load(Ordering::SeqCst) { return Err(SandboxError::NotFound(_id.as_str().to_owned())); } + self.set_observed_status(_id.as_str(), SandboxStatus::Running); Ok(()) } } @@ -6145,6 +8708,156 @@ mod adoption_tests { (io, stdout_far, stdin_far) } + async fn spawn_principal_capability_stub() -> (String, tokio::task::JoinHandle<()>) { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("bind principal stub"); + let base_url = format!("http://{}", listener.local_addr().expect("stub address")); + let handle = tokio::spawn(async move { + loop { + let Ok((mut stream, _)) = listener.accept().await else { + return; + }; + let mut request = Vec::new(); + let mut buf = [0u8; 1024]; + while !request.windows(4).any(|window| window == b"\r\n\r\n") { + match stream.read(&mut buf).await { + Ok(0) | Err(_) => break, + Ok(read) => request.extend_from_slice(&buf[..read]), + } + } + let request = String::from_utf8_lossy(&request); + let path = request + .lines() + .next() + .and_then(|line| line.split_whitespace().nth(1)) + .unwrap_or_default(); + let (status, body) = match path { + "/api/v1/principals/prn_public" => ( + "200 OK", + json!({"data": { + "id": "prn_public", + "namespace": "default", + "foreign_id": "public-caller", + "name": "Public caller", + "labels": {(SANDBOX_REPO_CACHE_LABEL): "public"}, + "sandbox_repo_cache_enabled": false, + "sandbox_observability_enabled": true, + "sandbox_api_server_enabled": true + }}) + .to_string(), + ), + "/api/v1/principals/prn_none" => ( + "200 OK", + json!({"data": { + "id": "prn_none", + "namespace": "default", + "foreign_id": "restricted-caller", + "name": "Restricted caller", + "labels": {(SANDBOX_REPO_CACHE_LABEL): "none"}, + "sandbox_repo_cache_enabled": false, + "sandbox_observability_enabled": false, + "sandbox_api_server_enabled": true + }}) + .to_string(), + ), + _ => ("404 Not Found", json!({"error": "not found"}).to_string()), + }; + let response = format!( + "HTTP/1.1 {status}\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + let _ = stream.write_all(response.as_bytes()).await; + let _ = stream.shutdown().await; + } + }); + (base_url, handle) + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn principal_bound_feedback_sessions_preserve_public_and_none_repo_access() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let (base_url, server) = spawn_principal_capability_stub().await; + + for (principal_id, expected_access) in [ + ("prn_public", SessionRepoCacheAccess::Public), + ("prn_none", SessionRepoCacheAccess::None), + ] { + let thread_key = ThreadKey::parse(format!( + "feedback-improvement:test:{principal_id}:{}", + uuid::Uuid::new_v4() + )) + .unwrap(); + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = + runtime_with(&store, backend.clone()).with_iron_control(SessionRegistrar::new( + IronControlClient::new(base_url.clone(), "test-key"), + "default", + vec!["role_default_must_not_be_assigned".to_owned()], + )); + let session = runtime + .create_or_get_session_for_principal( + &thread_key, + &HarnessType::Codex, + None, + Some(json!({"source": "feedback-test"})), + HarnessConflictPolicy::Reject, + principal_id, + ) + .await + .expect("create principal-bound feedback session") + .session; + assert_eq!( + session.iron_control_principal.as_deref(), + Some(principal_id) + ); + + let desired_capabilities = runtime + .resolve_sandbox_capabilities(session.iron_control_principal.as_deref()) + .await + .expect("resolve caller capabilities"); + assert_eq!(desired_capabilities.repo_cache, expected_access); + assert_ne!(desired_capabilities.repo_cache, SessionRepoCacheAccess::All); + + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create feedback execution") + .execution + .execution_id; + claim_test_execution(&store, &runtime, &execution_id).await; + runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: Some(principal_id), + desired_capabilities: &desired_capabilities, + execution_id: &execution_id, + }) + .await + .expect("create capability-scoped feedback sandbox"); + let created_specs = backend.created_specs(); + assert_eq!(created_specs.len(), 1); + let expected_backend_access = match expected_access { + SessionRepoCacheAccess::None => RepoCacheAccess::None, + SessionRepoCacheAccess::Public => RepoCacheAccess::Public, + SessionRepoCacheAccess::All => RepoCacheAccess::All, + }; + assert_eq!( + created_specs[0].capabilities.repo_cache, + expected_backend_access + ); + } + + server.abort(); + } + fn completed_output_lines(result_text: &str) -> Vec { vec![ json!({ @@ -6173,11 +8886,20 @@ mod adoption_tests { eprintln!("skipping: SESSION_RUNTIME_TEST_DATABASE_URL not set"); return None; }; - let store = PgSessionStore::connect(&url) - .await - .expect("connect test db"); - store.run_migrations().await.expect("run migrations"); - Some(store) + static MIGRATIONS: OnceCell<()> = OnceCell::const_new(); + MIGRATIONS + .get_or_init(|| async { + let store = PgSessionStore::connect(&url) + .await + .expect("connect test db"); + store.run_migrations().await.expect("run migrations"); + }) + .await; + Some( + PgSessionStore::connect(&url) + .await + .expect("connect test db after migrations"), + ) } async fn orphaned_execution( @@ -6210,6 +8932,39 @@ mod adoption_tests { execution_id } + /// Ages an execution row past `PRE_SANDBOX_ORPHAN_GRACE` so adoption treats it + /// as a genuine orphan instead of a young row racing a live execute. + async fn backdate_execution(store: &PgSessionStore, execution_id: &str, seconds: f64) { + let result = sqlx::query( + "update session_executions \ + set created_at = created_at - make_interval(secs => $2), \ + started_at = started_at - make_interval(secs => $2) \ + where execution_id = $1", + ) + .bind(execution_id) + .bind(seconds) + .execute(store.pool()) + .await + .expect("backdate execution"); + assert_eq!(result.rows_affected(), 1, "expected to backdate one row"); + } + + /// Expires an execution's stdout-owner lease in place, simulating an + /// owner that died without releasing, deterministically (no sleeps + /// racing real lease TTLs). + async fn expire_stdout_lease(store: &PgSessionStore, execution_id: &str) { + let result = sqlx::query( + "update session_executions \ + set stdout_owner_lease_expires_at = now() - interval '1 second' \ + where execution_id = $1", + ) + .bind(execution_id) + .execute(store.pool()) + .await + .expect("expire stdout lease"); + assert_eq!(result.rows_affected(), 1, "expected to expire one lease"); + } + async fn wait_for_event(store: &PgSessionStore, thread_key: &ThreadKey, event_type: &str) { let deadline = Instant::now() + Duration::from_secs(10); loop { @@ -6228,6 +8983,22 @@ mod adoption_tests { } } + async fn wait_for_session_title( + store: &PgSessionStore, + thread_key: &ThreadKey, + expected: &str, + ) { + let deadline = Instant::now() + Duration::from_secs(10); + loop { + let session = store.get_session(thread_key).await.expect("get session"); + if session.title.as_deref() == Some(expected) { + return; + } + assert!(Instant::now() < deadline, "timed out waiting for title"); + sleep(Duration::from_millis(25)).await; + } + } + async fn events(store: &PgSessionStore, thread_key: &ThreadKey) -> Vec { store .list_events_after(thread_key, 0, None, 1000) @@ -6242,22 +9013,221 @@ mod adoption_tests { ) } - fn env_value<'a>(spec: &'a SandboxSpec, name: &str) -> Option<&'a str> { - spec.env - .iter() - .find(|env| env.name == name) - .map(|env| env.value.as_str()) - } - - fn default_capabilities() -> SessionSandboxCapabilities { - SessionSandboxCapabilities::default_enabled() + fn runtime_with_content_revision( + store: &PgSessionStore, + backend: Arc, + revision: &str, + ) -> SessionRuntime { + let mut runtime = runtime_with(store, backend); + runtime.sandbox_runtime.content_revision = Some(revision.to_owned()); + runtime } - fn restricted_capabilities() -> SessionSandboxCapabilities { - SessionSandboxCapabilities { - repo_cache_enabled: false, - observability_enabled: false, - } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn execution_scoped_event_stream_completes_after_terminal_event() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:stream-close-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, None, false).await; + store + .append_event( + &thread_key, + Some(&execution_id), + "session.output.line", + json!({ "line": "working" }), + ) + .await + .expect("append output event"); + store + .append_event( + &thread_key, + Some(&execution_id), + "session.execution_completed", + json!({ "execution_id": execution_id }), + ) + .await + .expect("append terminal event"); + + // Execution-scoped: the stream must end on its own after emitting the + // terminal event, releasing the response and its listener connection. + let listener = store.listen_session_events().await.expect("listener"); + let scoped = session_event_stream( + store.clone(), + thread_key.clone(), + 0, + Some(execution_id.clone()), + listener, + tracing::Span::none(), + ); + let emitted = tokio::time::timeout(Duration::from_secs(10), scoped.collect::>()) + .await + .expect("execution-scoped stream should complete after the terminal event"); + let kinds: Vec<_> = emitted + .into_iter() + .map(|result| result.expect("stream event").event_type) + .collect(); + assert_eq!( + kinds, + vec!["session.output.line", "session.execution_completed"] + ); + + // Control: an unscoped stream over the same events stays open for + // future events instead of completing. + let listener = store.listen_session_events().await.expect("listener"); + let unscoped = session_event_stream( + store.clone(), + thread_key.clone(), + 0, + None, + listener, + tracing::Span::none(), + ); + let mut unscoped = std::pin::pin!(unscoped); + for _ in 0..2 { + unscoped + .next() + .await + .expect("buffered event") + .expect("stream event"); + } + assert!( + tokio::time::timeout(Duration::from_millis(300), unscoped.next()) + .await + .is_err(), + "unscoped stream should stay open after a terminal event" + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn append_messages_generates_missing_session_title_once() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = ThreadKey::parse(format!("test:title-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + + let calls = Arc::new(AtomicUsize::new(0)); + let sources = Arc::new(Mutex::new(Vec::::new())); + let generator_started = Arc::new(tokio::sync::Notify::new()); + let generator_release = Arc::new(tokio::sync::Notify::new()); + let calls_for_generator = calls.clone(); + let sources_for_generator = sources.clone(); + let started_for_generator = generator_started.clone(); + let release_for_generator = generator_release.clone(); + let runtime = runtime_with( + &store, + Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())), + ) + .with_session_title_generator(move |source| { + let calls = calls_for_generator.clone(); + let sources = sources_for_generator.clone(); + let started = started_for_generator.clone(); + let release = release_for_generator.clone(); + async move { + calls.fetch_add(1, Ordering::SeqCst); + sources.lock().await.push(source); + started.notify_one(); + release.notified().await; + Ok("Fix worker memory leak".to_owned()) + } + }); + + tokio::time::timeout( + Duration::from_secs(1), + runtime.append_messages( + &thread_key, + &[SessionMessageInput { + client_message_id: Some("first".to_owned()), + role: MessageRole::User, + parts: vec![ + json!({ + "type": "text", + "text": "# Requester Context\n\nThe Slack user who prompted this turn is Alice." + }), + json!({ + "type": "text", + "text": "<@U123> please fix the memory leak in the worker" + }), + ], + metadata: json!({}), + }], + ), + ) + .await + .expect("append first message should not wait for title generation") + .expect("append first message"); + + generator_started.notified().await; + + let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!(session.title, None); + assert_eq!(calls.load(Ordering::SeqCst), 1); + assert_eq!( + sources.lock().await.clone(), + vec!["please fix the memory leak in the worker".to_owned()] + ); + + runtime + .append_messages( + &thread_key, + &[SessionMessageInput { + client_message_id: Some("burst".to_owned()), + role: MessageRole::User, + parts: vec![json!({"type": "text", "text": "add more logging"})], + metadata: json!({}), + }], + ) + .await + .expect("append burst message"); + + assert_eq!(calls.load(Ordering::SeqCst), 1); + + generator_release.notify_one(); + wait_for_session_title(&store, &thread_key, "Fix worker memory leak").await; + assert_eq!(calls.load(Ordering::SeqCst), 1); + + runtime + .append_messages( + &thread_key, + &[SessionMessageInput { + client_message_id: Some("second".to_owned()), + role: MessageRole::User, + parts: vec![json!({"type": "text", "text": "add more logging"})], + metadata: json!({}), + }], + ) + .await + .expect("append second message"); + + let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!(session.title.as_deref(), Some("Fix worker memory leak")); + assert_eq!(calls.load(Ordering::SeqCst), 1); + } + + fn env_value<'a>(spec: &'a SandboxSpec, name: &str) -> Option<&'a str> { + spec.env + .iter() + .find(|env| env.name == name) + .map(|env| env.value.as_str()) + } + + fn default_capabilities() -> SessionSandboxCapabilities { + SessionSandboxCapabilities::default_enabled() + } + + fn restricted_capabilities() -> SessionSandboxCapabilities { + SessionSandboxCapabilities { + repo_cache: SessionRepoCacheAccess::None, + observability_enabled: false, + api_server_enabled: false, + } } fn runtime_with_warm_pool( @@ -6294,12 +9264,178 @@ mod adoption_tests { target_size: 1, replenish_interval: Duration::from_secs(60), bootstrap_iron_control_principal: None, + max_running_sandboxes: None, }, )); runtime.warm_pool = Some(warm_pool); runtime } + async fn claim_test_execution( + store: &PgSessionStore, + runtime: &SessionRuntime, + execution_id: &str, + ) { + store + .mark_execution_running(execution_id) + .await + .expect("mark test execution running"); + claim_test_stdout_owner(store, runtime, execution_id).await; + } + + async fn claim_test_stdout_owner( + store: &PgSessionStore, + runtime: &SessionRuntime, + execution_id: &str, + ) { + assert!( + store + .claim_stdout_owner( + execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60), + ) + .await + .expect("claim test stdout owner") + ); + } + + async fn cancel_execution_before_assignment( + store: &PgSessionStore, + thread_key: &ThreadKey, + ) -> String { + store + .create_or_get_session(thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark execution running"); + assert!(matches!( + store + .release_session_if_sandbox_matches( + thread_key, + None, + true, + "remote release before sandbox assignment", + ) + .await + .expect("release before assignment"), + ReleaseSessionResult::Released { .. } + )); + execution_id + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn cold_sandbox_is_stopped_when_release_wins_assignment_cas() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:cold-release-race-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = cancel_execution_before_assignment(&store, &thread_key).await; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + + let result = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &default_capabilities(), + execution_id: &execution_id, + }) + .await; + + assert!(matches!(result, Err(SessionRuntimeError::BadRequest(_)))); + assert_eq!(backend.stopped(), vec!["mock-sbx".to_owned()]); + assert_eq!( + backend.opens(), + 0, + "rejected sandbox must never receive input" + ); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("released session") + .sandbox_id, + None + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn claimed_warm_sandbox_is_retired_when_release_wins_assignment_cas() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:warm-release-race-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = cancel_execution_before_assignment(&store, &thread_key).await; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with_warm_pool(&store, backend.clone(), thread_key.as_str()); + let workload_key = runtime + .warm_pool + .as_ref() + .expect("warm pool") + .workload_key() + .to_owned(); + let warm_sandbox_id = format!("warm-release-race-{}", uuid::Uuid::new_v4()); + store + .insert_ready_warm_sandbox(&warm_sandbox_id, &workload_key) + .await + .expect("insert ready warm sandbox"); + + let result = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &default_capabilities(), + execution_id: &execution_id, + }) + .await; + + assert!(matches!(result, Err(SessionRuntimeError::BadRequest(_)))); + assert_eq!(backend.stopped(), vec![warm_sandbox_id.clone()]); + assert_eq!( + backend.opens(), + 0, + "rejected warm sandbox must never receive input" + ); + let warm_status = sqlx::query_scalar::<_, String>( + "select status from session_warm_sandboxes where sandbox_id = $1", + ) + .bind(&warm_sandbox_id) + .fetch_one(store.pool()) + .await + .expect("warm sandbox status"); + assert_eq!(warm_status, "failed"); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("released session") + .sandbox_id, + None + ); + } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] async fn capability_mismatch_replaces_existing_sandbox() { let Some(store) = test_store().await else { @@ -6313,7 +9449,12 @@ mod adoption_tests { .await .expect("create session"); store - .update_sandbox_assignment(&thread_key, "sbx-full", &default_capabilities()) + .update_sandbox_assignment( + &thread_key, + "sbx-full", + Some("content-revision"), + &default_capabilities(), + ) .await .expect("assign default sandbox"); let session = store.get_session(&thread_key).await.unwrap(); @@ -6326,6 +9467,7 @@ mod adoption_tests { let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); let runtime = runtime_with_warm_pool(&store, backend.clone(), thread_key.as_str()); + claim_test_execution(&store, &runtime, &execution_id).await; let sandbox_id = runtime .ensure_session_sandbox(EnsureSessionSandboxRequest { thread_key: &thread_key, @@ -6349,12 +9491,17 @@ mod adoption_tests { Some(restricted_capabilities()) ); let spec = backend.created_specs().pop().expect("created cold spec"); - assert!(!spec.capabilities.repo_cache_enabled); + assert!(!spec.capabilities.repo_cache.enabled()); assert!(!spec.capabilities.observability_enabled); + assert!(!spec.capabilities.api_server_enabled); assert_eq!( env_value(&spec, "CENTAUR_SANDBOX_OBSERVABILITY_ENABLED"), Some("false") ); + assert_eq!( + env_value(&spec, "CENTAUR_SANDBOX_API_SERVER_ENABLED"), + Some("false") + ); let blocklist = env_value(&spec, "TOOL_BLOCKLIST").unwrap_or(""); for tool in OBSERVABILITY_TOOL_BLOCKLIST.split(',') { assert!(blocklist.split(',').any(|blocked| blocked == tool)); @@ -6373,288 +9520,324 @@ mod adoption_tests { } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn non_default_capabilities_skip_warm_pool() { + async fn legacy_assigned_sandbox_is_replaced_on_first_turn_after_content_rollout() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; - let thread_key = - ThreadKey::parse(format!("test:cap-warm-skip-{}", uuid::Uuid::new_v4())).unwrap(); + let thread_key = ThreadKey::parse(format!("slack:T1:C1:{}", uuid::Uuid::new_v4())).unwrap(); store .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); + store + .update_sandbox_id(&thread_key, Some("sbx-stale-content")) + .await + .expect("assign legacy sandbox"); + let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!(session.sandbox_content_revision, None); let execution_id = store .create_execution(&thread_key, None, json!({})) .await .expect("create execution") .execution .execution_id; - - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with_warm_pool(&store, backend.clone(), thread_key.as_str()); - let workload_key = runtime - .warm_pool - .as_ref() - .unwrap() - .workload_key() - .to_owned(); - let warm_sandbox_id = format!("warm-sbx-{}", uuid::Uuid::new_v4()); - store - .insert_ready_warm_sandbox(&warm_sandbox_id, &workload_key) - .await - .expect("insert warm sandbox"); + let backend = Arc::new(MockBackend::new(SandboxStatus::Suspended, Vec::new())); + let runtime = runtime_with_content_revision(&store, backend.clone(), "content-current"); + claim_test_execution(&store, &runtime, &execution_id).await; let sandbox_id = runtime .ensure_session_sandbox(EnsureSessionSandboxRequest { thread_key: &thread_key, harness_type: &HarnessType::Codex, persona_id: None, - existing_sandbox_id: None, - existing_sandbox_capabilities: None, + existing_sandbox_id: session.sandbox_id.as_deref(), + existing_sandbox_capabilities: session.sandbox_capabilities.as_ref(), iron_control_principal: None, - desired_capabilities: &restricted_capabilities(), + desired_capabilities: &default_capabilities(), execution_id: &execution_id, }) .await - .expect("ensure sandbox"); + .expect("replace stale assigned sandbox"); assert_eq!(sandbox_id, "mock-sbx"); - assert_eq!( - store - .claim_ready_warm_sandbox(&workload_key, thread_key.as_str()) - .await - .expect("warm row should remain ready"), - Some(warm_sandbox_id) - ); + assert_eq!(backend.stopped(), vec!["sbx-stale-content".to_owned()]); let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!(session.sandbox_id.as_deref(), Some("mock-sbx")); assert_eq!( - session.sandbox_capabilities, - Some(restricted_capabilities()) + session.sandbox_content_revision.as_deref(), + Some(sandbox_assignment_content_revision("content-current", "mock-sbx").as_str()) ); - let spec = backend.created_specs().pop().expect("created cold spec"); - assert!(!spec.capabilities.repo_cache_enabled); - assert!(!spec.capabilities.observability_enabled); + assert!(events(&store, &thread_key).await.iter().any(|event| { + event.event_type == "session.sandbox_content_replaced" + && event.payload["previous_content_revision"].is_null() + && event.payload["desired_content_revision"] + == json!(sandbox_assignment_content_revision( + "content-current", + "sbx-stale-content" + )) + })); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn existing_running_sandbox_ensures_proxy_before_reuse() { + async fn matching_content_revision_reuses_existing_sandbox() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:proxy-reuse-{}", uuid::Uuid::new_v4())).unwrap(); + ThreadKey::parse(format!("test:content-reuse-{}", uuid::Uuid::new_v4())).unwrap(); store .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); - let execution_id = store + store + .update_sandbox_assignment( + &thread_key, + "sbx-current-content", + Some( + sandbox_assignment_content_revision("content-current", "sbx-current-content") + .as_str(), + ), + &default_capabilities(), + ) + .await + .expect("assign current sandbox"); + let session = store.get_session(&thread_key).await.unwrap(); + let execution_id = store .create_execution(&thread_key, None, json!({})) .await .expect("create execution") .execution .execution_id; - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); + let runtime = runtime_with_content_revision(&store, backend.clone(), "content-current"); + claim_test_execution(&store, &runtime, &execution_id).await; + let sandbox_id = runtime .ensure_session_sandbox(EnsureSessionSandboxRequest { thread_key: &thread_key, harness_type: &HarnessType::Codex, persona_id: None, - existing_sandbox_id: Some("sbx-existing"), - existing_sandbox_capabilities: None, - iron_control_principal: Some("principal-existing"), - desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + existing_sandbox_id: session.sandbox_id.as_deref(), + existing_sandbox_capabilities: session.sandbox_capabilities.as_ref(), + iron_control_principal: None, + desired_capabilities: &default_capabilities(), execution_id: &execution_id, }) .await - .expect("reuse existing sandbox"); + .expect("reuse current sandbox"); - assert_eq!(sandbox_id, "sbx-existing"); - assert_eq!( - backend.proxy_ensures(), - vec![("sbx-existing".to_owned(), "principal-existing".to_owned())] - ); + assert_eq!(sandbox_id, "sbx-current-content"); + assert!(backend.stopped().is_empty()); + assert!(backend.created_specs().is_empty()); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn workflow_cleanup_stops_and_clears_owned_sandbox() { + async fn rollback_era_clear_and_reassignment_cannot_spoof_forward_content_stamp() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; - let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); - let thread_key = - ThreadKey::parse(format!("test:wf-cleanup-{}", uuid::Uuid::new_v4())).unwrap(); + let thread_key = ThreadKey::parse(format!( + "test:rollback-reassignment-{}", + uuid::Uuid::new_v4() + )) + .unwrap(); store - .create_or_get_session( - &thread_key, - &HarnessType::Codex, - None, - json!({ - "source": "absurd_workflow", - "workflow_run_id": workflow_run_id, - "workflow_owned_thread": true, - }), - ) + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); + let forward_stamp = sandbox_assignment_content_revision("content-current", "sbx-forward"); store - .update_sandbox_id(&thread_key, Some("sbx-owned")) + .update_sandbox_assignment( + &thread_key, + "sbx-forward", + Some(&forward_stamp), + &default_capabilities(), + ) .await - .expect("set sandbox id"); - store - .insert_ready_warm_sandbox("sbx-owned", "test-workload") + .expect("assign forward sandbox"); + + // Simulate the pre-0043 binary: it clears and later assigns sandbox_id + // without knowing about sandbox_content_revision. The schema must not + // reject those rollback writes, and the old ID-bound stamp must not + // authenticate the different rollback-era sandbox on re-forward. + sqlx::query("update sessions set sandbox_id = null where thread_key = $1") + .bind(thread_key.as_str()) + .execute(store.pool()) .await - .expect("insert warm sandbox"); + .expect("old binary clear remains backward compatible"); + sqlx::query("update sessions set sandbox_id = 'sbx-rollback' where thread_key = $1") + .bind(thread_key.as_str()) + .execute(store.pool()) + .await + .expect("old binary assignment remains backward compatible"); + let session = store.get_session(&thread_key).await.unwrap(); assert_eq!( - store - .claim_ready_warm_sandbox("test-workload", thread_key.as_str()) - .await - .expect("claim warm sandbox"), - Some("sbx-owned".to_owned()) - ); - assert!( - store - .list_referenced_sandbox_ids() - .await - .expect("list referenced sandboxes") - .contains(&"sbx-owned".to_owned()) + session.sandbox_content_revision.as_deref(), + Some(forward_stamp.as_str()) ); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); - let report = runtime - .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + let runtime = runtime_with_content_revision(&store, backend.clone(), "content-current"); + claim_test_execution(&store, &runtime, &execution_id).await; + let sandbox_id = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: session.sandbox_id.as_deref(), + existing_sandbox_capabilities: session.sandbox_capabilities.as_ref(), + iron_control_principal: None, + desired_capabilities: &default_capabilities(), + execution_id: &execution_id, + }) .await - .expect("cleanup workflow sandboxes"); + .expect("replace rollback-era sandbox on re-forward"); - assert_eq!(report.stopped, vec!["sbx-owned".to_owned()]); - assert_eq!(backend.stopped(), vec!["sbx-owned".to_owned()]); - assert_eq!( - store.get_session(&thread_key).await.unwrap().sandbox_id, - None - ); - assert!( - !store - .list_referenced_sandbox_ids() - .await - .expect("list referenced sandboxes") - .contains(&"sbx-owned".to_owned()) - ); - let all = events(&store, &thread_key).await; - assert!(all.iter().any(|event| { - event.event_type == "session.workflow_sandbox_stopped" - && event.payload["workflow_run_id"] == json!(workflow_run_id) - && event.payload["cleared"] == json!(true) - })); + assert_eq!(sandbox_id, "mock-sbx"); + assert_eq!(backend.stopped(), vec!["sbx-rollback".to_owned()]); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn workflow_cleanup_preserves_explicit_unowned_thread_key() { + async fn content_replacement_stop_failure_restores_old_assignment_and_revision() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; - let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); let thread_key = - ThreadKey::parse(format!("test:wf-explicit-{}", uuid::Uuid::new_v4())).unwrap(); + ThreadKey::parse(format!("test:content-restore-{}", uuid::Uuid::new_v4())).unwrap(); store - .create_or_get_session( - &thread_key, - &HarnessType::Codex, - None, - json!({ - "source": "absurd_workflow", - "workflow_run_id": workflow_run_id, - }), - ) + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); store - .update_sandbox_id(&thread_key, Some("sbx-explicit")) + .update_sandbox_assignment( + &thread_key, + "sbx-old-content", + Some( + sandbox_assignment_content_revision("content-old", "sbx-old-content").as_str(), + ), + &default_capabilities(), + ) .await - .expect("set sandbox id"); - - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); - let report = runtime - .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + .expect("assign old sandbox"); + let session = store.get_session(&thread_key).await.unwrap(); + let execution_id = store + .create_execution(&thread_key, None, json!({})) .await - .expect("cleanup workflow sandboxes"); + .expect("create execution") + .execution + .execution_id; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + backend.fail_stop(); + let runtime = runtime_with_content_revision(&store, backend.clone(), "content-current"); + claim_test_execution(&store, &runtime, &execution_id).await; - assert!(report.stopped.is_empty()); - assert!(backend.stopped().is_empty()); + let result = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: session.sandbox_id.as_deref(), + existing_sandbox_capabilities: session.sandbox_capabilities.as_ref(), + iron_control_principal: None, + desired_capabilities: &default_capabilities(), + execution_id: &execution_id, + }) + .await; + + assert!(matches!(result, Err(SessionRuntimeError::Sandbox(_)))); + let restored = store.get_session(&thread_key).await.unwrap(); + assert_eq!(restored.sandbox_id.as_deref(), Some("sbx-old-content")); assert_eq!( - store.get_session(&thread_key).await.unwrap().sandbox_id, - Some("sbx-explicit".to_owned()) + restored.sandbox_content_revision.as_deref(), + Some(sandbox_assignment_content_revision("content-old", "sbx-old-content").as_str()) ); + assert_eq!(restored.sandbox_capabilities, Some(default_capabilities())); + assert!(backend.created_specs().is_empty()); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn workflow_cleanup_clears_owned_sandbox_when_backend_reports_missing() { + async fn warm_claim_stamps_the_current_content_revision_atomically() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; - let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); let thread_key = - ThreadKey::parse(format!("test:wf-missing-{}", uuid::Uuid::new_v4())).unwrap(); + ThreadKey::parse(format!("test:warm-content-{}", uuid::Uuid::new_v4())).unwrap(); store - .create_or_get_session( - &thread_key, - &HarnessType::Codex, - None, - json!({ - "source": "absurd_workflow", - "workflow_run_id": workflow_run_id, - "workflow_owned_thread": true, - }), - ) + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let mut runtime = runtime_with_warm_pool(&store, backend, thread_key.as_str()); + runtime.sandbox_runtime.content_revision = Some("content-current".to_owned()); + let workload_key = runtime + .warm_pool + .as_ref() + .expect("warm pool") + .workload_key() + .to_owned(); store - .update_sandbox_id(&thread_key, Some("sbx-missing")) + .insert_ready_warm_sandbox("sbx-warm-current", &workload_key) .await - .expect("set sandbox id"); + .expect("insert warm sandbox"); + claim_test_execution(&store, &runtime, &execution_id).await; - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - backend.mark_stop_missing("sbx-missing"); - let runtime = runtime_with(&store, backend); - let report = runtime - .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + let sandbox_id = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &default_capabilities(), + execution_id: &execution_id, + }) .await - .expect("cleanup workflow sandboxes"); + .expect("claim warm sandbox"); - assert_eq!(report.missing, vec!["sbx-missing".to_owned()]); + assert_eq!(sandbox_id, "sbx-warm-current"); assert_eq!( - store.get_session(&thread_key).await.unwrap().sandbox_id, - None + store + .get_session(&thread_key) + .await + .unwrap() + .sandbox_content_revision + .as_deref(), + Some( + sandbox_assignment_content_revision("content-current", "sbx-warm-current").as_str() + ) ); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn resume_failure_replaces_sandbox_and_preserves_harness_thread_id() { + async fn non_default_capabilities_skip_warm_pool() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:resume-failed-{}", uuid::Uuid::new_v4())).unwrap(); + ThreadKey::parse(format!("test:cap-warm-skip-{}", uuid::Uuid::new_v4())).unwrap(); store .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) .await .expect("create session"); - store - .update_sandbox_id(&thread_key, Some("sbx-old")) - .await - .expect("set sandbox id"); - store - .update_harness_thread_id(&thread_key, Some("harness-thread-1")) - .await - .expect("set harness thread id"); let execution_id = store .create_execution(&thread_key, None, json!({})) .await @@ -6662,52 +9845,732 @@ mod adoption_tests { .execution .execution_id; - let backend = Arc::new(MockBackend::new(SandboxStatus::Suspended, Vec::new())); - backend.fail_resume(); - let runtime = runtime_with(&store, backend); + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with_warm_pool(&store, backend.clone(), thread_key.as_str()); + claim_test_execution(&store, &runtime, &execution_id).await; + let workload_key = runtime + .warm_pool + .as_ref() + .unwrap() + .workload_key() + .to_owned(); + let warm_sandbox_id = format!("warm-sbx-{}", uuid::Uuid::new_v4()); + store + .insert_ready_warm_sandbox(&warm_sandbox_id, &workload_key) + .await + .expect("insert warm sandbox"); + let sandbox_id = runtime .ensure_session_sandbox(EnsureSessionSandboxRequest { thread_key: &thread_key, harness_type: &HarnessType::Codex, persona_id: None, - existing_sandbox_id: Some("sbx-old"), + existing_sandbox_id: None, existing_sandbox_capabilities: None, iron_control_principal: None, - desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + desired_capabilities: &restricted_capabilities(), execution_id: &execution_id, }) .await - .expect("resume failure should fall through to replacement"); + .expect("ensure sandbox"); assert_eq!(sandbox_id, "mock-sbx"); - let session = store.get_session(&thread_key).await.unwrap(); - assert_eq!(session.sandbox_id, Some("mock-sbx".to_owned())); assert_eq!( - session.harness_thread_id, - Some("harness-thread-1".to_owned()) + store + .claim_ready_warm_sandbox(&workload_key, thread_key.as_str()) + .await + .expect("warm row should remain ready"), + Some(warm_sandbox_id) ); - let all = events(&store, &thread_key).await; - assert!( - all.iter() - .any(|event| event.event_type == "session.sandbox_resume_failed") + let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!( + session.sandbox_capabilities, + Some(restricted_capabilities()) ); + let spec = backend.created_specs().pop().expect("created cold spec"); + assert!(!spec.capabilities.repo_cache.enabled()); + assert!(!spec.capabilities.observability_enabled); + assert!(!spec.capabilities.api_server_enabled); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn concurrent_pipe_ensure_opens_one_io_per_sandbox() { + async fn existing_running_sandbox_ensures_proxy_before_reuse() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:pipe-race-{}", uuid::Uuid::new_v4())).unwrap(); - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let (first_io, _first_stdout, _first_stdin) = mock_io(); - let (second_io, _second_stdout, _second_stdin) = mock_io(); - backend.push_io(first_io).await; - backend.push_io(second_io).await; - - let runtime = runtime_with(&store, backend.clone()); + ThreadKey::parse(format!("test:proxy-reuse-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some("sbx-existing")) + .await + .expect("persist existing sandbox assignment"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + claim_test_execution(&store, &runtime, &execution_id).await; + let sandbox_id = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: Some("sbx-existing"), + existing_sandbox_capabilities: None, + iron_control_principal: Some("principal-existing"), + desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + execution_id: &execution_id, + }) + .await + .expect("reuse existing sandbox"); + + assert_eq!(sandbox_id, "sbx-existing"); + assert_eq!( + backend.proxy_ensures(), + vec![("sbx-existing".to_owned(), "principal-existing".to_owned())] + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn drain_waits_for_cold_allocation_then_fences_future_allocations() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:drain-cold-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + backend.block_next_create(); + let runtime = runtime_with(&store, backend.clone()); + claim_test_execution(&store, &runtime, &execution_id).await; + + let allocator = { + let runtime = runtime.clone(); + let thread_key = thread_key.clone(); + let execution_id = execution_id.clone(); + tokio::spawn(async move { + runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + execution_id: &execution_id, + }) + .await + }) + }; + backend.wait_for_create_start().await; + let mut drain = { + let runtime = runtime.clone(); + tokio::spawn(async move { runtime.drain().await }) + }; + assert!( + timeout(Duration::from_millis(50), &mut drain) + .await + .is_err(), + "drain must wait for the in-flight allocation read guard" + ); + backend.continue_create(); + assert_eq!( + allocator + .await + .expect("allocation task") + .expect("allocation"), + "mock-sbx" + ); + let report = drain.await.expect("drain task").expect("drain"); + assert_eq!(report.stopped, vec!["mock-sbx".to_owned()]); + assert!(report.failed.is_empty()); + assert_eq!(backend.status_of("mock-sbx"), Some(SandboxStatus::Stopped)); + + let future_thread = + ThreadKey::parse(format!("test:drain-cold-future-{}", uuid::Uuid::new_v4())).unwrap(); + let error = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &future_thread, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + execution_id: "future-execution", + }) + .await + .expect_err("post-drain allocations must stay fenced"); + assert!(matches!(error, SessionRuntimeError::ShuttingDown)); + assert_eq!(backend.created_specs().len(), 1); + store + .fail_execution_if_active(&execution_id, "test cleanup") + .await + .expect("terminalize execution"); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn drain_waits_for_auxiliary_allocation_and_inventories_its_sandbox() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let (io, _stdout, _stdin) = mock_io(); + backend.push_io(io).await; + backend.block_next_create(); + let runtime = runtime_with(&store, backend.clone()); + + let allocator = { + let runtime = runtime.clone(); + tokio::spawn(async move { + let permit = runtime.acquire_sandbox_allocation_permit().await?; + let result = runtime + .sandbox_runtime_handle() + .create_running_io(SandboxSpec::new("workflow-host")) + .await; + drop(permit); + result + }) + }; + backend.wait_for_create_start().await; + let mut drain = { + let runtime = runtime.clone(); + tokio::spawn(async move { runtime.drain().await }) + }; + assert!( + timeout(Duration::from_millis(50), &mut drain) + .await + .is_err(), + "drain must wait for an auxiliary allocator holding the shared permit" + ); + backend.continue_create(); + let (sandbox_id, _io) = allocator + .await + .expect("auxiliary allocation task") + .expect("auxiliary allocation"); + assert_eq!(sandbox_id.as_str(), "mock-sbx"); + + let report = drain.await.expect("drain task").expect("drain"); + assert_eq!(report.stopped, vec!["mock-sbx".to_owned()]); + assert!(report.failed.is_empty()); + assert_eq!(backend.status_of("mock-sbx"), Some(SandboxStatus::Stopped)); + assert!(matches!( + runtime.acquire_sandbox_allocation_permit().await, + Err(SessionRuntimeError::ShuttingDown) + )); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn drain_waits_for_warm_claim_then_stops_claimed_sandbox() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:drain-warm-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + let warm_sandbox_id = format!("warm-drain-{}", uuid::Uuid::new_v4()); + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + backend.set_observed_status(&warm_sandbox_id, SandboxStatus::Running); + let runtime = runtime_with_warm_pool(&store, backend.clone(), thread_key.as_str()); + let workload_key = runtime + .warm_pool + .as_ref() + .expect("warm pool") + .workload_key() + .to_owned(); + store + .insert_ready_warm_sandbox(&warm_sandbox_id, &workload_key) + .await + .expect("insert warm sandbox"); + backend.block_next_status(); + claim_test_execution(&store, &runtime, &execution_id).await; + + let allocator = { + let runtime = runtime.clone(); + let thread_key = thread_key.clone(); + let execution_id = execution_id.clone(); + tokio::spawn(async move { + runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: None, + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + execution_id: &execution_id, + }) + .await + }) + }; + backend.wait_for_status_start().await; + let mut drain = { + let runtime = runtime.clone(); + tokio::spawn(async move { runtime.drain().await }) + }; + assert!( + timeout(Duration::from_millis(50), &mut drain) + .await + .is_err(), + "drain must wait for a warm claim already inside the allocation gate" + ); + backend.continue_status(); + assert_eq!( + allocator.await.expect("claim task").expect("warm claim"), + warm_sandbox_id + ); + let report = drain.await.expect("drain task").expect("drain"); + assert_eq!(report.stopped, vec![warm_sandbox_id.clone()]); + assert!(report.failed.is_empty()); + assert_eq!( + backend.status_of(&warm_sandbox_id), + Some(SandboxStatus::Stopped) + ); + assert!(backend.created_specs().is_empty()); + store + .fail_execution_if_active(&execution_id, "test cleanup") + .await + .expect("terminalize execution"); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn capacity_pressure_pauses_oldest_idle_assigned_sandbox() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + // Capacity admission intentionally scans database-wide. The exact CI + // command runs the warm-pool crate first against this same database, + // so remove its test fixtures before constructing this test's ordered + // candidate set. + sqlx::query("delete from session_warm_sandboxes") + .execute(store.pool()) + .await + .expect("clear prior warm-pool test rows"); + sqlx::query( + "update sessions set sandbox_id = null, sandbox_content_revision = null, \ + sandbox_repo_cache_enabled = null, sandbox_repo_cache_access = null, \ + sandbox_observability_enabled = null, sandbox_api_server_enabled = null, \ + sandbox_last_active_at = null \ + where sandbox_id is not null", + ) + .execute(store.pool()) + .await + .expect("clear prior sandbox assignment test rows"); + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + backend.set_observed_status( + "sbx-old", + SandboxStatus::Unknown("status temporarily unavailable".to_owned()), + ); + backend.set_observed_status("sbx-hot", SandboxStatus::Running); + backend.set_observed_status("sbx-stale", SandboxStatus::Gone); + backend.set_observed_status("sbx-paused", SandboxStatus::Suspended); + + let stale_thread = + ThreadKey::parse(format!("test:capacity-stale-{}", uuid::Uuid::new_v4())).unwrap(); + let paused_thread = + ThreadKey::parse(format!("test:capacity-paused-{}", uuid::Uuid::new_v4())).unwrap(); + let old_thread = + ThreadKey::parse(format!("test:capacity-old-{}", uuid::Uuid::new_v4())).unwrap(); + let hot_thread = + ThreadKey::parse(format!("test:capacity-hot-{}", uuid::Uuid::new_v4())).unwrap(); + let trigger_thread = + ThreadKey::parse(format!("test:capacity-trigger-{}", uuid::Uuid::new_v4())).unwrap(); + + store + .create_or_get_session(&stale_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create stale session"); + store + .update_sandbox_id(&stale_thread, Some("sbx-stale")) + .await + .expect("assign stale sandbox"); + store + .create_or_get_session(&paused_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create paused session"); + store + .update_sandbox_id(&paused_thread, Some("sbx-paused")) + .await + .expect("assign paused sandbox"); + store + .append_event( + &paused_thread, + None, + "session.sandbox_paused", + json!({ + "thread_key": paused_thread.as_str(), + "sandbox_id": "sbx-paused", + "reason": "capacity_pressure", + }), + ) + .await + .expect("append paused event"); + store + .create_or_get_session(&old_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create old session"); + store + .update_sandbox_id(&old_thread, Some("sbx-old")) + .await + .expect("assign old sandbox"); + store + .create_or_get_session(&hot_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create hot session"); + store + .update_sandbox_id(&hot_thread, Some("sbx-hot")) + .await + .expect("assign hot sandbox"); + sqlx::query( + r#" + update sessions + set sandbox_last_active_at = case + when thread_key = $1 then now() - interval '3 hours' + when thread_key = $2 then now() - interval '2 hours' + when thread_key = $3 then now() - interval '1 hour' + end + where thread_key in ($1, $2, $3) + "#, + ) + .bind(stale_thread.as_str()) + .bind(paused_thread.as_str()) + .bind(old_thread.as_str()) + .execute(store.pool()) + .await + .expect("age capacity candidates"); + + let controller = SandboxCapacityController::new( + store.clone(), + Arc::new(SandboxManager::new(backend.clone())), + Arc::new(DashMap::new()), + SandboxCapacityConfig { + max_running: 2, + hot_idle_grace: Duration::from_secs(300), + }, + ); + + controller + .run_with_capacity(&trigger_thread, "exe-trigger", "cold_create", || async { + Ok(()) + }) + .await + .expect("admit under capacity"); + + assert_eq!(backend.status_of("sbx-old"), Some(SandboxStatus::Suspended)); + assert_eq!(backend.status_of("sbx-hot"), Some(SandboxStatus::Running)); + assert_eq!( + store + .get_session(&stale_thread) + .await + .expect("get stale session") + .sandbox_id, + None + ); + assert_eq!( + store + .get_session(&paused_thread) + .await + .expect("get paused session") + .sandbox_id + .as_deref(), + Some("sbx-paused") + ); + let old_events = store + .list_events_after(&old_thread, 0, None, 100) + .await + .expect("list old events"); + assert!(old_events.iter().any(|event| { + event.event_type == "session.sandbox_paused" + && event.payload.get("reason").and_then(Value::as_str) == Some("capacity_pressure") + })); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn workflow_cleanup_stops_and_clears_owned_sandbox() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); + let thread_key = + ThreadKey::parse(format!("test:wf-cleanup-{}", uuid::Uuid::new_v4())).unwrap(); + let sandbox_id = format!("sbx-owned-{}", uuid::Uuid::new_v4()); + store + .create_or_get_session( + &thread_key, + &HarnessType::Codex, + None, + json!({ + "source": "absurd_workflow", + "workflow_run_id": workflow_run_id, + "workflow_owned_thread": true, + }), + ) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some(&sandbox_id)) + .await + .expect("set sandbox id"); + store + .insert_ready_warm_sandbox(&sandbox_id, "test-workload") + .await + .expect("insert warm sandbox"); + assert_eq!( + store + .claim_ready_warm_sandbox("test-workload", thread_key.as_str()) + .await + .expect("claim warm sandbox"), + Some(sandbox_id.clone()) + ); + assert!( + store + .list_referenced_sandbox_ids() + .await + .expect("list referenced sandboxes") + .contains(&sandbox_id) + ); + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create workflow-owned execution") + .execution + .execution_id; + claim_test_execution(&store, &runtime, &execution_id).await; + let report = runtime + .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + .await + .expect("cleanup workflow sandboxes"); + + assert_eq!(report.stopped, vec![sandbox_id.clone()]); + assert_eq!(backend.stopped(), vec![sandbox_id.clone()]); + let execution = store + .latest_execution_for_thread(&thread_key) + .await + .expect("read workflow-owned execution") + .expect("workflow-owned execution"); + assert_eq!(execution.status, ExecutionStatus::Cancelled); + assert_eq!( + store + .count_executions_with_stdout_owner(&runtime.stdout_owner_id) + .await + .expect("count stdout owners"), + 0, + "workflow cleanup must release stdout ownership" + ); + assert_eq!( + store.get_session(&thread_key).await.unwrap().sandbox_id, + None + ); + assert!( + !store + .list_referenced_sandbox_ids() + .await + .expect("list referenced sandboxes") + .contains(&sandbox_id) + ); + let all = events(&store, &thread_key).await; + assert!(all.iter().any(|event| { + event.event_type == "session.workflow_sandbox_stopped" + && event.payload["workflow_run_id"] == json!(workflow_run_id) + && event.payload["cleared"] == json!(true) + })); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn workflow_cleanup_preserves_explicit_unowned_thread_key() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); + let thread_key = + ThreadKey::parse(format!("test:wf-explicit-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session( + &thread_key, + &HarnessType::Codex, + None, + json!({ + "source": "absurd_workflow", + "workflow_run_id": workflow_run_id, + }), + ) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some("sbx-explicit")) + .await + .expect("set sandbox id"); + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + let report = runtime + .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + .await + .expect("cleanup workflow sandboxes"); + + assert!(report.stopped.is_empty()); + assert!(backend.stopped().is_empty()); + assert_eq!( + store.get_session(&thread_key).await.unwrap().sandbox_id, + Some("sbx-explicit".to_owned()) + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn workflow_cleanup_clears_owned_sandbox_when_backend_reports_missing() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let workflow_run_id = format!("run-{}", uuid::Uuid::new_v4()); + let thread_key = + ThreadKey::parse(format!("test:wf-missing-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session( + &thread_key, + &HarnessType::Codex, + None, + json!({ + "source": "absurd_workflow", + "workflow_run_id": workflow_run_id, + "workflow_owned_thread": true, + }), + ) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some("sbx-missing")) + .await + .expect("set sandbox id"); + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + backend.mark_stop_missing("sbx-missing"); + let runtime = runtime_with(&store, backend); + let report = runtime + .stop_workflow_owned_sandboxes(&workflow_run_id, "test") + .await + .expect("cleanup workflow sandboxes"); + + assert_eq!(report.missing, vec!["sbx-missing".to_owned()]); + assert_eq!( + store.get_session(&thread_key).await.unwrap().sandbox_id, + None + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn resume_failure_replaces_sandbox_and_preserves_harness_thread_id() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:resume-failed-{}", uuid::Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some("sbx-old")) + .await + .expect("set sandbox id"); + store + .update_harness_thread_id(&thread_key, Some("harness-thread-1")) + .await + .expect("set harness thread id"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Suspended, Vec::new())); + backend.fail_resume(); + let runtime = runtime_with(&store, backend); + claim_test_execution(&store, &runtime, &execution_id).await; + let sandbox_id = runtime + .ensure_session_sandbox(EnsureSessionSandboxRequest { + thread_key: &thread_key, + harness_type: &HarnessType::Codex, + persona_id: None, + existing_sandbox_id: Some("sbx-old"), + existing_sandbox_capabilities: None, + iron_control_principal: None, + desired_capabilities: &SessionSandboxCapabilities::default_enabled(), + execution_id: &execution_id, + }) + .await + .expect("resume failure should fall through to replacement"); + + assert_eq!(sandbox_id, "mock-sbx"); + let session = store.get_session(&thread_key).await.unwrap(); + assert_eq!(session.sandbox_id, Some("mock-sbx".to_owned())); + assert_eq!( + session.harness_thread_id, + Some("harness-thread-1".to_owned()) + ); + let all = events(&store, &thread_key).await; + assert!( + all.iter() + .any(|event| event.event_type == "session.sandbox_resume_failed") + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn concurrent_pipe_ensure_opens_one_io_per_sandbox() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:pipe-race-{}", uuid::Uuid::new_v4())).unwrap(); + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let (first_io, _first_stdout, _first_stdin) = mock_io(); + let (second_io, _second_stdout, _second_stdin) = mock_io(); + backend.push_io(first_io).await; + backend.push_io(second_io).await; + + let runtime = runtime_with(&store, backend.clone()); let (first, second) = tokio::join!( runtime.ensure_session_pipe(&thread_key, "sbx-pipe-race"), runtime.ensure_session_pipe(&thread_key, "sbx-pipe-race"), @@ -6726,13 +10589,15 @@ mod adoption_tests { let _serial = TEST_LOCK.lock().await; let thread_key = ThreadKey::parse(format!("test:eof-recorded-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-recorded"), true).await; + let execution_id = + orphaned_execution(&store, &thread_key, Some("sbx-recorded"), true).await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); let (io, stdout, _stdin) = mock_io(); backend.push_io(io).await; let runtime = runtime_with(&store, backend.clone()); + claim_test_stdout_owner(&store, &runtime, &execution_id).await; runtime .ensure_session_pipe(&thread_key, "sbx-recorded") .await @@ -6776,7 +10641,8 @@ mod adoption_tests { let _serial = TEST_LOCK.lock().await; let thread_key = ThreadKey::parse(format!("test:eof-reattach-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-reattach"), true).await; + let execution_id = + orphaned_execution(&store, &thread_key, Some("sbx-reattach"), true).await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); let (first_io, mut first_stdout, _first_stdin) = mock_io(); @@ -6785,6 +10651,7 @@ mod adoption_tests { backend.push_io(second_io).await; let runtime = runtime_with(&store, backend.clone()); + claim_test_stdout_owner(&store, &runtime, &execution_id).await; runtime .ensure_session_pipe(&thread_key, "sbx-reattach") .await @@ -6816,261 +10683,700 @@ mod adoption_tests { completed.payload["result_text"].as_str(), Some("Completed after reattach.") ); - assert_eq!(backend.opens(), 2); + assert_eq!(backend.opens(), 2); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn stdout_eof_fails_when_sandbox_no_longer_accepts_io() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:eof-gone-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-gone"), true).await; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let (io, stdout, _stdin) = mock_io(); + backend.push_io(io).await; + + let runtime = runtime_with(&store, backend.clone()); + claim_test_stdout_owner(&store, &runtime, &execution_id).await; + runtime + .ensure_session_pipe(&thread_key, "sbx-gone") + .await + .expect("open initial pipe"); + backend.set_status(SandboxStatus::Gone); + drop(stdout); + + wait_for_event(&store, &thread_key, "session.execution_failed").await; + let all = events(&store, &thread_key).await; + let failed = all + .iter() + .find(|event| event.event_type == "session.execution_failed") + .expect("failed event"); + let error = failed.payload["error"].as_str().unwrap_or_default(); + assert!( + error.contains("sandbox stdout closed before terminal output"), + "unexpected error: {error}" + ); + assert!( + error.contains("sandbox no longer accepts io"), + "expected sandbox status detail: {error}" + ); + assert!( + !all.iter() + .any(|event| event.event_type == "session.stdout_pump_reattached"), + "gone sandbox should not reattach" + ); + assert_eq!(backend.opens(), 1); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn adopts_finished_turn_from_recorded_sandbox_output() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-logs-{}", uuid::Uuid::new_v4())).unwrap(); + orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + + let backend = Arc::new(MockBackend::new( + SandboxStatus::Running, + vec![ + json!({"type": "item.completed", "item": {"id": "msg-1", "type": "agentMessage", "text": "Done: pushed commit abc123.", "phase": "final_answer"}}).to_string(), + json!({"type": "turn.completed", "turn": {"id": "turn-1", "status": "completed"}}).to_string(), + ], + )); + let runtime = runtime_with(&store, backend.clone()); + runtime.adopt_orphaned_executions().await; + + wait_for_event(&store, &thread_key, "session.execution_completed").await; + let all = events(&store, &thread_key).await; + assert!( + all.iter() + .any(|event| event.event_type == "session.execution_adopted"), + "expected an adoption event" + ); + let completed = all + .iter() + .find(|event| event.event_type == "session.execution_completed") + .expect("completed event"); + assert_eq!( + completed.payload["result_text"].as_str(), + Some("Done: pushed commit abc123.") + ); + // The terminal came from recorded output; no live attach was needed. + assert_eq!(backend.opens(), 0); + let session = store.get_session(&thread_key).await.unwrap(); + assert_ne!(session.status.as_ref(), "failed"); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn adopts_live_when_recorded_output_has_no_terminal() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-live-{}", uuid::Uuid::new_v4())).unwrap(); + orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let (io, mut stdout, _stdin) = mock_io(); + backend.push_io(io).await; + + let runtime = runtime_with(&store, backend.clone()); + runtime.adopt_orphaned_executions().await; + assert_eq!(backend.opens(), 1); + + stdout + .write_all( + b"{\"type\":\"turn.completed\",\"turn\":{\"id\":\"turn-1\",\"status\":\"completed\"}}\n", + ) + .await + .unwrap(); + wait_for_event(&store, &thread_key, "session.execution_completed").await; + let all = events(&store, &thread_key).await; + assert!( + all.iter().any(|event| { + event.event_type == "session.execution_adopted" + && event.payload["mode"] == json!("live_attach") + }), + "expected a live adoption event" + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn fails_orphans_whose_sandbox_is_gone() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-gone-{}", uuid::Uuid::new_v4())).unwrap(); + orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Gone, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + runtime.adopt_orphaned_executions().await; + + wait_for_event(&store, &thread_key, "session.execution_failed").await; + let all = events(&store, &thread_key).await; + let failed = all + .iter() + .find(|event| event.event_type == "session.execution_failed") + .expect("failed event"); + let error = failed.payload["error"].as_str().unwrap_or_default(); + assert!( + error.contains("execution orphaned by control plane restart"), + "unexpected error: {error}" + ); + assert!( + error.contains("sandbox no longer accepts io"), + "expected status detail: {error}" + ); + assert_eq!(backend.opens(), 0); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn fails_queued_orphans_that_never_received_input() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-queued-{}", uuid::Uuid::new_v4())).unwrap(); + orphaned_execution(&store, &thread_key, Some("sbx-mock"), false).await; + + // The one-shot scan has no later tick to revisit skipped rows, so it + // fails queued orphans immediately regardless of age. + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + runtime.adopt_orphaned_executions().await; + + wait_for_event(&store, &thread_key, "session.execution_failed").await; + let all = events(&store, &thread_key).await; + let failed = all + .iter() + .find(|event| event.event_type == "session.execution_failed") + .expect("failed event"); + let error = failed.payload["error"].as_str().unwrap_or_default(); + assert!( + error.contains("orphaned before input was sent"), + "unexpected error: {error}" + ); + assert_eq!(backend.opens(), 0); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn periodic_scan_skips_young_pre_sandbox_executions_until_grace_passes() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-young-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), false).await; + + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend.clone()); + let mut state = OrphanAdoptionState::default(); + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + + // A queued row younger than the grace window may belong to a live + // execute_session mid-transition; a periodic scan must leave it + // alone and revisit it later. + let all = events(&store, &thread_key).await; + assert!( + all.iter() + .all(|event| event.event_type != "session.execution_failed"), + "young queued execution must not be failed" + ); + let active = store + .list_active_executions() + .await + .expect("list active executions"); + assert!( + active + .iter() + .any(|execution| execution.execution_id == execution_id), + "young queued execution must stay active" + ); + + // Once the row ages past the grace window, a later tick fails it. + backdate_execution(&store, &execution_id, 300.0).await; + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + wait_for_event(&store, &thread_key, "session.execution_failed").await; + + // A newly running execution can still be waiting for its warm + // sandbox assignment. It gets the same periodic grace, but is failed + // if it remains unassigned after the grace window. + let running_thread = + ThreadKey::parse(format!("test:adopt-young-running-{}", uuid::Uuid::new_v4())).unwrap(); + let running_execution = orphaned_execution(&store, &running_thread, None, true).await; + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + assert!( + events(&store, &running_thread) + .await + .iter() + .all(|event| event.event_type != "session.execution_failed"), + "young running execution must survive sandbox assignment" + ); + + backdate_execution(&store, &running_execution, 300.0).await; + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + wait_for_event(&store, &running_thread, "session.execution_failed").await; + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn adopts_deferred_execution_after_lease_expires() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; + let thread_key = + ThreadKey::parse(format!("test:adopt-deferred-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + store + .claim_stdout_owner( + &execution_id, + "other-control-plane", + Duration::from_secs(60), + ) + .await + .expect("claim lease for other owner"); + + let backend = Arc::new(MockBackend::new( + SandboxStatus::Running, + vec![ + json!({"type": "item.completed", "item": {"id": "msg-1", "type": "agentMessage", "text": "Done: recovered after handoff.", "phase": "final_answer"}}).to_string(), + json!({"type": "turn.completed", "turn": {"id": "turn-1", "status": "completed"}}).to_string(), + ], + )); + let runtime = runtime_with(&store, backend.clone()); + + // While another control plane holds the stdout-owner lease the scan + // must defer instead of stealing the execution. + runtime.adopt_orphaned_executions().await; + wait_for_event(&store, &thread_key, "session.execution_adoption_deferred").await; + let all = events(&store, &thread_key).await; + assert!( + all.iter() + .all(|event| event.event_type != "session.execution_completed"), + "deferred execution must not be terminalized" + ); + + // Once the lease expires (owner died without releasing), a later + // scan adopts the execution and recovers the recorded terminal. The + // expiry is forced in the database rather than slept through so slow + // test databases cannot turn the first scan into the adopting one. + expire_stdout_lease(&store, &execution_id).await; + runtime.adopt_orphaned_executions().await; + wait_for_event(&store, &thread_key, "session.execution_adopted").await; + wait_for_event(&store, &thread_key, "session.execution_completed").await; } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn stdout_eof_fails_when_sandbox_no_longer_accepts_io() { + async fn periodic_scan_ignores_executions_owned_by_this_process() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:eof-gone-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-gone"), true).await; - + ThreadKey::parse(format!("test:adopt-own-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let (io, stdout, _stdin) = mock_io(); - backend.push_io(io).await; - let runtime = runtime_with(&store, backend.clone()); + assert!( + store + .claim_stdout_owner( + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60) + ) + .await + .expect("claim as this control plane") + ); + + // A healthy execution owned by the scanning process must be skipped + // silently: no deferral event, no sandbox status probe. + let mut state = OrphanAdoptionState::default(); runtime - .ensure_session_pipe(&thread_key, "sbx-gone") - .await - .expect("open initial pipe"); - backend.set_status(SandboxStatus::Gone); - drop(stdout); + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; - wait_for_event(&store, &thread_key, "session.execution_failed").await; let all = events(&store, &thread_key).await; - let failed = all - .iter() - .find(|event| event.event_type == "session.execution_failed") - .expect("failed event"); - let error = failed.payload["error"].as_str().unwrap_or_default(); - assert!( - error.contains("sandbox stdout closed before terminal output"), - "unexpected error: {error}" - ); - assert!( - error.contains("sandbox no longer accepts io"), - "expected sandbox status detail: {error}" - ); assert!( - !all.iter() - .any(|event| event.event_type == "session.stdout_pump_reattached"), - "gone sandbox should not reattach" + all.iter().all(|event| { + event.event_type != "session.execution_adoption_deferred" + && event.event_type != "session.execution_adopted" + && event.event_type != "session.execution_failed" + }), + "self-owned execution must not be touched by the scan" ); - assert_eq!(backend.opens(), 1); + store + .fail_execution_if_active(&execution_id, "test cleanup") + .await + .expect("terminalize execution"); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn adopts_finished_turn_from_recorded_sandbox_output() { + async fn spawned_adoption_loop_recovers_orphans() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:adopt-logs-{}", uuid::Uuid::new_v4())).unwrap(); + ThreadKey::parse(format!("test:adopt-loop-{}", uuid::Uuid::new_v4())).unwrap(); orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; let backend = Arc::new(MockBackend::new( SandboxStatus::Running, vec![ - json!({"type": "item.completed", "item": {"id": "msg-1", "type": "agentMessage", "text": "Done: pushed commit abc123.", "phase": "final_answer"}}).to_string(), + json!({"type": "item.completed", "item": {"id": "msg-1", "type": "agentMessage", "text": "Done: recovered by the loop.", "phase": "final_answer"}}).to_string(), json!({"type": "turn.completed", "turn": {"id": "turn-1", "status": "completed"}}).to_string(), ], )); let runtime = runtime_with(&store, backend.clone()); - runtime.adopt_orphaned_executions().await; + let adoption_loop = runtime.spawn_orphan_adoption(Duration::from_millis(50)); + wait_for_event(&store, &thread_key, "session.execution_adopted").await; wait_for_event(&store, &thread_key, "session.execution_completed").await; - let all = events(&store, &thread_key).await; - assert!( - all.iter() - .any(|event| event.event_type == "session.execution_adopted"), - "expected an adoption event" - ); - let completed = all - .iter() - .find(|event| event.event_type == "session.execution_completed") - .expect("completed event"); - assert_eq!( - completed.payload["result_text"].as_str(), - Some("Done: pushed commit abc123.") - ); - // The terminal came from recorded output; no live attach was needed. - assert_eq!(backend.opens(), 0); - let session = store.get_session(&thread_key).await.unwrap(); - assert_ne!(session.status.as_ref(), "failed"); + adoption_loop.abort(); + let _ = adoption_loop.await; } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn adopts_live_when_recorded_output_has_no_terminal() { + async fn periodic_scans_record_deferral_once() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:adopt-live-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; - - let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let (io, mut stdout, _stdin) = mock_io(); - backend.push_io(io).await; + ThreadKey::parse(format!("test:adopt-dedup-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + store + .claim_stdout_owner( + &execution_id, + "other-control-plane", + Duration::from_secs(60), + ) + .await + .expect("claim lease for other owner"); + let backend = Arc::new(MockBackend::new( + SandboxStatus::Running, + vec![ + json!({"type": "item.completed", "item": {"id": "msg-1", "type": "agentMessage", "text": "Done: recovered after release.", "phase": "final_answer"}}).to_string(), + json!({"type": "turn.completed", "turn": {"id": "turn-1", "status": "completed"}}).to_string(), + ], + )); let runtime = runtime_with(&store, backend.clone()); - runtime.adopt_orphaned_executions().await; - assert_eq!(backend.opens(), 1); - stdout - .write_all( - b"{\"type\":\"turn.completed\",\"turn\":{\"id\":\"turn-1\",\"status\":\"completed\"}}\n", - ) + // Repeated periodic scans over the same held lease must record the + // deferral event only once. + let mut state = OrphanAdoptionState::default(); + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; + let all = events(&store, &thread_key).await; + let deferrals = all + .iter() + .filter(|event| event.event_type == "session.execution_adoption_deferred") + .count(); + assert_eq!(deferrals, 1, "deferral event must be recorded once"); + + // Releasing the lease (a clean shutdown handoff) lets the next scan + // adopt immediately; this also terminalizes the execution before the + // test releases TEST_LOCK. + store + .release_stdout_owner(&execution_id, "other-control-plane") .await - .unwrap(); + .expect("release lease"); + runtime + .run_orphan_adoption_scan(&mut state, Some(PRE_SANDBOX_ORPHAN_GRACE)) + .await; wait_for_event(&store, &thread_key, "session.execution_completed").await; - let all = events(&store, &thread_key).await; - assert!( - all.iter().any(|event| { - event.event_type == "session.execution_adopted" - && event.payload["mode"] == json!("live_attach") - }), - "expected a live adoption event" - ); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn fails_orphans_whose_sandbox_is_gone() { + async fn shutdown_handoff_releases_owned_leases() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:adopt-gone-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; - - let backend = Arc::new(MockBackend::new(SandboxStatus::Gone, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); - runtime.adopt_orphaned_executions().await; - - wait_for_event(&store, &thread_key, "session.execution_failed").await; - let all = events(&store, &thread_key).await; - let failed = all - .iter() - .find(|event| event.event_type == "session.execution_failed") - .expect("failed event"); - let error = failed.payload["error"].as_str().unwrap_or_default(); + ThreadKey::parse(format!("test:handoff-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend); assert!( - error.contains("execution orphaned by control plane restart"), - "unexpected error: {error}" + store + .claim_stdout_owner( + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60) + ) + .await + .expect("claim as this control plane") ); + + runtime.handoff_owned_executions(Duration::ZERO).await; + + wait_for_event(&store, &thread_key, "session.stdout_owner_released").await; + // The lease is immediately claimable by a peer control plane; without + // the handoff it would only expire after the lease TTL. assert!( - error.contains("sandbox no longer accepts io"), - "expected status detail: {error}" + store + .claim_stdout_owner(&execution_id, "peer-control-plane", Duration::from_secs(5)) + .await + .expect("peer claims released lease") ); - assert_eq!(backend.opens(), 0); + store + .fail_execution_if_active(&execution_id, "test cleanup") + .await + .expect("terminalize execution"); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn fails_queued_orphans_that_never_received_input() { + async fn shutdown_handoff_waits_for_pre_fence_adoption_claim() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; - let thread_key = - ThreadKey::parse(format!("test:adopt-queued-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-mock"), false).await; - + let thread_key = ThreadKey::parse(format!( + "test:handoff-adoption-race-{}", + uuid::Uuid::new_v4() + )) + .unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); - runtime.adopt_orphaned_executions().await; + let runtime = runtime_with(&store, backend); - wait_for_event(&store, &thread_key, "session.execution_failed").await; - let all = events(&store, &thread_key).await; - let failed = all - .iter() - .find(|event| event.event_type == "session.execution_failed") - .expect("failed event"); - let error = failed.payload["error"].as_str().unwrap_or_default(); + // Model an adoption scan that crossed the read barrier before the + // shutdown fence but has not claimed its lease yet. + let adoption_permit = runtime + .acquire_sandbox_allocation_permit() + .await + .expect("pre-shutdown adoption permit"); + let mut handoff = { + let runtime = runtime.clone(); + tokio::spawn(async move { runtime.handoff_owned_executions(Duration::ZERO).await }) + }; + let deadline = Instant::now() + Duration::from_secs(2); + while !runtime.shutting_down.load(Ordering::SeqCst) { + assert!( + Instant::now() < deadline, + "handoff did not set shutdown fence" + ); + tokio::task::yield_now().await; + } assert!( - error.contains("orphaned before input was sent"), - "unexpected error: {error}" + timeout(Duration::from_millis(50), &mut handoff) + .await + .is_err(), + "handoff must wait for the pre-fence adoption pass" ); - assert_eq!(backend.opens(), 0); + assert!( + store + .claim_stdout_owner( + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60), + ) + .await + .expect("adoption claims while holding the permit") + ); + drop(adoption_permit); + handoff.await.expect("handoff task"); + + assert!( + store + .claim_stdout_owner(&execution_id, "peer-control-plane", Duration::from_secs(5)) + .await + .expect("peer claims the lease released by handoff") + ); + store + .fail_execution_if_active(&execution_id, "test cleanup") + .await + .expect("terminalize execution"); } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn release_thread_cancels_active_execution_and_clears_sandbox() { + async fn release_thread_cancels_owned_execution_and_stops_snapshotted_sandbox() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:release-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-release"), true).await; - + ThreadKey::parse(format!("test:release-runtime-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = + orphaned_execution(&store, &thread_key, Some("sbx-release-runtime"), true).await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); let runtime = runtime_with(&store, backend.clone()); + assert!( + store + .claim_stdout_owner( + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60), + ) + .await + .expect("claim release execution") + ); + + let outcome = runtime + .release_thread( + &thread_key, + Some("rel-stale-selection"), + Some("sbx-replaced"), + true, + ) + .await; + assert!(matches!(outcome, Err(SessionRuntimeError::BadRequest(_)))); + assert!(backend.stopped().is_empty()); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("get still-assigned session") + .sandbox_id + .as_deref(), + Some("sbx-release-runtime") + ); let outcome = runtime - .release_thread(&thread_key, Some("rel-test-1"), true) + .release_thread( + &thread_key, + Some("rel-runtime"), + Some("sbx-release-runtime"), + true, + ) .await .expect("release thread"); - assert_eq!(outcome.release_id.as_deref(), Some("rel-test-1")); - assert!(outcome.cancel_inflight); - assert!(outcome.sandbox_released); + assert_eq!(outcome.release_id.as_deref(), Some("rel-runtime")); assert!(outcome.execution_cancelled); - assert_eq!(backend.stopped(), vec!["sbx-release".to_owned()]); - - let session = store.get_session(&thread_key).await.expect("get session"); - assert_eq!(session.sandbox_id, None); - assert_eq!(session.status.as_ref(), "idle"); - - let execution = store - .latest_execution_for_thread(&thread_key) + assert!(outcome.sandbox_released); + assert_eq!(outcome.execution_id.as_deref(), Some(execution_id.as_str())); + assert_eq!(backend.stopped(), vec!["sbx-release-runtime"]); + let session = store + .get_session(&thread_key) .await - .expect("latest execution") - .expect("execution row"); - assert_eq!(execution.status.as_ref(), "cancelled"); - - wait_for_event(&store, &thread_key, "session.execution_cancelled").await; + .expect("get released session"); + assert_eq!(session.sandbox_id, None); + assert_eq!(session.harness_thread_id, None); + assert_eq!(session.status, SessionStatus::Idle); + assert!( + store + .append_event_if_stdout_owner( + &thread_key, + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60), + SESSION_OUTPUT_LINE_EVENT, + json!("stale output"), + ) + .await + .expect("stale output is fenced") + .is_none() + ); wait_for_event(&store, &thread_key, "session.released").await; } #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn release_thread_rejects_active_execution_without_cancel() { + async fn shutdown_handoff_waits_for_executions_to_finish() { let Some(store) = test_store().await else { return; }; let _serial = TEST_LOCK.lock().await; let thread_key = - ThreadKey::parse(format!("test:release-reject-{}", uuid::Uuid::new_v4())).unwrap(); - orphaned_execution(&store, &thread_key, Some("sbx-release-reject"), true).await; + ThreadKey::parse(format!("test:handoff-wait-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; + let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); + let runtime = runtime_with(&store, backend); + assert!( + store + .claim_stdout_owner( + &execution_id, + &runtime.stdout_owner_id, + Duration::from_secs(60) + ) + .await + .expect("claim as this control plane") + ); + + // The execution finishes while the drain is waiting; no lease should + // be released and no handoff event recorded. + let completer_store = store.clone(); + let completer_id = execution_id.clone(); + let completer = tokio::spawn(async move { + sleep(Duration::from_millis(300)).await; + completer_store + .complete_execution_if_active(&completer_id) + .await + .expect("complete execution") + }); + runtime + .handoff_owned_executions(Duration::from_secs(5)) + .await; + let completed = completer.await.expect("completer task"); + assert!( + completed.is_some(), + "the completer, not the handoff, must terminalize the execution" + ); + + let all = events(&store, &thread_key).await; + assert!( + all.iter() + .all(|event| event.event_type != "session.stdout_owner_released"), + "finished execution must not be handed off" + ); + } + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn shutdown_fences_new_stdout_claims() { + let Some(store) = test_store().await else { + return; + }; + let _serial = TEST_LOCK.lock().await; let backend = Arc::new(MockBackend::new(SandboxStatus::Running, Vec::new())); - let runtime = runtime_with(&store, backend.clone()); + let runtime = runtime_with(&store, backend); + // Nothing owned: the handoff returns immediately but still flips + // the shutdown fence. + runtime.handoff_owned_executions(Duration::ZERO).await; + + let thread_key = + ThreadKey::parse(format!("test:handoff-fence-{}", uuid::Uuid::new_v4())).unwrap(); + let execution_id = orphaned_execution(&store, &thread_key, Some("sbx-mock"), true).await; let error = runtime - .release_thread(&thread_key, Some("rel-test-reject"), false) + .claim_stdout_owner(&execution_id) .await - .expect_err("release should reject active executions without cancellation"); - + .expect_err("claims after shutdown must be rejected"); assert!( - error.to_string().contains("pass cancel_inflight=true"), + matches!(error, SessionRuntimeError::ShuttingDown), "unexpected error: {error}" ); - assert_eq!(backend.stopped(), Vec::::new()); - - let session = store.get_session(&thread_key).await.expect("get session"); - assert_eq!(session.sandbox_id.as_deref(), Some("sbx-release-reject")); - - let execution = store - .latest_execution_for_thread(&thread_key) + store + .fail_execution_if_active(&execution_id, "test cleanup") .await - .expect("latest execution") - .expect("execution row"); - assert_eq!(execution.status.as_ref(), "running"); + .expect("terminalize execution"); } } diff --git a/services/api-rs/crates/centaur-session-runtime/src/title_generator.rs b/services/api-rs/crates/centaur-session-runtime/src/title_generator.rs new file mode 100644 index 000000000..f8c80c16e --- /dev/null +++ b/services/api-rs/crates/centaur-session-runtime/src/title_generator.rs @@ -0,0 +1,438 @@ +use std::{env, sync::Arc, time::Duration}; + +use serde_json::{Value, json}; +use thiserror::Error; + +const SESSION_TITLE_MODEL: &str = "gpt-5.4-nano"; +const SESSION_TITLE_MAX_SOURCE_CHARS: usize = 4_000; +const SESSION_TITLE_MAX_CHARS: usize = 80; +const SESSION_TITLE_REQUEST_TIMEOUT: Duration = Duration::from_secs(4); + +#[derive(Clone)] +pub(crate) struct OpenAiSessionTitleGenerator { + api_key: Arc, + client: reqwest::Client, +} + +impl OpenAiSessionTitleGenerator { + pub(crate) fn from_env() -> Option { + let api_key = env::var("OPENAI_API_KEY").ok()?; + let api_key = api_key.trim(); + if api_key.is_empty() || api_key == "OPENAI_API_KEY" { + return None; + } + let client = reqwest::Client::builder() + .timeout(SESSION_TITLE_REQUEST_TIMEOUT) + .build() + .ok()?; + Some(Self { + api_key: Arc::from(api_key.to_owned()), + client, + }) + } + + pub(crate) async fn generate( + &self, + source: String, + ) -> Result { + let body = json!({ + "model": SESSION_TITLE_MODEL, + "instructions": "Generate a short session title for the user's request. Return only the title. Use commit-message style with an imperative verb first, such as Fix, Investigate, Add, Update, Debug, Review, Explain, or Analyze. Keep it to 5 words max; 6-7 words are okay only when needed for a product name. Do not include punctuation, quotes, emoji, markdown, or a trailing period.", + "input": format!("User request:\n{}", source), + "max_output_tokens": 24, + }); + let response = self + .client + .post("https://api.openai.com/v1/responses") + .bearer_auth(self.api_key.as_ref()) + .json(&body) + .send() + .await?; + let status = response.status(); + let text = response.text().await?; + if !status.is_success() { + return Err(SessionTitleGenerationError::HttpStatus { status, body: text }); + } + openai_response_output_text(&text).ok_or(SessionTitleGenerationError::MissingOutput) + } +} + +pub(crate) fn session_title_source_from_parts(parts: &[Value]) -> Option { + let mut text_blocks = Vec::new(); + let mut slack_thread_source = None; + let mut attachment_names = Vec::new(); + for part in parts { + match part { + Value::String(text) => { + collect_title_source_text(text, &mut text_blocks, &mut slack_thread_source); + } + Value::Object(object) => { + if let Some(text) = object.get("text").and_then(Value::as_str) { + collect_title_source_text(text, &mut text_blocks, &mut slack_thread_source); + } + for key in ["name", "title", "filename"] { + if let Some(name) = object.get(key).and_then(Value::as_str) + && let Some(name) = clean_nonempty(name) + { + attachment_names.push(name.to_owned()); + break; + } + } + } + _ => {} + } + } + let source = slack_thread_source + .or_else(|| text_blocks.first().cloned()) + .or_else(|| { + attachment_names + .first() + .map(|name| format!("Analyze attachment {name}")) + })?; + Some(truncate_chars(&source, SESSION_TITLE_MAX_SOURCE_CHARS)) +} + +fn collect_title_source_text( + raw_text: &str, + text_blocks: &mut Vec, + slack_thread_source: &mut Option, +) { + if slack_thread_source.is_none() + && let Some(text) = slack_thread_context_title_source(raw_text) + && title_source_has_signal(&text) + { + *slack_thread_source = Some(text); + } + if is_session_context_text(raw_text) { + return; + } + if let Some(text) = clean_title_source_text(raw_text) + && title_source_has_signal(&text) + { + text_blocks.push(text); + } +} + +fn slack_thread_context_title_source(text: &str) -> Option { + if !text.trim_start().starts_with("# Slack Thread Context") { + return None; + } + + let mut in_first_message = false; + let mut lines = Vec::new(); + for line in text.lines() { + let trimmed = line.trim(); + if trimmed == "# Current Request" { + break; + } + if !in_first_message { + if trimmed.starts_with("1. ") && trimmed.ends_with(':') { + in_first_message = true; + } + continue; + } + if trimmed.starts_with("2. ") && trimmed.ends_with(':') { + break; + } + if trimmed.is_empty() { + if lines.is_empty() { + continue; + } + break; + } + lines.push(trimmed); + } + + let text = lines.join(" "); + clean_title_source_text(&text) +} + +fn title_source_has_signal(text: &str) -> bool { + let normalized = normalize_low_signal_text(text); + if normalized.is_empty() { + return false; + } + if is_low_signal_phrase(&normalized) { + return false; + } + + let words = normalized.split_whitespace().collect::>(); + if words.iter().all(|word| is_low_signal_word(word)) { + return false; + } + + true +} + +fn normalize_low_signal_text(text: &str) -> String { + let mut output = String::with_capacity(text.len()); + let lowercase = text.to_lowercase(); + let mut chars = lowercase.chars().peekable(); + while let Some(ch) = chars.next() { + if ch == ':' { + let mut emoji_name = String::new(); + while let Some(next) = chars.peek().copied() { + chars.next(); + if next == ':' { + break; + } + if next.is_ascii_alphanumeric() || matches!(next, '_' | '-' | '+') { + emoji_name.push(next); + continue; + } + output.push(' '); + output.push_str(&emoji_name); + output.push(next); + emoji_name.clear(); + break; + } + continue; + } + if ch.is_alphanumeric() { + output.push(ch); + } else { + output.push(' '); + } + } + output.split_whitespace().collect::>().join(" ") +} + +fn is_low_signal_phrase(text: &str) -> bool { + matches!( + text, + "hey" + | "hi" + | "hello" + | "yo" + | "hey bot" + | "hi bot" + | "hello bot" + | "hey ai" + | "hi ai" + | "hello ai" + | "thread" + | "help" + | "can you help" + | "can you help me" + | "please help" + | "pls help" + ) +} + +fn is_low_signal_word(word: &str) -> bool { + matches!( + word, + "hey" | "hi" | "hello" | "yo" | "bot" | "ai" | "thread" | "please" | "pls" + ) +} + +fn clean_title_source_text(text: &str) -> Option { + let text = strip_slack_user_mentions(text) + .replace('\r', "\n") + .split_whitespace() + .collect::>() + .join(" "); + let mut text = clean_nonempty(&text)?.to_owned(); + if text.starts_with('@') { + text = text + .char_indices() + .find(|(_, ch)| ch.is_whitespace()) + .map(|(index, _)| text[index..].trim_start().to_owned()) + .unwrap_or_default(); + } + clean_nonempty(&text).map(str::to_owned) +} + +fn strip_slack_user_mentions(text: &str) -> String { + let mut output = String::with_capacity(text.len()); + let mut rest = text; + while let Some(start) = rest.find("<@") { + output.push_str(&rest[..start]); + let mention = &rest[start + 2..]; + let Some(end) = mention.find('>') else { + output.push_str(&rest[start..]); + return output; + }; + rest = &mention[end + 1..]; + } + output.push_str(rest); + output +} + +fn is_session_context_text(text: &str) -> bool { + let text = text.trim_start(); + [ + "# Requester Context", + "# Slack Session Context", + "# Slack Thread Context", + "Earlier Slack thread attachment", + ] + .iter() + .any(|prefix| text.starts_with(prefix)) +} + +pub(crate) fn sanitize_session_title(title: &str) -> Option { + let title = title + .trim() + .trim_matches(|ch: char| { + matches!( + ch, + '"' | '\'' | '`' | '*' | '_' | '-' | ':' | ';' | ',' | '.' + ) + }) + .split_whitespace() + .collect::>() + .join(" "); + let title = clean_nonempty(&title)?; + let words = title + .split_whitespace() + .take(7) + .map(|word| { + word.trim_matches(|ch: char| matches!(ch, '"' | '\'' | '`' | ',' | '.' | ':' | ';')) + }) + .filter(|word| !word.is_empty()) + .collect::>(); + if words.is_empty() { + return None; + } + Some(truncate_chars(&words.join(" "), SESSION_TITLE_MAX_CHARS)) +} + +fn openai_response_output_text(body: &str) -> Option { + let value: Value = serde_json::from_str(body).ok()?; + if let Some(text) = value.get("output_text").and_then(Value::as_str) + && clean_nonempty(text).is_some() + { + return Some(text.to_owned()); + } + for output in value.get("output").and_then(Value::as_array)? { + let Some(content) = output.get("content").and_then(Value::as_array) else { + continue; + }; + for item in content { + if let Some(text) = item.get("text").and_then(Value::as_str) + && clean_nonempty(text).is_some() + { + return Some(text.to_owned()); + } + } + } + None +} + +fn clean_nonempty(value: &str) -> Option<&str> { + let value = value.trim(); + if value.is_empty() { None } else { Some(value) } +} + +fn truncate_chars(value: &str, max_chars: usize) -> String { + let mut truncated = value.chars().take(max_chars).collect::(); + if truncated.ends_with(char::is_whitespace) { + truncated = truncated.trim_end().to_owned(); + } + truncated +} + +#[derive(Debug, Error)] +pub enum SessionTitleGenerationError { + #[error("OpenAI title response did not include output text")] + MissingOutput, + #[error("OpenAI title request failed with status {status}: {body}")] + HttpStatus { + status: reqwest::StatusCode, + body: String, + }, + #[error(transparent)] + Http(#[from] reqwest::Error), +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn session_title_source_prefers_user_ask_over_slack_context() { + let parts = vec![ + json!({ + "type": "text", + "text": "# Requester Context\n\nThe Slack user who prompted this turn is Alice." + }), + json!({ + "type": "text", + "text": "<@U123> please fix the memory leak in the worker" + }), + ]; + + assert_eq!( + session_title_source_from_parts(&parts), + Some("please fix the memory leak in the worker".to_owned()) + ); + } + + #[test] + fn session_title_source_uses_first_slack_thread_message() { + let parts = vec![ + json!({ + "type": "text", + "text": "# Slack Thread Context\n\nEarlier messages in this Slack thread, in chronological order:\n\n1. Alice:\n Planning to replace the billing export job with a streaming worker because the nightly batch keeps timing out\n\n# Current Request\n\nThe user message follows in the next content block.\n---" + }), + json!({ + "type": "text", + "text": "<@U123> investigate this" + }), + ]; + + assert_eq!( + session_title_source_from_parts(&parts), + Some( + "Planning to replace the billing export job with a streaming worker because the nightly batch keeps timing out" + .to_owned() + ) + ); + } + + #[test] + fn session_title_source_skips_low_signal_wakeups() { + assert_eq!( + session_title_source_from_parts(&[ + json!({"type": "text", "text": "<@U123> Hey"}), + json!({"type": "text", "text": ":thread:"}), + ]), + None + ); + + assert_eq!( + session_title_source_from_parts(&[ + json!({"type": "text", "text": "<@U123> Hey"}), + json!({"type": "text", "text": "Can you investigate queue stalls?"}), + ]), + Some("Can you investigate queue stalls?".to_owned()) + ); + } + + #[test] + fn sanitize_session_title_keeps_model_wording() { + assert_eq!( + sanitize_session_title("Memory leak in worker queue needs investigation immediately"), + Some("Memory leak in worker queue needs investigation".to_owned()) + ); + assert_eq!( + sanitize_session_title("\"Fix worker memory leak.\""), + Some("Fix worker memory leak".to_owned()) + ); + } + + #[test] + fn openai_response_output_text_reads_responses_api_shapes() { + assert_eq!( + openai_response_output_text(r#"{"output_text":"Fix worker memory leak"}"#), + Some("Fix worker memory leak".to_owned()) + ); + assert_eq!( + openai_response_output_text( + r#"{"output":[{"content":[{"type":"output_text","text":"Add Tempo Explorer filter"}]}]}"# + ), + Some("Add Tempo Explorer filter".to_owned()) + ); + } +} diff --git a/services/api-rs/crates/centaur-session-sqlx/MIGRATIONS.md b/services/api-rs/crates/centaur-session-sqlx/MIGRATIONS.md new file mode 100644 index 000000000..2e7237f12 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/MIGRATIONS.md @@ -0,0 +1,50 @@ +# Session SQLx migration safety + +The API embeds this crate's migrations at build time. SQLx validates every +applied version and SHA-384 checksum before the API starts, so an applied file +must never be edited, renamed, or removed. + +## TipLink lineage + +TipLink production has versions `0001` through `0032` applied. Those files are +the canonical history for the fork and are intentionally different from the +same version numbers in upstream Centaur. In particular, `0019`, `0023`, and +`0025` contain TipLink-specific behavior and checksums. + +For the July 2026 upstream sync, upstream versions `0032` through `0038` are +shifted to TipLink versions `0033` through `0039`. Upstream versions `0040` +through `0042` retain their original numbers. Fork migration `0039` also +contains the forward-only reconciliation for Fineas public Slack company +context. Fork migration `0043` appends assignment-bound sandbox content +revision tracking; it is deliberately backward compatible with older binaries +that update `sandbox_id` without knowing the new nullable column. + +The checksum manifests checked by `.github/scripts/check-migration-order.sh` +lock the release migration tree. Append a manifest entry for a genuinely new +migration; never replace an existing entry. + +The Rails migration +`20260624000100_add_password_grant_to_broker_credentials.rb` is also the +TipLink compatibility version. It converts the historical `credential_kind` +column before enforcing the upstream `grant` schema and must not be replaced +with upstream's simpler body. Rails does not validate migration checksums, so +the release manifest is the immutability boundary for that history. + +## Rollback requirements + +These migrations are forward-only. There are no SQLx down migrations, and a +failed later migration does not undo earlier successful versions. + +After a new migration has applied, an older API image whose embedded migration +set does not contain that version will fail startup when `RUN_MIGRATIONS=true`. +Before rollout, prepare and test one of these application rollback paths: + +1. A bridge image with the prior application behavior and the exact new + migration files and checksums embedded. +2. The prior image with `RUN_MIGRATIONS=false`, tested against the forward + schema and any new enum or state values created by the release. + +Restoring a database snapshot is the only full schema rollback. It is a last +resort because it discards writes made after the snapshot. Apply console/Rails +migrations before rolling the API whenever new API behavior depends on the +console schema or routes. diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/.checksums.sha384 b/services/api-rs/crates/centaur-session-sqlx/migrations/.checksums.sha384 new file mode 100644 index 000000000..b5a0ab59c --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/.checksums.sha384 @@ -0,0 +1,43 @@ +92b33283d76e4265c9fff457e81bb83a09e489a9a27e78dffcf29006eba7e850d8b1d80a3e6f1f08d728e2cfa2b2db15 services/api-rs/crates/centaur-session-sqlx/migrations/0001_session_control_plane.sql +d87e505d95c7e91c463f675961f87197bad6fd8e132d5e1fc021ea78a4f84a4a3ef38315ad8c51a8ebfaf988d4f0a923 services/api-rs/crates/centaur-session-sqlx/migrations/0002_session_event_notifications.sql +4a9f482eb8190f4c2abf599d1630e4f4d0f442b1f495533b223be64a4858854725a23d55b5db3df4cfb39b7e60cd4a33 services/api-rs/crates/centaur-session-sqlx/migrations/0003_session_iron_control_principal.sql +640d069dc8b5acf2d3a9bc62f1544e84f9968ed4aa041929c2dbb3ef3e79de5093338f87cae2d32c3f95314e1a0ddd3a services/api-rs/crates/centaur-session-sqlx/migrations/0004_session_warm_pool.sql +7485641d877d0ea09d06aee5ef78093fc2ec8a716f5b46b5850c217cf567cb3458f518c3b20521f678af862d7da1d971 services/api-rs/crates/centaur-session-sqlx/migrations/0005_session_handoff_idempotency.sql +f9e369500e6e3e140498797b083436260d8d88c46b5145b94cbdbbc8c693ad070d780966b87110922499b071a75940a6 services/api-rs/crates/centaur-session-sqlx/migrations/0006_session_persona_id.sql +f3e356ca2ffa0a9dc42cfafeed9ea507a30e1a402ce70b888b7f48f6ebde907e1e9c3a3b0b7be96110e0feef1741be30 services/api-rs/crates/centaur-session-sqlx/migrations/0007_absurd_workflows.sql +01a123b518bcbe7ba288f08f471bd2ee65a70e69aca2e56beb3849986880a6c8ab538dc17bf6b799d8b8a6bb5405ad75 services/api-rs/crates/centaur-session-sqlx/migrations/0008_absurd_complete_run_clears_failure.sql +06fb36fa25467808c5c74cd3df51389c10d8ede921bec55d5e4719ffa329ff0792e4f02635e2496889ba330b4477a753 services/api-rs/crates/centaur-session-sqlx/migrations/0009_absurd_await_event_task_guard.sql +2cc855ad49192675edfec7e460c1edeca83d95c7387db854c92c319dfd9957380a024b1a614387d8c384dc3db6a4c898 services/api-rs/crates/centaur-session-sqlx/migrations/0010_user_feedback.sql +af6802d40d2281fcee9afd4f44970a8a1d39b928ba5ff8b504f8cdf63a70f4257c5bbf73b9c80c581505dd7479fa28b8 services/api-rs/crates/centaur-session-sqlx/migrations/0011_slack_sync_tables.sql +4c5484e974feda89ced85376ababe05457ebc1e9d2756b835c39fcf9c010a318f22166988da9f82d6b10da4587d1de38 services/api-rs/crates/centaur-session-sqlx/migrations/0012_company_context_documents.sql +5a76b4f51b74d6da97ca7af81dd9baa7a414ddfef3514d176f24d954124f6ad7628c8ba4624814c4ee49890b1d6cdd20 services/api-rs/crates/centaur-session-sqlx/migrations/0013_google_drive_sync_tables.sql +f6e89154b724cc1433a158f6adf485767339ab2875538bc08db0726b8d8e65034d3bd8a73444862b93f6ac5fcd79c2dc services/api-rs/crates/centaur-session-sqlx/migrations/0014_google_calendar_sync_tables.sql +5226709400e6bab8e1c95cc4bae30bb927e5778c9f847697f73a241c1d4bb44ff3234ff6b5711af1e8f0d541125f3a82 services/api-rs/crates/centaur-session-sqlx/migrations/0015_linear_sync_tables.sql +47f2607b186daf18cf2623c57db836b4a8c41524b886f406de4ac69bc0c4cd531d82d3c43ae7817693e85d9b90f2c866 services/api-rs/crates/centaur-session-sqlx/migrations/0016_slack_context_rls.sql +8f9c444884532c79dc109c56bc63fc30b16f5367c3bad88d44383295ebac1de63688242a55fede8451767b62f34c8d4e services/api-rs/crates/centaur-session-sqlx/migrations/0017_slack_sync_message_attachments.sql +8b1bfaec97b359a782c0036289a7c63a1799555e32af208a61dbc8e7f51733452f644dc6599ffbc8032f822af4f29e5e services/api-rs/crates/centaur-session-sqlx/migrations/0018_slack_context_rls_admin_channels.sql +01c4a4daf4fcd3941d81947a89bafb079882c1195312d86fdf8d8cdfc66e9c54e06eea448b18bd44a3f579346376d271 services/api-rs/crates/centaur-session-sqlx/migrations/0019_company_context_public_slack_docs.sql +e510559d9fa76a5d561203a002d7ef253ae4bfdb4991d0b7593ad4aa7e3e45e3f9b2836be5309d9e5d446f7ea276e075 services/api-rs/crates/centaur-session-sqlx/migrations/0020_centaur_readonly_role.sql +4a794c94abd60a11b0f582c67fe556295bc24a7420188e41d6f5f8ac084619b45ed7a699b1cf26be72a83a0c74b4f527 services/api-rs/crates/centaur-session-sqlx/migrations/0021_centaur_readonly_role_only.sql +48290511f4040be3592da6355a49f9fa9c4b73f1fe98dd62de7f50d16546559211ce8881df585ed1d61041d123ca264d services/api-rs/crates/centaur-session-sqlx/migrations/0022_etl_context_rls.sql +2459805495164decff2d73452cfab4d74653d145da4de71b7b3f9f482e241fe09bed900f6f2d8500b0116ae1ea29fc67 services/api-rs/crates/centaur-session-sqlx/migrations/0023_drop_slack_context_rls_admin_channels.sql +d36a02d3cb1586cfa6311f5254da2a65c66238f65738b0dc94e277ae61d62485620e52e25c2e3ced6bd0e4c08ede41b6 services/api-rs/crates/centaur-session-sqlx/migrations/0024_centaur_readonly_rls_policies.sql +74e25f7193d9975eaca20ca8b31afbcb5e50616fc51414a097f3cc76ced278cfdcbe9f50b980a9f6dcc3db86bc7be83e services/api-rs/crates/centaur-session-sqlx/migrations/0025_session_event_execution_type_idx.sql +d998621b4d6db9d86692f3a853842ab239380ea61628f810722d371c390a1a44947a7d71f75153b6cb1a9997526651cc services/api-rs/crates/centaur-session-sqlx/migrations/0026_slack_archive_imports.sql +552d69ebfefb0e6c50f04f941b99f6484b2f927695f1ef65c228b903e13eb7c88341301ae5170b00c311f74d6f86f9ac services/api-rs/crates/centaur-session-sqlx/migrations/0027_drop_slack_archive_import_workspace.sql +3431ef06d7af8234f78d3b335487b10c4910d7a76ab44a2ff213d5e191a550b90802bfe7a7c217dc3effb2d9a6a15e68 services/api-rs/crates/centaur-session-sqlx/migrations/0028_slack_dm_sync_tables.sql +fd4aaae62dd5fd310ac028c335dfda19eb2169c199b738c068fb1ef0716426ff00623382f18be5f1a33ef4f7ebf109e3 services/api-rs/crates/centaur-session-sqlx/migrations/0029_slack_dm_context_documents.sql +c472c35a307682a84bee8172af27348457442b6d7598dab1eaa23ac8068c79d33c544fa078ded09b130792ab673f0142 services/api-rs/crates/centaur-session-sqlx/migrations/0030_slack_dm_conversation_context_documents.sql +2feeec8685e4e70ab831710247fe775c160fb97a7d5802cdfc9abaaebca914b5b6ddd49640cb5779bdf256d6595ea1b0 services/api-rs/crates/centaur-session-sqlx/migrations/0031_google_docs_oauth_sync_tables.sql +2acc60a0f3c8c9f8fafab4ae1ff394f1b3b7d7f2e088afc573ebe7e528b8a3501e1877508c2df53540fba51743535193 services/api-rs/crates/centaur-session-sqlx/migrations/0032_session_sandbox_capabilities.sql +875bf729aa1324d8db01e420825ca56f978a33f98ca73605c0a52d97a730ff7c625c27b6cf11caacb78f8062b9ad85b1 services/api-rs/crates/centaur-session-sqlx/migrations/0033_session_title.sql +94fd3890f9791d15a206570839139c8a1c13b6ff47e286635f16951eaf80090025a85dc158dd28c3d207de420662ae45 services/api-rs/crates/centaur-session-sqlx/migrations/0034_session_sandbox_activity.sql +f4e1eb86575901edea9ea540ec53354f4b159616080850fa79b8f68f91a3a6c592905af94d0b9e4299208418ad254775 services/api-rs/crates/centaur-session-sqlx/migrations/0035_session_execution_stdout_owner.sql +e4aed269a8eb82b921399de3fc399e7e2687c04519d9e581a5337b673f9ea2f237b71a1a5c49c5f62856682db971259d services/api-rs/crates/centaur-session-sqlx/migrations/0036_session_sandbox_api_server_capability.sql +0ffce7c31d8a104b6d7bbd6e56a8036325bc7cd8d1226eb671b08ed999583b9dd56059f60400fef2d687fc463c0c993f services/api-rs/crates/centaur-session-sqlx/migrations/0037_readonly_all_workflow_queues.sql +b1a1b0fe52fdd876bd53a96b2fff43568cc4631bf59712f5231f37f25cf209d9f72db0b5a3b3566cbd83b72b5e1861bf services/api-rs/crates/centaur-session-sqlx/migrations/0038_session_sandbox_repo_cache_access.sql +8a2d3e308204abe4fd807f9c9bebdc060a7c3c6e8950a4df6469bc07e973867486c0357921779a46a03a68cc6c828ec4 services/api-rs/crates/centaur-session-sqlx/migrations/0039_slack_private_channels.sql +65cbd5bafcfd4e124d51bd99cc501fee923e87882d1032dad34f4cfba3fd78b5d4869b61bb765f8d76310e520f44cac2 services/api-rs/crates/centaur-session-sqlx/migrations/0040_granola_sync_tables.sql +38f3d13f44fa29264529012118f6b3921de1cf7e8b75e0e3ffcd1b207124eb7e0f6bde7504cdf3ac51d29a99291a77b0 services/api-rs/crates/centaur-session-sqlx/migrations/0041_attio_sync_tables.sql +dcd05c897c50a15b8bc2342e76d520133ac2370c23e88e119d6a7ff4a615a88b5512209a33ee04c5070f4f4afeda020e services/api-rs/crates/centaur-session-sqlx/migrations/0042_centaur_readonly_slack_dm_rls.sql +fd353077080a2cbeaaa7242f5726415663baeb00b74dd1743db01cbeda51f0748714f0f59400907411bed45c52fcac2a services/api-rs/crates/centaur-session-sqlx/migrations/0043_session_sandbox_content_revision.sql diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0033_session_title.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0033_session_title.sql new file mode 100644 index 000000000..7e20b465e --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0033_session_title.sql @@ -0,0 +1,2 @@ +alter table sessions + add column if not exists title text; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0034_session_sandbox_activity.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0034_session_sandbox_activity.sql new file mode 100644 index 000000000..eac1434ca --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0034_session_sandbox_activity.sql @@ -0,0 +1,21 @@ +alter table sessions + add column if not exists sandbox_last_active_at timestamptz; + +update sessions +set sandbox_last_active_at = coalesce(sandbox_last_active_at, updated_at, created_at) +where sandbox_id is not null; + +create index if not exists sessions_sandbox_activity_idx + on sessions (sandbox_last_active_at, thread_key) + where sandbox_id is not null; + +alter table session_warm_sandboxes + drop constraint if exists session_warm_sandboxes_status_supported; + +alter table session_warm_sandboxes + add constraint session_warm_sandboxes_status_supported + check (status in ('ready', 'claimed', 'evicting', 'failed')); + +create index if not exists session_warm_sandboxes_evicting_idx + on session_warm_sandboxes (updated_at, sandbox_id) + where status = 'evicting'; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0035_session_execution_stdout_owner.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0035_session_execution_stdout_owner.sql new file mode 100644 index 000000000..9dd0c44b1 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0035_session_execution_stdout_owner.sql @@ -0,0 +1,7 @@ +alter table session_executions + add column if not exists stdout_owner_id text, + add column if not exists stdout_owner_lease_expires_at timestamptz; + +create index if not exists session_executions_stdout_owner_lease_idx + on session_executions (stdout_owner_lease_expires_at) + where status in ('queued', 'running') and stdout_owner_id is not null; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0036_session_sandbox_api_server_capability.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0036_session_sandbox_api_server_capability.sql new file mode 100644 index 000000000..f7faa61c5 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0036_session_sandbox_api_server_capability.sql @@ -0,0 +1,7 @@ +alter table sessions + add column if not exists sandbox_api_server_enabled boolean; + +update sessions +set sandbox_api_server_enabled = true +where sandbox_observability_enabled is not null + and sandbox_api_server_enabled is null; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0037_readonly_all_workflow_queues.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0037_readonly_all_workflow_queues.sql new file mode 100644 index 000000000..5a0095898 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0037_readonly_all_workflow_queues.sql @@ -0,0 +1,91 @@ +select absurd.create_queue('centaur_workflows'); +select absurd.create_queue('centaur_workflows_slack_live'); +select absurd.create_queue('centaur_workflows_etl'); +select absurd.create_queue('centaur_workflows_etl_backfill'); + +create or replace view centaur_readonly_workflow_runs as +select + 'centaur_workflows'::text as queue_name, + r.run_id::text as run_id, + t.task_id::text as task_id, + t.task_name, + t.params ->> 'workflow_name' as workflow_name, + t.params ->> 'harness_type' as harness_type, + t.state, + t.attempts, + t.max_attempts, + t.enqueue_at as created_at, + t.first_started_at, + r.started_at, + r.completed_at, + r.failed_at, + r.available_at, + r.claimed_by is not null as claimed, + t.cancelled_at +from absurd.t_centaur_workflows t +left join absurd.r_centaur_workflows r on r.run_id = t.last_attempt_run +union all +select + 'centaur_workflows_slack_live'::text as queue_name, + r.run_id::text as run_id, + t.task_id::text as task_id, + t.task_name, + t.params ->> 'workflow_name' as workflow_name, + t.params ->> 'harness_type' as harness_type, + t.state, + t.attempts, + t.max_attempts, + t.enqueue_at as created_at, + t.first_started_at, + r.started_at, + r.completed_at, + r.failed_at, + r.available_at, + r.claimed_by is not null as claimed, + t.cancelled_at +from absurd.t_centaur_workflows_slack_live t +left join absurd.r_centaur_workflows_slack_live r on r.run_id = t.last_attempt_run +union all +select + 'centaur_workflows_etl'::text as queue_name, + r.run_id::text as run_id, + t.task_id::text as task_id, + t.task_name, + t.params ->> 'workflow_name' as workflow_name, + t.params ->> 'harness_type' as harness_type, + t.state, + t.attempts, + t.max_attempts, + t.enqueue_at as created_at, + t.first_started_at, + r.started_at, + r.completed_at, + r.failed_at, + r.available_at, + r.claimed_by is not null as claimed, + t.cancelled_at +from absurd.t_centaur_workflows_etl t +left join absurd.r_centaur_workflows_etl r on r.run_id = t.last_attempt_run +union all +select + 'centaur_workflows_etl_backfill'::text as queue_name, + r.run_id::text as run_id, + t.task_id::text as task_id, + t.task_name, + t.params ->> 'workflow_name' as workflow_name, + t.params ->> 'harness_type' as harness_type, + t.state, + t.attempts, + t.max_attempts, + t.enqueue_at as created_at, + t.first_started_at, + r.started_at, + r.completed_at, + r.failed_at, + r.available_at, + r.claimed_by is not null as claimed, + t.cancelled_at +from absurd.t_centaur_workflows_etl_backfill t +left join absurd.r_centaur_workflows_etl_backfill r on r.run_id = t.last_attempt_run; + +grant select on table centaur_readonly_workflow_runs to centaur_readonly; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0038_session_sandbox_repo_cache_access.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0038_session_sandbox_repo_cache_access.sql new file mode 100644 index 000000000..a8e4dab30 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0038_session_sandbox_repo_cache_access.sql @@ -0,0 +1,10 @@ +alter table sessions + add column if not exists sandbox_repo_cache_access text; + +update sessions +set sandbox_repo_cache_access = case + when sandbox_repo_cache_enabled then 'all' + else 'none' +end +where sandbox_repo_cache_access is null + and sandbox_repo_cache_enabled is not null; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0039_slack_private_channels.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0039_slack_private_channels.sql new file mode 100644 index 000000000..eda032787 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0039_slack_private_channels.sql @@ -0,0 +1,130 @@ +alter table slack_sync_channels + add column if not exists is_private boolean; + +-- Existing rows predate the dedicated privacy column. Trust an explicit +-- boolean from the stored Slack payload; when privacy is absent or malformed, +-- fail closed until a live Slack sync can classify the channel. +update slack_sync_channels +set is_private = case + when jsonb_typeof(raw_payload -> 'is_private') = 'boolean' + then (raw_payload ->> 'is_private')::boolean + else true +end; + +alter table slack_sync_channels + alter column is_private set default true, + alter column is_private set not null; + +create index if not exists idx_slack_sync_channels_private + on slack_sync_channels (is_private, channel_id); + +drop policy if exists centaur_readonly_slack_sync_channels_select + on slack_sync_channels; +create policy centaur_readonly_slack_sync_channels_select + on slack_sync_channels + for select + to centaur_readonly + using ( + not is_private + or channel_id = centaur_current_slack_channel_id() + ); + +drop policy if exists centaur_readonly_slack_sync_message_attachments_select + on slack_sync_message_attachments; +create policy centaur_readonly_slack_sync_message_attachments_select + on slack_sync_message_attachments + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_sync_channels channels + where channels.channel_id = slack_sync_message_attachments.channel_id + ) + ); + +drop policy if exists centaur_readonly_slack_sync_messages_select + on slack_sync_messages; +create policy centaur_readonly_slack_sync_messages_select + on slack_sync_messages + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_sync_channels channels + where channels.channel_id = slack_sync_messages.channel_id + ) + ); + +drop policy if exists centaur_readonly_company_context_documents_select + on company_context_documents; +create policy centaur_readonly_company_context_documents_select + on company_context_documents + for select + to centaur_readonly + using ( + source <> 'slack' + or exists ( + select 1 + from slack_sync_channels channels + where channels.channel_id = metadata ->> 'channel_id' + ) + ); + +-- Fineas company context intentionally exposes documents from public, +-- syncable Slack channels across channel-scoped principals. Keep direct access +-- to the principal's current channel (including a private channel), but never +-- use the Slack channel-id prefix as a privacy signal. +create or replace function centaur_slack_channel_is_public_syncable( + _schema name, + _channel_id text +) +returns boolean +language plpgsql +stable +security definer +set search_path = pg_catalog +as $$ +declare + public_syncable boolean; +begin + execute format( + 'select exists ( + select 1 + from %I.slack_sync_channels channels + where channels.channel_id = $1 + and channels.is_syncable + and not channels.is_private + )', + _schema + ) + into public_syncable + using _channel_id; + return coalesce(public_syncable, false); +end +$$; + +revoke all on function centaur_slack_channel_is_public_syncable(name, text) + from public; +grant execute on function centaur_slack_channel_is_public_syncable(name, text) + to centaur_slack_reader; + +drop policy if exists centaur_context_docs_reader_select + on company_context_documents; +create policy centaur_context_docs_reader_select + on company_context_documents + for select + to centaur_slack_reader + using ( + source <> 'slack' + or metadata ->> 'channel_id' = centaur_current_slack_channel_id() + or centaur_slack_channel_is_public_syncable( + current_schema(), + metadata ->> 'channel_id' + ) + ); + +-- The old helper treated a C-prefixed id as public and was executable by +-- PUBLIC. Its only policy dependency was replaced immediately above. +drop function if exists centaur_slack_channel_is_syncable(name, text); diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0040_granola_sync_tables.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0040_granola_sync_tables.sql new file mode 100644 index 000000000..20284fba3 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0040_granola_sync_tables.sql @@ -0,0 +1,277 @@ +create extension if not exists pg_search; + +create table if not exists granola_sync_runs ( + run_id text primary key, + workflow_run_id text, + mode text not null default 'incremental', + status text not null, + scopes_requested jsonb not null default '[]'::jsonb, + scopes_synced jsonb not null default '[]'::jsonb, + scopes_failed jsonb not null default '[]'::jsonb, + notes_seen integer not null default 0, + notes_upserted integer not null default 0, + transcripts_seen integer not null default 0, + transcripts_upserted integer not null default 0, + started_at timestamptz not null default now(), + finished_at timestamptz, + error_text text not null default '', + metadata jsonb not null default '{}'::jsonb +); + +create index if not exists idx_granola_sync_runs_started + on granola_sync_runs (started_at desc); + +create table if not exists granola_sync_notes ( + note_id text primary key, + title text not null default '', + owner_id text not null default '', + owner_email text not null default '', + owner_name text not null default '', + attendees jsonb not null default '[]'::jsonb, + access_emails text[] not null default array[]::text[], + calendar_event jsonb not null default '{}'::jsonb, + summary_markdown text not null default '', + summary_text text not null default '', + transcript_text text not null default '', + transcript_payload jsonb not null default '[]'::jsonb, + url text not null default '', + content_text text not null default '', + content_hash text not null default '', + source_created_at timestamptz, + source_updated_at timestamptz, + raw_payload jsonb not null default '{}'::jsonb, + source_run_id text references granola_sync_runs(run_id) on delete set null, + first_seen_at timestamptz not null default now(), + last_seen_at timestamptz not null default now(), + last_error text not null default '', + updated_at timestamptz not null default now() +); + +create index if not exists idx_granola_sync_notes_source_updated + on granola_sync_notes (source_updated_at desc); + +create index if not exists idx_granola_sync_notes_owner + on granola_sync_notes (owner_email, source_created_at desc); + +create index if not exists idx_granola_sync_notes_access_emails + on granola_sync_notes using gin (access_emails); + +create index if not exists idx_granola_sync_notes_text + on granola_sync_notes + using gin (to_tsvector('english', coalesce(content_text, ''))); + +create table if not exists granola_context_documents ( + document_id text primary key, + note_id text not null references granola_sync_notes(note_id) on delete cascade, + title text not null default '', + body text not null default '', + url text not null default '', + owner_id text not null default '', + owner_email text not null default '', + owner_name text not null default '', + access_emails text[] not null default array[]::text[], + attendee_labels text[] not null default array[]::text[], + occurred_at timestamptz, + source_updated_at timestamptz, + content_hash text not null default '', + metadata jsonb not null default '{}'::jsonb, + created_at timestamptz not null default now(), + updated_at timestamptz not null default now(), + unique (note_id), + check (document_id <> ''), + check (note_id <> '') +); + +create index if not exists idx_granola_context_documents_note_time + on granola_context_documents (note_id, occurred_at desc); + +create index if not exists idx_granola_context_documents_owner_time + on granola_context_documents (owner_email, occurred_at desc); + +create index if not exists idx_granola_context_documents_access_emails + on granola_context_documents using gin (access_emails); + +create index if not exists idx_granola_context_documents_metadata + on granola_context_documents using gin (metadata); + +drop index if exists idx_granola_context_documents_bm25; + +create index idx_granola_context_documents_bm25 + on granola_context_documents + using bm25 ( + document_id, + note_id, + title, + body, + url, + owner_id, + owner_email, + owner_name, + occurred_at, + source_updated_at, + metadata + ) + with ( + key_field = 'document_id', + text_fields = '{ + "document_id": { + "tokenizer": {"type": "keyword"} + }, + "note_id": { + "tokenizer": {"type": "keyword"} + }, + "owner_id": { + "tokenizer": {"type": "keyword"} + }, + "owner_email": { + "tokenizer": {"type": "keyword"} + } + }' + ); + +create table if not exists granola_sync_checkpoints ( + scope_id text primary key, + watermark_time timestamptz, + last_run_id text references granola_sync_runs(run_id) on delete set null, + last_success_at timestamptz, + last_error text not null default '', + created_at timestamptz not null default now(), + updated_at timestamptz not null default now() +); + +do $$ +declare + role_name text; +begin + foreach role_name in array array[ + 'centaur_slack_reader', + 'centaur_slack_admin', + 'centaur_readonly' + ] loop + if exists (select 1 from pg_roles where rolname = role_name) then + execute format( + 'grant select on %s to %I', + 'granola_sync_runs, granola_sync_notes, granola_context_documents, granola_sync_checkpoints', + role_name + ); + end if; + end loop; +end $$; + +alter table granola_sync_runs enable row level security; +alter table granola_sync_notes enable row level security; +alter table granola_context_documents enable row level security; +alter table granola_sync_checkpoints enable row level security; + +create or replace function centaur_current_slack_user_email() +returns text +language sql +stable +security definer +set search_path = public +as $$ + select coalesce( + lower(nullif(current_setting('centaur.user_email', true), '')), + ( + select lower(nullif(coalesce( + users.raw_payload #>> '{profile,email}', + users.raw_payload ->> 'email' + ), '')) + from slack_sync_users users + where users.team_id = centaur_current_slack_team_id() + and users.user_id = centaur_current_slack_user_id() + limit 1 + ) + ) +$$; + +create or replace function centaur_granola_current_user_can_read( + p_access_emails text[] +) +returns boolean +language sql +stable +as $$ + select coalesce( + centaur_current_slack_user_email() = any(coalesce(p_access_emails, array[]::text[])), + false + ) +$$; + +do $$ +declare + role_name text; +begin + foreach role_name in array array[ + 'centaur_slack_reader', + 'centaur_slack_admin', + 'centaur_readonly' + ] loop + if exists (select 1 from pg_roles where rolname = role_name) then + execute format( + 'grant execute on function centaur_current_slack_user_email() to %I', + role_name + ); + execute format( + 'grant execute on function centaur_granola_current_user_can_read(text[]) to %I', + role_name + ); + end if; + end loop; +end $$; + +drop policy if exists centaur_granola_runs_admin_select on granola_sync_runs; +drop policy if exists centaur_granola_runs_reader_select on granola_sync_runs; +create policy centaur_granola_runs_reader_select + on granola_sync_runs for select to centaur_slack_reader + using (false); +drop policy if exists centaur_readonly_granola_sync_runs_select on granola_sync_runs; +create policy centaur_readonly_granola_sync_runs_select + on granola_sync_runs for select to centaur_readonly using (false); + +drop policy if exists centaur_granola_notes_admin_select on granola_sync_notes; +drop policy if exists centaur_granola_notes_reader_select on granola_sync_notes; +create policy centaur_granola_notes_reader_select + on granola_sync_notes for select to centaur_slack_reader + using (centaur_granola_current_user_can_read(access_emails)); +drop policy if exists centaur_readonly_granola_sync_notes_select on granola_sync_notes; +create policy centaur_readonly_granola_sync_notes_select + on granola_sync_notes for select to centaur_readonly using (false); + +drop policy if exists centaur_granola_context_documents_admin_select + on granola_context_documents; +drop policy if exists centaur_granola_context_documents_reader_select + on granola_context_documents; +create policy centaur_granola_context_documents_reader_select + on granola_context_documents for select to centaur_slack_reader + using (centaur_granola_current_user_can_read(access_emails)); +drop policy if exists centaur_readonly_granola_context_documents_select + on granola_context_documents; +create policy centaur_readonly_granola_context_documents_select + on granola_context_documents for select to centaur_readonly using (false); + +drop policy if exists centaur_granola_checkpoints_admin_select + on granola_sync_checkpoints; +drop policy if exists centaur_granola_checkpoints_reader_select + on granola_sync_checkpoints; +create policy centaur_granola_checkpoints_reader_select + on granola_sync_checkpoints for select to centaur_slack_reader + using (false); +drop policy if exists centaur_readonly_granola_sync_checkpoints_select + on granola_sync_checkpoints; +create policy centaur_readonly_granola_sync_checkpoints_select + on granola_sync_checkpoints for select to centaur_readonly using (false); + +do $$ +begin + if exists (select 1 from pg_roles where rolname = 'centaur_slack_admin') then + create policy centaur_granola_runs_admin_select + on granola_sync_runs for select to centaur_slack_admin using (true); + create policy centaur_granola_notes_admin_select + on granola_sync_notes for select to centaur_slack_admin using (true); + create policy centaur_granola_context_documents_admin_select + on granola_context_documents for select to centaur_slack_admin using (true); + create policy centaur_granola_checkpoints_admin_select + on granola_sync_checkpoints for select to centaur_slack_admin using (true); + end if; +end $$; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0041_attio_sync_tables.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0041_attio_sync_tables.sql new file mode 100644 index 000000000..442a374fa --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0041_attio_sync_tables.sql @@ -0,0 +1,131 @@ +create table if not exists attio_sync_runs ( + run_id text primary key, + workflow_run_id text, + mode text not null default 'incremental', + status text not null, + scopes_requested jsonb not null default '[]'::jsonb, + scopes_synced jsonb not null default '[]'::jsonb, + scopes_failed jsonb not null default '[]'::jsonb, + meetings_seen integer not null default 0, + meetings_upserted integer not null default 0, + call_recordings_seen integer not null default 0, + transcripts_upserted integer not null default 0, + started_at timestamptz not null default now(), + finished_at timestamptz, + error_text text not null default '', + metadata jsonb not null default '{}'::jsonb +); + +create index if not exists idx_attio_sync_runs_started + on attio_sync_runs (started_at desc); + +create table if not exists attio_sync_meetings ( + meeting_id text primary key, + title text not null default '', + description text not null default '', + url text not null default '', + linked_records jsonb not null default '[]'::jsonb, + participants jsonb not null default '[]'::jsonb, + organizer_id text not null default '', + organizer_name text not null default '', + organizer_email text not null default '', + call_recording_ids jsonb not null default '[]'::jsonb, + transcript_text text not null default '', + transcript_payload jsonb not null default '[]'::jsonb, + content_text text not null default '', + content_hash text not null default '', + started_at timestamptz, + ended_at timestamptz, + source_created_at timestamptz, + source_updated_at timestamptz, + raw_payload jsonb not null default '{}'::jsonb, + source_run_id text references attio_sync_runs(run_id) on delete set null, + first_seen_at timestamptz not null default now(), + last_seen_at timestamptz not null default now(), + last_error text not null default '', + updated_at timestamptz not null default now() +); + +create index if not exists idx_attio_sync_meetings_source_updated + on attio_sync_meetings (source_updated_at desc); + +create index if not exists idx_attio_sync_meetings_time + on attio_sync_meetings (started_at desc); + +create index if not exists idx_attio_sync_meetings_text + on attio_sync_meetings + using gin (to_tsvector('english', coalesce(content_text, ''))); + +create table if not exists attio_sync_checkpoints ( + scope_id text primary key, + watermark_time timestamptz, + last_run_id text references attio_sync_runs(run_id) on delete set null, + last_success_at timestamptz, + last_error text not null default '', + created_at timestamptz not null default now(), + updated_at timestamptz not null default now() +); + +do $$ +declare + role_name text; +begin + foreach role_name in array array[ + 'centaur_slack_reader', + 'centaur_slack_admin', + 'centaur_readonly' + ] loop + if exists (select 1 from pg_roles where rolname = role_name) then + execute format( + 'grant select on %s to %I', + 'attio_sync_runs, attio_sync_meetings, attio_sync_checkpoints', + role_name + ); + end if; + end loop; +end $$; + +alter table attio_sync_runs enable row level security; +alter table attio_sync_meetings enable row level security; +alter table attio_sync_checkpoints enable row level security; + +drop policy if exists centaur_attio_runs_admin_select on attio_sync_runs; +drop policy if exists centaur_attio_runs_reader_select on attio_sync_runs; +create policy centaur_attio_runs_reader_select + on attio_sync_runs for select to centaur_slack_reader + using (false); +drop policy if exists centaur_readonly_attio_sync_runs_select on attio_sync_runs; +create policy centaur_readonly_attio_sync_runs_select + on attio_sync_runs for select to centaur_readonly using (true); + +drop policy if exists centaur_attio_meetings_admin_select on attio_sync_meetings; +drop policy if exists centaur_attio_meetings_reader_select on attio_sync_meetings; +create policy centaur_attio_meetings_reader_select + on attio_sync_meetings for select to centaur_slack_reader + using (false); +drop policy if exists centaur_readonly_attio_sync_meetings_select + on attio_sync_meetings; +create policy centaur_readonly_attio_sync_meetings_select + on attio_sync_meetings for select to centaur_readonly using (true); + +drop policy if exists centaur_attio_checkpoints_admin_select on attio_sync_checkpoints; +drop policy if exists centaur_attio_checkpoints_reader_select on attio_sync_checkpoints; +create policy centaur_attio_checkpoints_reader_select + on attio_sync_checkpoints for select to centaur_slack_reader + using (false); +drop policy if exists centaur_readonly_attio_sync_checkpoints_select + on attio_sync_checkpoints; +create policy centaur_readonly_attio_sync_checkpoints_select + on attio_sync_checkpoints for select to centaur_readonly using (true); + +do $$ +begin + if exists (select 1 from pg_roles where rolname = 'centaur_slack_admin') then + create policy centaur_attio_runs_admin_select + on attio_sync_runs for select to centaur_slack_admin using (true); + create policy centaur_attio_meetings_admin_select + on attio_sync_meetings for select to centaur_slack_admin using (true); + create policy centaur_attio_checkpoints_admin_select + on attio_sync_checkpoints for select to centaur_slack_admin using (true); + end if; +end $$; diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0042_centaur_readonly_slack_dm_rls.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0042_centaur_readonly_slack_dm_rls.sql new file mode 100644 index 000000000..4d0cf6dde --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0042_centaur_readonly_slack_dm_rls.sql @@ -0,0 +1,139 @@ +-- Keep centaur_readonly useful for public channel context while allowing a +-- principal that carries Slack identity settings to see only its own DMs. + +drop policy if exists centaur_readonly_slack_dm_sync_conversations_select + on slack_dm_sync_conversations; +create policy centaur_readonly_slack_dm_sync_conversations_select + on slack_dm_sync_conversations + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_sync_conversations.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_sync_conversations.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); + +drop policy if exists centaur_readonly_slack_dm_sync_conversation_members_select + on slack_dm_sync_conversation_members; +create policy centaur_readonly_slack_dm_sync_conversation_members_select + on slack_dm_sync_conversation_members + for select + to centaur_readonly + using ( + home_team_id = centaur_current_slack_team_id() + and user_id = centaur_current_slack_user_id() + and is_current_member + ); + +drop policy if exists centaur_readonly_slack_dm_sync_messages_select + on slack_dm_sync_messages; +create policy centaur_readonly_slack_dm_sync_messages_select + on slack_dm_sync_messages + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_sync_messages.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_sync_messages.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); + +drop policy if exists centaur_readonly_slack_dm_sync_message_attachments_select + on slack_dm_sync_message_attachments; +create policy centaur_readonly_slack_dm_sync_message_attachments_select + on slack_dm_sync_message_attachments + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_sync_message_attachments.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_sync_message_attachments.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); + +drop policy if exists centaur_readonly_slack_dm_sync_checkpoints_select + on slack_dm_sync_checkpoints; +create policy centaur_readonly_slack_dm_sync_checkpoints_select + on slack_dm_sync_checkpoints + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_sync_checkpoints.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_sync_checkpoints.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); + +-- Operational rows never belong in user-visible company context. +drop policy if exists centaur_readonly_slack_dm_sync_runs_select + on slack_dm_sync_runs; +create policy centaur_readonly_slack_dm_sync_runs_select + on slack_dm_sync_runs + for select + to centaur_readonly + using (false); + +drop policy if exists centaur_readonly_slack_dm_sync_backfill_jobs_select + on slack_dm_sync_backfill_jobs; +create policy centaur_readonly_slack_dm_sync_backfill_jobs_select + on slack_dm_sync_backfill_jobs + for select + to centaur_readonly + using (false); + +drop policy if exists centaur_readonly_slack_dm_context_documents_select + on slack_dm_context_documents; +create policy centaur_readonly_slack_dm_context_documents_select + on slack_dm_context_documents + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_context_documents.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_context_documents.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); + +drop policy if exists centaur_readonly_slack_dm_conversation_context_documents_select + on slack_dm_conversation_context_documents; +create policy centaur_readonly_slack_dm_conversation_context_documents_select + on slack_dm_conversation_context_documents + for select + to centaur_readonly + using ( + exists ( + select 1 + from slack_dm_sync_conversation_members members + where members.home_team_id = slack_dm_conversation_context_documents.home_team_id + and members.home_team_id = centaur_current_slack_team_id() + and members.conversation_id = slack_dm_conversation_context_documents.conversation_id + and members.user_id = centaur_current_slack_user_id() + and members.is_current_member + ) + ); diff --git a/services/api-rs/crates/centaur-session-sqlx/migrations/0043_session_sandbox_content_revision.sql b/services/api-rs/crates/centaur-session-sqlx/migrations/0043_session_sandbox_content_revision.sql new file mode 100644 index 000000000..74bf6f4d0 --- /dev/null +++ b/services/api-rs/crates/centaur-session-sqlx/migrations/0043_session_sandbox_content_revision.sql @@ -0,0 +1,21 @@ +alter table sessions + add column if not exists sandbox_content_revision text; + +comment on column sessions.sandbox_content_revision is + 'Assignment-bound digest of the immutable deployment boot-content generation and sandbox ID; NULL on legacy assignments.'; + +create or replace view centaur_readonly_sessions as +select + thread_key, + sandbox_id, + harness_type, + harness_thread_id, + persona_id, + status, + metadata ->> 'source' as source, + metadata ->> 'platform' as platform, + metadata ->> 'thread_id' as external_thread_id, + created_at, + updated_at, + sandbox_content_revision +from sessions; diff --git a/services/api-rs/crates/centaur-session-sqlx/src/lib.rs b/services/api-rs/crates/centaur-session-sqlx/src/lib.rs index fc3b8daac..04ac1f97f 100644 --- a/services/api-rs/crates/centaur-session-sqlx/src/lib.rs +++ b/services/api-rs/crates/centaur-session-sqlx/src/lib.rs @@ -1,10 +1,11 @@ //! SQLx-backed session repository. -use std::str::FromStr; +use std::{str::FromStr, time::Duration}; use centaur_session_core::{ - ExecutionStatus, HarnessType, SandboxCapabilities, Session, SessionEvent, SessionExecution, - SessionMessage, SessionMessageInput, SessionStatus, ThreadKey, empty_object, + ExecutionStatus, HarnessType, MessageRole, SandboxCapabilities, SandboxRepoCacheAccess, + Session, SessionEvent, SessionExecution, SessionMessage, SessionMessageInput, SessionStatus, + ThreadKey, empty_object, }; use serde::Deserialize; use serde_json::Value; @@ -13,7 +14,7 @@ use sqlx::{ postgres::{PgListener, PgPoolOptions}, }; use thiserror::Error; -use time::OffsetDateTime; +use time::{Duration as TimeDuration, OffsetDateTime}; use uuid::Uuid; // The API binary embeds these migrations at compile time. @@ -37,17 +38,64 @@ pub struct ClaimExecutionResult { pub claimed: bool, } +/// An active execution whose stdout-owner lease was released by +/// [`PgSessionStore::release_stdout_owned_executions`]. +#[derive(Clone, Debug)] +pub struct ReleasedExecution { + pub execution_id: String, + pub thread_key: ThreadKey, +} + +/// An active execution together with its stdout-owner lease state, as +/// returned by [`PgSessionStore::list_active_executions_with_ownership`]. +/// The lease snapshot is advisory — only the conditional +/// `claim_expired_stdout_owner` update decides ownership — but it lets an +/// adoption scan skip executions with a live owner without touching the +/// session row or the sandbox backend. +#[derive(Clone, Debug)] +pub struct ActiveExecutionOwnership { + pub execution: SessionExecution, + pub stdout_owner_id: Option, + /// True when a stdout-owner lease exists and has not expired yet. + pub stdout_owner_lease_active: bool, +} + +/// Outcome of the transactional database fence used before stopping a +/// session sandbox. Locking the session row serializes release with execution +/// creation, while the sandbox snapshot prevents an old release request from +/// clearing a newly assigned sandbox. +#[derive(Clone, Debug)] +pub enum ReleaseSessionResult { + Released { + session: Box, + cancelled_execution: Option, + }, + ActiveExecution(SessionExecution), + SandboxMismatch { + current_sandbox_id: Option, + }, +} + #[derive(Clone, Debug, Eq, PartialEq)] pub struct IdleSandboxCandidate { pub thread_key: ThreadKey, pub sandbox_id: String, pub execution_id: String, + pub idle_timeout: Duration, } #[derive(Clone, Debug, Eq, PartialEq)] -pub struct WorkflowOwnedSandbox { +pub struct SandboxCapacityCandidate { pub thread_key: ThreadKey, pub sandbox_id: String, + pub latest_execution_id: Option, + pub last_active_at: OffsetDateTime, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct WorkflowOwnedSandbox { + pub thread_key: ThreadKey, + pub sandbox_id: Option, } #[derive(Clone)] @@ -123,10 +171,70 @@ impl PgSessionStore { Ok(session) } + /// Create a child session already bound to an authenticated principal, or + /// load it only when the existing binding belongs to that same principal. + /// The principal is written by the insert itself, so a competing creator + /// cannot observe and claim an unbound row. + pub async fn create_or_get_session_for_principal( + &self, + thread_key: &ThreadKey, + harness_type: &HarnessType, + persona_id: Option<&str>, + metadata: Value, + iron_control_principal: &str, + ) -> Result { + sqlx::query( + r#" + insert into sessions ( + thread_key, + harness_type, + persona_id, + status, + metadata, + iron_control_principal + ) + values ($1, $2, $3, $4, $5, $6) + on conflict (thread_key) do nothing + "#, + ) + .bind(thread_key.as_str()) + .bind(harness_type.as_ref()) + .bind(persona_id) + .bind(SessionStatus::Idle.as_ref()) + .bind(metadata) + .bind(iron_control_principal) + .execute(&self.pool) + .await?; + + let session = self.get_session(thread_key).await?; + if session.iron_control_principal.as_deref() != Some(iron_control_principal) { + return Err(SessionStoreError::PrincipalConflict { + thread_key: thread_key.as_str().to_owned(), + existing: session.iron_control_principal, + requested: iron_control_principal.to_owned(), + }); + } + if session.harness_type != *harness_type { + return Err(SessionStoreError::HarnessConflict { + thread_key: thread_key.as_str().to_owned(), + existing: session.harness_type.to_string(), + requested: harness_type.as_ref().to_owned(), + }); + } + if session.persona_id.as_deref() != persona_id { + return Err(SessionStoreError::PersonaConflict { + thread_key: thread_key.as_str().to_owned(), + existing: session.persona_id, + requested: persona_id.map(str::to_owned), + }); + } + Ok(session) + } + pub async fn get_session(&self, thread_key: &ThreadKey) -> Result { let row = sqlx::query_as::<_, SessionRow>( r#" - select thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + select thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at from sessions where thread_key = $1 "#, @@ -141,6 +249,25 @@ impl PgSessionStore { row.try_into() } + pub async fn get_session_title( + &self, + thread_key: &ThreadKey, + ) -> Result, SessionStoreError> { + let title = sqlx::query_scalar::<_, Option>( + r#" + select title + from sessions + where thread_key = $1 + "#, + ) + .bind(thread_key.as_str()) + .fetch_optional(&self.pool) + .await? + .flatten(); + + Ok(title) + } + pub async fn append_messages( &self, thread_key: &ThreadKey, @@ -178,6 +305,59 @@ impl PgSessionStore { Ok(message_ids) } + pub async fn title_generation_candidate( + &self, + thread_key: &ThreadKey, + ) -> Result>, SessionStoreError> { + let rows = sqlx::query_scalar::<_, Value>( + r#" + select m.parts + from sessions s + join session_messages m on m.thread_key = s.thread_key + where s.thread_key = $1 and s.title is null + and m.role = $2 + order by m.created_at, m.message_id + "#, + ) + .bind(thread_key.as_str()) + .bind(MessageRole::User.as_ref()) + .fetch_all(&self.pool) + .await?; + + if rows.is_empty() { + return Ok(None); + } + + let parts = rows + .into_iter() + .flat_map(|parts| match parts { + Value::Array(parts) => parts, + other => vec![other], + }) + .collect(); + Ok(Some(parts)) + } + + pub async fn set_session_title_if_empty( + &self, + thread_key: &ThreadKey, + title: &str, + ) -> Result { + let result = sqlx::query( + r#" + update sessions + set title = $2, updated_at = now() + where thread_key = $1 and title is null + "#, + ) + .bind(thread_key.as_str()) + .bind(title) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) + } + pub async fn list_messages( &self, thread_key: &ThreadKey, @@ -204,6 +384,18 @@ impl PgSessionStore { metadata: Value, ) -> Result { let execution_id = prefixed_id("exe"); + let mut tx = self.pool.begin().await?; + let session_exists = + sqlx::query_scalar::<_, i32>("select 1 from sessions where thread_key = $1 for update") + .bind(thread_key.as_str()) + .fetch_optional(&mut *tx) + .await? + .is_some(); + if !session_exists { + return Err(SessionStoreError::NotFound { + thread_key: thread_key.as_str().to_owned(), + }); + } let row = sqlx::query_as::<_, CreateExecutionRow>( r#" insert into session_executions @@ -231,12 +423,124 @@ impl PgSessionStore { .bind(idempotency_key) .bind(ExecutionStatus::Queued.as_ref()) .bind(metadata) - .fetch_one(&self.pool) + .fetch_one(&mut *tx) .await?; + tx.commit().await?; + row.try_into() } + pub async fn release_session_if_sandbox_matches( + &self, + thread_key: &ThreadKey, + expected_sandbox_id: Option<&str>, + cancel_inflight: bool, + cancellation_reason: &str, + ) -> Result { + let mut tx = self.pool.begin().await?; + let locked = sqlx::query_as::<_, SessionRow>( + r#" + select thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at + from sessions + where thread_key = $1 + for update + "#, + ) + .bind(thread_key.as_str()) + .fetch_optional(&mut *tx) + .await? + .ok_or_else(|| SessionStoreError::NotFound { + thread_key: thread_key.as_str().to_owned(), + })?; + + if locked.sandbox_id.as_deref() != expected_sandbox_id { + let current_sandbox_id = locked.sandbox_id; + tx.commit().await?; + return Ok(ReleaseSessionResult::SandboxMismatch { current_sandbox_id }); + } + + let active = sqlx::query_as::<_, SessionExecutionRow>( + r#" + select execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at + from session_executions + where thread_key = $1 and status in ($2, $3) + order by created_at desc, execution_id desc + limit 1 + for update + "#, + ) + .bind(thread_key.as_str()) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_optional(&mut *tx) + .await?; + + if let Some(active) = active.as_ref() + && !cancel_inflight + { + let execution = active.clone().try_into()?; + tx.commit().await?; + return Ok(ReleaseSessionResult::ActiveExecution(execution)); + } + + let cancelled_execution = if let Some(active) = active { + let row = sqlx::query_as::<_, SessionExecutionRow>( + r#" + update session_executions + set status = $2, + error = $3, + completed_at = coalesce(completed_at, now()), + stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where execution_id = $1 and status in ($4, $5) + returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at + "#, + ) + .bind(active.execution_id) + .bind(ExecutionStatus::Cancelled.as_ref()) + .bind(cancellation_reason) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_optional(&mut *tx) + .await?; + row.map(TryInto::try_into).transpose()? + } else { + None + }; + + let row = sqlx::query_as::<_, SessionRow>( + r#" + update sessions + set sandbox_id = null, + sandbox_content_revision = null, + sandbox_repo_cache_enabled = null, + sandbox_repo_cache_access = null, + sandbox_observability_enabled = null, + sandbox_api_server_enabled = null, + sandbox_last_active_at = null, + harness_thread_id = null, + status = $3, + updated_at = now() + where thread_key = $1 + and sandbox_id is not distinct from $2 + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at + "#, + ) + .bind(thread_key.as_str()) + .bind(expected_sandbox_id) + .bind(SessionStatus::Idle.as_ref()) + .fetch_one(&mut *tx) + .await?; + let session = row.try_into()?; + tx.commit().await?; + Ok(ReleaseSessionResult::Released { + session: Box::new(session), + cancelled_execution, + }) + } + pub async fn active_execution_for_thread( &self, thread_key: &ThreadKey, @@ -278,6 +582,35 @@ impl PgSessionStore { rows.into_iter().map(TryInto::try_into).collect() } + pub async fn list_active_executions_with_ownership( + &self, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_as::<_, ActiveExecutionOwnershipRow>( + r#" + select execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at, + stdout_owner_id, + coalesce(stdout_owner_lease_expires_at > now(), false) as stdout_owner_lease_active + from session_executions + where status in ($1, $2) + order by created_at, execution_id + "#, + ) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_all(&self.pool) + .await?; + + rows.into_iter() + .map(|row| { + Ok(ActiveExecutionOwnership { + execution: row.execution.try_into()?, + stdout_owner_id: row.stdout_owner_id, + stdout_owner_lease_active: row.stdout_owner_lease_active, + }) + }) + .collect() + } + pub async fn latest_execution_for_thread( &self, thread_key: &ThreadKey, @@ -344,6 +677,175 @@ impl PgSessionStore { }) } + pub async fn claim_stdout_owner( + &self, + execution_id: &str, + owner_id: &str, + lease: Duration, + ) -> Result { + let lease_expires_at = stdout_lease_expires_at(lease); + let result = sqlx::query( + r#" + update session_executions + set stdout_owner_id = $2, + stdout_owner_lease_expires_at = $3, + updated_at = now() + where execution_id = $1 + and status in ($4, $5) + and ( + stdout_owner_id is null + or stdout_owner_id = $2 + or stdout_owner_lease_expires_at < now() + ) + "#, + ) + .bind(execution_id) + .bind(owner_id) + .bind(lease_expires_at) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) + } + + pub async fn claim_expired_stdout_owner( + &self, + execution_id: &str, + owner_id: &str, + lease: Duration, + ) -> Result { + let lease_expires_at = stdout_lease_expires_at(lease); + let result = sqlx::query( + r#" + update session_executions + set stdout_owner_id = $2, + stdout_owner_lease_expires_at = $3, + updated_at = now() + where execution_id = $1 + and status in ($4, $5) + and ( + stdout_owner_id is null + or stdout_owner_lease_expires_at < now() + ) + "#, + ) + .bind(execution_id) + .bind(owner_id) + .bind(lease_expires_at) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) + } + + pub async fn renew_stdout_owner( + &self, + execution_id: &str, + owner_id: &str, + lease: Duration, + ) -> Result { + let lease_expires_at = stdout_lease_expires_at(lease); + let result = sqlx::query( + r#" + update session_executions + set stdout_owner_lease_expires_at = $3, + updated_at = now() + where execution_id = $1 + and stdout_owner_id = $2 + and status in ($4, $5) + "#, + ) + .bind(execution_id) + .bind(owner_id) + .bind(lease_expires_at) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) + } + + pub async fn release_stdout_owner( + &self, + execution_id: &str, + owner_id: &str, + ) -> Result { + let result = sqlx::query( + r#" + update session_executions + set stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where execution_id = $1 and stdout_owner_id = $2 + "#, + ) + .bind(execution_id) + .bind(owner_id) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) + } + + pub async fn count_executions_with_stdout_owner( + &self, + owner_id: &str, + ) -> Result { + let count = sqlx::query_scalar::<_, i64>( + r#" + select count(*) + from session_executions + where stdout_owner_id = $1 and status in ($2, $3) + "#, + ) + .bind(owner_id) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_one(&self.pool) + .await?; + + Ok(u64::try_from(count).unwrap_or_default()) + } + + /// Releases every active stdout-owner lease held by `owner_id` in one + /// statement, returning the affected executions. Used by a clean + /// control-plane shutdown so a peer's adoption scan can claim the + /// executions immediately instead of waiting out the lease TTL. + pub async fn release_stdout_owned_executions( + &self, + owner_id: &str, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_as::<_, (String, String)>( + r#" + update session_executions + set stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where stdout_owner_id = $1 and status in ($2, $3) + returning execution_id, thread_key + "#, + ) + .bind(owner_id) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_all(&self.pool) + .await?; + + rows.into_iter() + .map(|(execution_id, thread_key)| { + Ok(ReleasedExecution { + execution_id, + thread_key: parse_persisted(thread_key)?, + }) + }) + .collect() + } + pub async fn complete_execution( &self, execution_id: &str, @@ -393,40 +895,112 @@ impl PgSessionStore { row.try_into().map(Some) } - pub async fn fail_execution( + pub async fn complete_execution_if_active_and_stdout_owner( + &self, + execution_id: &str, + owner_id: &str, + ) -> Result, SessionStoreError> { + let row = sqlx::query_as::<_, SessionExecutionRow>( + r#" + update session_executions + set status = $2, + completed_at = coalesce(completed_at, now()), + stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where execution_id = $1 + and status in ($3, $4) + and stdout_owner_id = $5 + returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at + "#, + ) + .bind(execution_id) + .bind(ExecutionStatus::Completed.as_ref()) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .bind(owner_id) + .fetch_optional(&self.pool) + .await?; + + let Some(row) = row else { + return Ok(None); + }; + self.set_session_status(&row.thread_key, SessionStatus::Idle) + .await?; + row.try_into().map(Some) + } + + pub async fn fail_execution( + &self, + execution_id: &str, + error: &str, + ) -> Result { + let row = sqlx::query_as::<_, SessionExecutionRow>( + r#" + update session_executions + set status = $2, error = $3, completed_at = coalesce(completed_at, now()), updated_at = now() + where execution_id = $1 + returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at + "#, + ) + .bind(execution_id) + .bind(ExecutionStatus::Failed.as_ref()) + .bind(error) + .fetch_one(&self.pool) + .await?; + + self.set_session_status(&row.thread_key, SessionStatus::Failed) + .await?; + row.try_into() + } + + pub async fn fail_execution_if_active( &self, execution_id: &str, error: &str, - ) -> Result { + ) -> Result, SessionStoreError> { let row = sqlx::query_as::<_, SessionExecutionRow>( r#" update session_executions set status = $2, error = $3, completed_at = coalesce(completed_at, now()), updated_at = now() - where execution_id = $1 + where execution_id = $1 and status in ($4, $5) returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at "#, ) .bind(execution_id) .bind(ExecutionStatus::Failed.as_ref()) .bind(error) - .fetch_one(&self.pool) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .fetch_optional(&self.pool) .await?; + let Some(row) = row else { + return Ok(None); + }; self.set_session_status(&row.thread_key, SessionStatus::Failed) .await?; - row.try_into() + row.try_into().map(Some) } - pub async fn fail_execution_if_active( + pub async fn fail_execution_if_active_and_stdout_owner( &self, execution_id: &str, + owner_id: &str, error: &str, ) -> Result, SessionStoreError> { let row = sqlx::query_as::<_, SessionExecutionRow>( r#" update session_executions - set status = $2, error = $3, completed_at = coalesce(completed_at, now()), updated_at = now() - where execution_id = $1 and status in ($4, $5) + set status = $2, + error = $3, + completed_at = coalesce(completed_at, now()), + stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where execution_id = $1 + and status in ($4, $5) + and stdout_owner_id = $6 returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at "#, ) @@ -435,6 +1009,7 @@ impl PgSessionStore { .bind(error) .bind(ExecutionStatus::Queued.as_ref()) .bind(ExecutionStatus::Running.as_ref()) + .bind(owner_id) .fetch_optional(&self.pool) .await?; @@ -446,24 +1021,33 @@ impl PgSessionStore { row.try_into().map(Some) } - pub async fn cancel_execution_if_active( + pub async fn cancel_execution_if_active_and_stdout_owner( &self, execution_id: &str, - error: &str, + owner_id: &str, + reason: &str, ) -> Result, SessionStoreError> { let row = sqlx::query_as::<_, SessionExecutionRow>( r#" update session_executions - set status = $2, error = $3, completed_at = coalesce(completed_at, now()), updated_at = now() - where execution_id = $1 and status in ($4, $5) + set status = $2, + error = $3, + completed_at = coalesce(completed_at, now()), + stdout_owner_id = null, + stdout_owner_lease_expires_at = null, + updated_at = now() + where execution_id = $1 + and status in ($4, $5) + and stdout_owner_id = $6 returning execution_id, idempotency_key, thread_key, status, metadata, error, created_at, updated_at, started_at, completed_at "#, ) .bind(execution_id) .bind(ExecutionStatus::Cancelled.as_ref()) - .bind(error) + .bind(reason) .bind(ExecutionStatus::Queued.as_ref()) .bind(ExecutionStatus::Running.as_ref()) + .bind(owner_id) .fetch_optional(&self.pool) .await?; @@ -499,6 +1083,75 @@ impl PgSessionStore { row.try_into() } + pub async fn append_event_if_stdout_owner( + &self, + thread_key: &ThreadKey, + execution_id: &str, + owner_id: &str, + lease: Duration, + event_type: &str, + payload: Value, + ) -> Result, SessionStoreError> { + let lease_expires_at = stdout_lease_expires_at(lease); + let mut tx = self.pool.begin().await?; + // Match the canonical release transaction's session -> execution lock + // order. Without this key-share lock, output append could lock the + // execution first and then block on the session FK while release held + // the session and waited for that execution, producing a deadlock. + let session_exists = sqlx::query_scalar::<_, i32>( + "select 1 from sessions where thread_key = $1 for key share", + ) + .bind(thread_key.as_str()) + .fetch_optional(&mut *tx) + .await? + .is_some(); + if !session_exists { + tx.commit().await?; + return Ok(None); + } + let result = sqlx::query( + r#" + update session_executions + set stdout_owner_lease_expires_at = $3, + updated_at = now() + where execution_id = $1 + and stdout_owner_id = $2 + and status in ($4, $5) + and thread_key = $6 + "#, + ) + .bind(execution_id) + .bind(owner_id) + .bind(lease_expires_at) + .bind(ExecutionStatus::Queued.as_ref()) + .bind(ExecutionStatus::Running.as_ref()) + .bind(thread_key.as_str()) + .execute(&mut *tx) + .await?; + + if result.rows_affected() == 0 { + tx.commit().await?; + return Ok(None); + } + + let row = sqlx::query_as::<_, SessionEventRow>( + r#" + insert into session_events (thread_key, execution_id, event_type, payload) + values ($1, $2, $3, $4) + returning event_id, thread_key, execution_id, event_type, payload, created_at + "#, + ) + .bind(thread_key.as_str()) + .bind(execution_id) + .bind(event_type) + .bind(payload) + .fetch_one(&mut *tx) + .await?; + + tx.commit().await?; + row.try_into().map(Some) + } + pub async fn list_events_after( &self, thread_key: &ThreadKey, @@ -562,7 +1215,7 @@ impl PgSessionStore { select sandbox_id from session_warm_sandboxes - where status in ('ready', 'claimed') + where status in ('ready', 'claimed', 'evicting') "#, ) .fetch_all(&self.pool) @@ -573,7 +1226,7 @@ impl PgSessionStore { pub async fn list_idle_sandbox_candidates( &self, - idle_backstop: std::time::Duration, + idle_backstop: Duration, ) -> Result, SessionStoreError> { let rows = sqlx::query_as::<_, IdleSandboxCandidateRow>( r#" @@ -582,20 +1235,22 @@ impl PgSessionStore { execution_id, thread_key, status, - completed_at + completed_at, + metadata from session_executions order by thread_key, created_at desc, execution_id desc ) select s.thread_key, s.sandbox_id as sandbox_id, - latest.execution_id + latest.execution_id, + latest.completed_at, + latest.metadata from sessions s join latest on latest.thread_key = s.thread_key where s.sandbox_id is not null and latest.status in ('completed', 'failed', 'cancelled') and latest.completed_at is not null - and latest.completed_at <= now() - ($1::float8 * interval '1 second') and not exists ( select 1 from session_executions active @@ -605,7 +1260,81 @@ impl PgSessionStore { order by latest.completed_at, s.thread_key "#, ) - .bind(idle_backstop.as_secs_f64()) + .fetch_all(&self.pool) + .await?; + + let now = OffsetDateTime::now_utc(); + rows.into_iter() + .filter_map(|row| idle_candidate_from_row(row, idle_backstop, now).transpose()) + .collect() + } + + pub async fn list_sandbox_capacity_candidates( + &self, + excluded_thread_key: Option<&ThreadKey>, + hot_idle_grace: std::time::Duration, + limit: i64, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_as::<_, SandboxCapacityCandidateRow>( + r#" + with latest as ( + select distinct on (thread_key) + execution_id, + thread_key, + completed_at + from session_executions + order by thread_key, created_at desc, execution_id desc + ) + select + s.thread_key, + s.sandbox_id as sandbox_id, + latest.execution_id as latest_execution_id, + coalesce( + s.sandbox_last_active_at, + latest.completed_at, + s.updated_at, + s.created_at + ) as last_active_at + from sessions s + left join latest on latest.thread_key = s.thread_key + where s.sandbox_id is not null + and ($1::text is null or s.thread_key != $1) + and not exists ( + select 1 + from lateral ( + select e.event_type + from session_events e + where e.thread_key = s.thread_key + and e.payload->>'sandbox_id' = s.sandbox_id + and e.event_type in ( + 'session.sandbox_paused', + 'session.sandbox_ready', + 'session.sandbox_resumed' + ) + order by e.created_at desc, e.event_id desc + limit 1 + ) latest_sandbox_event + where latest_sandbox_event.event_type = 'session.sandbox_paused' + ) + and coalesce( + s.sandbox_last_active_at, + latest.completed_at, + s.updated_at, + s.created_at + ) <= now() - ($2::float8 * interval '1 second') + and not exists ( + select 1 + from session_executions active + where active.thread_key = s.thread_key + and active.status in ('queued', 'running') + ) + order by last_active_at, s.thread_key + limit $3 + "#, + ) + .bind(excluded_thread_key.map(ThreadKey::as_str)) + .bind(hot_idle_grace.as_secs_f64()) + .bind(limit) .fetch_all(&self.pool) .await?; @@ -618,10 +1347,9 @@ impl PgSessionStore { ) -> Result, SessionStoreError> { let rows = sqlx::query_as::<_, WorkflowOwnedSandboxRow>( r#" - select thread_key, sandbox_id as sandbox_id + select thread_key, sandbox_id from sessions - where sandbox_id is not null - and metadata->>'workflow_owned_thread' = 'true' + where metadata->>'workflow_owned_thread' = 'true' and metadata->>'workflow_run_id' = $1 order by thread_key "#, @@ -643,11 +1371,18 @@ impl PgSessionStore { update sessions set sandbox_id = $2, + sandbox_content_revision = null, sandbox_repo_cache_enabled = null, + sandbox_repo_cache_access = null, sandbox_observability_enabled = null, + sandbox_api_server_enabled = null, + sandbox_last_active_at = case + when $2::text is null then null + else now() + end, updated_at = now() where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at "#, ) .bind(thread_key.as_str()) @@ -662,6 +1397,7 @@ impl PgSessionStore { &self, thread_key: &ThreadKey, sandbox_id: &str, + content_revision: Option<&str>, capabilities: &SandboxCapabilities, ) -> Result { let row = sqlx::query_as::<_, SessionRow>( @@ -669,23 +1405,214 @@ impl PgSessionStore { update sessions set sandbox_id = $2, - sandbox_repo_cache_enabled = $3, - sandbox_observability_enabled = $4, + sandbox_content_revision = $3, + sandbox_repo_cache_enabled = $4, + sandbox_repo_cache_access = $5, + sandbox_observability_enabled = $6, + sandbox_api_server_enabled = $7, + sandbox_last_active_at = now(), updated_at = now() where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at "#, ) .bind(thread_key.as_str()) .bind(sandbox_id) - .bind(capabilities.repo_cache_enabled) + .bind(content_revision) + .bind(capabilities.repo_cache_enabled()) + .bind(capabilities.repo_cache.as_str()) .bind(capabilities.observability_enabled) + .bind(capabilities.api_server_enabled) .fetch_one(&self.pool) .await?; row.try_into() } + /// Bind a sandbox only while the exact execution that allocated it is + /// still running and the session assignment has not crossed the caller's + /// fence. Lock order intentionally matches release: session first, then + /// execution. A concurrent release therefore either clears/cancels first + /// and this returns `None`, or waits until this assignment commits. + // These explicit fence fields are kept adjacent to the SQL transaction so + // a caller cannot accidentally omit ownership, assignment, revision, or + // capability state while committing a sandbox. + #[allow(clippy::too_many_arguments)] + pub async fn assign_sandbox_to_active_execution( + &self, + thread_key: &ThreadKey, + execution_id: &str, + stdout_owner_id: &str, + expected_sandbox_id: Option<&str>, + sandbox_id: &str, + content_revision: Option<&str>, + capabilities: &SandboxCapabilities, + ) -> Result, SessionStoreError> { + let mut tx = self.pool.begin().await?; + let current_sandbox_id = sqlx::query_scalar::<_, Option>( + r#" + select sandbox_id + from sessions + where thread_key = $1 + for update + "#, + ) + .bind(thread_key.as_str()) + .fetch_optional(&mut *tx) + .await? + .ok_or_else(|| SessionStoreError::NotFound { + thread_key: thread_key.as_str().to_owned(), + })?; + if current_sandbox_id.as_deref() != expected_sandbox_id { + tx.commit().await?; + return Ok(None); + } + + let execution = sqlx::query_as::<_, (String, bool)>( + r#" + select status, + coalesce( + stdout_owner_id = $3 + and stdout_owner_lease_expires_at > now(), + false + ) as owner_active + from session_executions + where execution_id = $1 and thread_key = $2 + for update + "#, + ) + .bind(execution_id) + .bind(thread_key.as_str()) + .bind(stdout_owner_id) + .fetch_optional(&mut *tx) + .await?; + if !matches!( + execution.as_ref(), + Some((status, true)) + if status == ExecutionStatus::Queued.as_ref() + || status == ExecutionStatus::Running.as_ref() + ) { + tx.commit().await?; + return Ok(None); + } + + let row = sqlx::query_as::<_, SessionRow>( + r#" + update sessions + set + sandbox_id = $3, + sandbox_content_revision = $4, + sandbox_repo_cache_enabled = $5, + sandbox_repo_cache_access = $6, + sandbox_observability_enabled = $7, + sandbox_api_server_enabled = $8, + sandbox_last_active_at = now(), + updated_at = now() + where thread_key = $1 + and sandbox_id is not distinct from $2 + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at + "#, + ) + .bind(thread_key.as_str()) + .bind(expected_sandbox_id) + .bind(sandbox_id) + .bind(content_revision) + .bind(capabilities.repo_cache_enabled()) + .bind(capabilities.repo_cache.as_str()) + .bind(capabilities.observability_enabled) + .bind(capabilities.api_server_enabled) + .fetch_optional(&mut *tx) + .await?; + let session = row.map(TryInto::try_into).transpose()?; + tx.commit().await?; + Ok(session) + } + + /// Clear an existing sandbox assignment only while the exact execution and + /// stdout-owner lease that observed it are still active. This is the first + /// phase of capability replacement: clearing before the external stop + /// prevents a stale worker from stopping or clearing a recovered worker's + /// newly assigned sandbox. + pub async fn clear_sandbox_from_active_execution( + &self, + thread_key: &ThreadKey, + execution_id: &str, + stdout_owner_id: &str, + expected_sandbox_id: &str, + ) -> Result, SessionStoreError> { + let mut tx = self.pool.begin().await?; + let current_sandbox_id = sqlx::query_scalar::<_, Option>( + r#" + select sandbox_id + from sessions + where thread_key = $1 + for update + "#, + ) + .bind(thread_key.as_str()) + .fetch_optional(&mut *tx) + .await? + .ok_or_else(|| SessionStoreError::NotFound { + thread_key: thread_key.as_str().to_owned(), + })?; + if current_sandbox_id.as_deref() != Some(expected_sandbox_id) { + tx.commit().await?; + return Ok(None); + } + + let execution = sqlx::query_as::<_, (String, bool)>( + r#" + select status, + coalesce( + stdout_owner_id = $3 + and stdout_owner_lease_expires_at > now(), + false + ) as owner_active + from session_executions + where execution_id = $1 and thread_key = $2 + for update + "#, + ) + .bind(execution_id) + .bind(thread_key.as_str()) + .bind(stdout_owner_id) + .fetch_optional(&mut *tx) + .await?; + if !matches!( + execution.as_ref(), + Some((status, true)) + if status == ExecutionStatus::Queued.as_ref() + || status == ExecutionStatus::Running.as_ref() + ) { + tx.commit().await?; + return Ok(None); + } + + let row = sqlx::query_as::<_, SessionRow>( + r#" + update sessions + set + sandbox_id = null, + sandbox_content_revision = null, + sandbox_repo_cache_enabled = null, + sandbox_repo_cache_access = null, + sandbox_observability_enabled = null, + sandbox_api_server_enabled = null, + sandbox_last_active_at = null, + updated_at = now() + where thread_key = $1 and sandbox_id = $2 + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at + "#, + ) + .bind(thread_key.as_str()) + .bind(expected_sandbox_id) + .fetch_optional(&mut *tx) + .await?; + let session = row.map(TryInto::try_into).transpose()?; + tx.commit().await?; + Ok(session) + } + pub async fn clear_sandbox_id_if_matches( &self, thread_key: &ThreadKey, @@ -696,8 +1623,12 @@ impl PgSessionStore { update sessions set sandbox_id = null, + sandbox_content_revision = null, sandbox_repo_cache_enabled = null, + sandbox_repo_cache_access = null, sandbox_observability_enabled = null, + sandbox_api_server_enabled = null, + sandbox_last_active_at = null, updated_at = now() where thread_key = $1 and sandbox_id = $2 "#, @@ -724,12 +1655,16 @@ impl PgSessionStore { set harness_type = $2, harness_thread_id = null, sandbox_id = null, + sandbox_content_revision = null, sandbox_repo_cache_enabled = null, + sandbox_repo_cache_access = null, sandbox_observability_enabled = null, + sandbox_api_server_enabled = null, + sandbox_last_active_at = null, status = $3, updated_at = now() where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at "#, ) .bind(thread_key.as_str()) @@ -754,7 +1689,7 @@ impl PgSessionStore { update sessions set iron_control_principal = $2, updated_at = now() where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at "#, ) .bind(thread_key.as_str()) @@ -800,6 +1735,20 @@ impl PgSessionStore { Ok(count) } + pub async fn list_ready_warm_sandbox_ids(&self) -> Result, SessionStoreError> { + let sandbox_ids = sqlx::query_scalar::<_, String>( + r#" + select sandbox_id + from session_warm_sandboxes + where status = 'ready' + order by created_at, sandbox_id + "#, + ) + .fetch_all(&self.pool) + .await?; + Ok(sandbox_ids) + } + pub async fn claim_ready_warm_sandbox( &self, workload_key: &str, @@ -807,30 +1756,120 @@ impl PgSessionStore { ) -> Result, SessionStoreError> { let sandbox_id = sqlx::query_scalar::<_, String>( r#" - with candidate as ( + with candidate as ( + select sandbox_id + from session_warm_sandboxes + where workload_key = $1 and status = 'ready' + order by created_at, sandbox_id + for update skip locked + limit 1 + ) + update session_warm_sandboxes warm + set + status = 'claimed', + claimed_thread_key = $2, + claimed_at = now(), + updated_at = now() + from candidate + where warm.sandbox_id = candidate.sandbox_id + returning warm.sandbox_id + "#, + ) + .bind(workload_key) + .bind(thread_key) + .fetch_optional(&self.pool) + .await?; + Ok(sandbox_id) + } + + /// Atomically reserves every unclaimed ready sandbox built for a different + /// workload. The `status = 'ready'` predicate is repeated on the update so + /// a concurrent claimant always wins or loses as one transaction; claimed + /// or otherwise bound sandboxes are never returned for backend eviction. + pub async fn reserve_ready_warm_sandboxes_for_workload_mismatch( + &self, + workload_key: &str, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_scalar::<_, String>( + r#" + with candidates as ( + select warm.sandbox_id + from session_warm_sandboxes warm + where warm.status = 'ready' + and warm.workload_key <> $1 + and not exists ( + select 1 from sessions session + where session.sandbox_id = warm.sandbox_id + ) + order by warm.created_at, warm.sandbox_id + for update skip locked + ) + update session_warm_sandboxes warm + set + status = 'evicting', + updated_at = now() + from candidates + where warm.sandbox_id = candidates.sandbox_id + and warm.status = 'ready' + and not exists ( + select 1 from sessions session + where session.sandbox_id = warm.sandbox_id + ) + returning warm.sandbox_id + "#, + ) + .bind(workload_key) + .fetch_all(&self.pool) + .await?; + Ok(rows) + } + + pub async fn reserve_ready_warm_sandboxes_for_eviction( + &self, + limit: i64, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_scalar::<_, String>( + r#" + with candidates as ( select sandbox_id from session_warm_sandboxes - where workload_key = $1 and status = 'ready' + where status = 'ready' order by created_at, sandbox_id for update skip locked - limit 1 + limit $1 ) update session_warm_sandboxes warm set - status = 'claimed', - claimed_thread_key = $2, - claimed_at = now(), + status = 'evicting', updated_at = now() - from candidate - where warm.sandbox_id = candidate.sandbox_id + from candidates + where warm.sandbox_id = candidates.sandbox_id returning warm.sandbox_id "#, ) - .bind(workload_key) - .bind(thread_key) - .fetch_optional(&self.pool) + .bind(limit) + .fetch_all(&self.pool) .await?; - Ok(sandbox_id) + Ok(rows) + } + + pub async fn list_stale_evicting_warm_sandbox_ids( + &self, + min_age: Duration, + ) -> Result, SessionStoreError> { + let rows = sqlx::query_scalar::<_, String>( + r#" + select sandbox_id + from session_warm_sandboxes + where status = 'evicting' + and updated_at <= now() - ($1::float8 * interval '1 second') + order by updated_at, sandbox_id + "#, + ) + .bind(min_age.as_secs_f64()) + .fetch_all(&self.pool) + .await?; + Ok(rows) } pub async fn mark_warm_sandbox_failed( @@ -852,6 +1891,35 @@ impl PgSessionStore { Ok(()) } + /// Mark a stale warm-pool candidate failed only if it is still unclaimed + /// and unbound. This is the compare-and-swap half of the status probe in + /// the reconciler: a concurrent session claim must win over stale probe + /// results collected while the backend call was in flight. + pub async fn mark_ready_warm_sandbox_failed_if_unclaimed( + &self, + sandbox_id: &str, + error: &str, + ) -> Result { + let result = sqlx::query( + r#" + update session_warm_sandboxes warm + set status = 'failed', last_error = $2, updated_at = now() + where warm.sandbox_id = $1 + and warm.status = 'ready' + and warm.claimed_thread_key is null + and not exists ( + select 1 from sessions session + where session.sandbox_id = warm.sandbox_id + ) + "#, + ) + .bind(sandbox_id) + .bind(error) + .execute(&self.pool) + .await?; + Ok(result.rows_affected() == 1) + } + pub async fn update_harness_thread_id( &self, thread_key: &ThreadKey, @@ -862,7 +1930,7 @@ impl PgSessionStore { update sessions set harness_thread_id = $2, updated_at = now() where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + returning thread_key, title, sandbox_id, sandbox_content_revision, sandbox_repo_cache_enabled, sandbox_repo_cache_access, sandbox_observability_enabled, sandbox_api_server_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, sandbox_last_active_at, created_at, updated_at "#, ) .bind(thread_key.as_str()) @@ -873,34 +1941,42 @@ impl PgSessionStore { row.try_into() } - pub async fn release_session( + pub async fn touch_session_sandbox_activity( &self, thread_key: &ThreadKey, - ) -> Result { - let row = sqlx::query_as::<_, SessionRow>( + ) -> Result { + let result = sqlx::query( r#" update sessions - set sandbox_id = null, - sandbox_repo_cache_enabled = null, - sandbox_observability_enabled = null, - status = $2, - updated_at = now() - where thread_key = $1 - returning thread_key, sandbox_id, sandbox_repo_cache_enabled, sandbox_observability_enabled, harness_type, harness_thread_id, persona_id, status, iron_control_principal, created_at, updated_at + set sandbox_last_active_at = now() + where thread_key = $1 and sandbox_id is not null "#, ) .bind(thread_key.as_str()) - .bind(SessionStatus::Idle.as_ref()) - .fetch_optional(&self.pool) + .execute(&self.pool) .await?; - let Some(row) = row else { - return Err(SessionStoreError::NotFound { - thread_key: thread_key.as_str().to_owned(), - }); - }; + Ok(result.rows_affected() > 0) + } - row.try_into() + pub async fn touch_sandbox_activity( + &self, + thread_key: &ThreadKey, + sandbox_id: &str, + ) -> Result { + let result = sqlx::query( + r#" + update sessions + set sandbox_last_active_at = now() + where thread_key = $1 and sandbox_id = $2 + "#, + ) + .bind(thread_key.as_str()) + .bind(sandbox_id) + .execute(&self.pool) + .await?; + + Ok(result.rows_affected() > 0) } async fn set_session_status( @@ -973,6 +2049,14 @@ pub enum SessionStoreError { existing: Option, requested: Option, }, + #[error( + "session {thread_key} is bound to principal {existing:?}, requested principal {requested}" + )] + PrincipalConflict { + thread_key: String, + existing: Option, + requested: String, + }, #[error("invalid persisted value: {0}")] InvalidPersistedValue(String), #[error("invalid notification payload on {channel}: {payload}: {error}")] @@ -990,14 +2074,19 @@ pub enum SessionStoreError { #[derive(Debug, FromRow)] struct SessionRow { thread_key: String, + title: Option, sandbox_id: Option, + sandbox_content_revision: Option, sandbox_repo_cache_enabled: Option, + sandbox_repo_cache_access: Option, sandbox_observability_enabled: Option, + sandbox_api_server_enabled: Option, harness_type: String, harness_thread_id: Option, persona_id: Option, status: String, iron_control_principal: Option, + sandbox_last_active_at: Option, created_at: OffsetDateTime, updated_at: OffsetDateTime, } @@ -1008,17 +2097,30 @@ impl TryFrom for Session { fn try_from(row: SessionRow) -> Result { Ok(Self { thread_key: parse_persisted(row.thread_key)?, + title: row.title, sandbox_id: row.sandbox_id, + sandbox_content_revision: row.sandbox_content_revision, sandbox_capabilities: match ( row.sandbox_repo_cache_enabled, + row.sandbox_repo_cache_access, row.sandbox_observability_enabled, + row.sandbox_api_server_enabled, ) { - (Some(repo_cache_enabled), Some(observability_enabled)) => { - Some(SandboxCapabilities { - repo_cache_enabled, - observability_enabled, - }) - } + ( + Some(repo_cache_enabled), + repo_cache_access, + Some(observability_enabled), + Some(api_server_enabled), + ) => Some(SandboxCapabilities { + repo_cache: repo_cache_access + .as_deref() + .and_then(SandboxRepoCacheAccess::parse) + .unwrap_or_else(|| { + SandboxRepoCacheAccess::from_legacy_enabled(repo_cache_enabled) + }), + observability_enabled, + api_server_enabled, + }), _ => None, }, harness_type: parse_persisted(row.harness_type)?, @@ -1026,6 +2128,7 @@ impl TryFrom for Session { persona_id: row.persona_id, status: parse_persisted(row.status)?, iron_control_principal: row.iron_control_principal, + sandbox_last_active_at: row.sandbox_last_active_at, created_at: row.created_at, updated_at: row.updated_at, }) @@ -1063,7 +2166,7 @@ impl TryFrom for SessionMessage { } } -#[derive(Debug, FromRow)] +#[derive(Clone, Debug, FromRow)] struct SessionExecutionRow { execution_id: String, idempotency_key: Option, @@ -1077,21 +2180,78 @@ struct SessionExecutionRow { completed_at: Option, } +#[derive(Debug, FromRow)] +struct ActiveExecutionOwnershipRow { + #[sqlx(flatten)] + execution: SessionExecutionRow, + stdout_owner_id: Option, + stdout_owner_lease_active: bool, +} + #[derive(Debug, FromRow)] struct IdleSandboxCandidateRow { thread_key: String, sandbox_id: String, execution_id: String, + completed_at: OffsetDateTime, + metadata: Value, } -impl TryFrom for IdleSandboxCandidate { +fn idle_candidate_from_row( + row: IdleSandboxCandidateRow, + idle_backstop: Duration, + now: OffsetDateTime, +) -> Result, SessionStoreError> { + let idle_timeout = effective_idle_timeout(&row.metadata, idle_backstop); + if !idle_deadline_elapsed(row.completed_at, idle_timeout, now) { + return Ok(None); + } + Ok(Some(IdleSandboxCandidate { + thread_key: parse_persisted(row.thread_key)?, + sandbox_id: row.sandbox_id, + execution_id: row.execution_id, + idle_timeout, + })) +} + +fn effective_idle_timeout(metadata: &Value, idle_backstop: Duration) -> Duration { + metadata + .get("idle_timeout_ms") + .and_then(Value::as_u64) + .filter(|value| *value > 0) + .map(Duration::from_millis) + .unwrap_or_else(|| std::cmp::max(idle_backstop, Duration::from_millis(1))) +} + +fn idle_deadline_elapsed( + completed_at: OffsetDateTime, + idle_timeout: Duration, + now: OffsetDateTime, +) -> bool { + let elapsed = now - completed_at; + if elapsed.is_negative() { + return false; + } + elapsed.whole_nanoseconds() >= idle_timeout.as_nanos() as i128 +} + +#[derive(Debug, FromRow)] +struct SandboxCapacityCandidateRow { + thread_key: String, + sandbox_id: String, + latest_execution_id: Option, + last_active_at: OffsetDateTime, +} + +impl TryFrom for SandboxCapacityCandidate { type Error = SessionStoreError; - fn try_from(row: IdleSandboxCandidateRow) -> Result { + fn try_from(row: SandboxCapacityCandidateRow) -> Result { Ok(Self { thread_key: parse_persisted(row.thread_key)?, sandbox_id: row.sandbox_id, - execution_id: row.execution_id, + latest_execution_id: row.latest_execution_id, + last_active_at: row.last_active_at, }) } } @@ -1099,7 +2259,7 @@ impl TryFrom for IdleSandboxCandidate { #[derive(Debug, FromRow)] struct WorkflowOwnedSandboxRow { thread_key: String, - sandbox_id: String, + sandbox_id: Option, } impl TryFrom for WorkflowOwnedSandbox { @@ -1213,9 +2373,46 @@ pub fn default_metadata(metadata: Option) -> Value { metadata.unwrap_or_else(empty_object) } +fn stdout_lease_expires_at(lease: Duration) -> OffsetDateTime { + let seconds = i64::try_from(lease.as_secs()).unwrap_or(i64::MAX); + OffsetDateTime::now_utc() + TimeDuration::new(seconds, lease.subsec_nanos() as i32) +} + #[cfg(test)] mod tests { - use super::SessionEventNotification; + use std::{sync::Arc, time::Duration}; + + use centaur_session_core::{ExecutionStatus, HarnessType, SandboxCapabilities, ThreadKey}; + use serde_json::json; + use time::{Duration as TimeDuration, OffsetDateTime}; + use tokio::sync::OnceCell; + use uuid::Uuid; + + use super::{ + IdleSandboxCandidateRow, PgSessionStore, ReleaseSessionResult, SessionEventNotification, + SessionStoreError, + }; + + async fn test_store() -> Option { + let Ok(url) = std::env::var("SESSION_RUNTIME_TEST_DATABASE_URL") else { + eprintln!("skipping: SESSION_RUNTIME_TEST_DATABASE_URL not set"); + return None; + }; + static MIGRATIONS: OnceCell<()> = OnceCell::const_new(); + MIGRATIONS + .get_or_init(|| async { + let store = PgSessionStore::connect(&url) + .await + .expect("connect test db"); + store.run_migrations().await.expect("run migrations"); + }) + .await; + Some( + PgSessionStore::connect(&url) + .await + .expect("connect test db after migrations"), + ) + } #[test] fn parses_session_event_notification_payload() { @@ -1230,4 +2427,853 @@ mod tests { } ); } + + #[tokio::test] + async fn principal_bound_session_rejects_cross_principal_restart_before_mutation() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = + ThreadKey::parse(format!("feedback-improvement:test:{}", Uuid::new_v4())).unwrap(); + let created = store + .create_or_get_session_for_principal( + &thread_key, + &HarnessType::Codex, + None, + json!({"source": "principal-a"}), + "prn_a", + ) + .await + .expect("create principal A session"); + assert_eq!(created.iron_control_principal.as_deref(), Some("prn_a")); + + let error = store + .create_or_get_session_for_principal( + &thread_key, + &HarnessType::Amp, + None, + json!({"source": "principal-b"}), + "prn_b", + ) + .await + .expect_err("principal B must not reach harness restart handling"); + assert!(matches!( + error, + SessionStoreError::PrincipalConflict { + existing: Some(existing), + requested, + .. + } if existing == "prn_a" && requested == "prn_b" + )); + + let unchanged = store + .get_session(&thread_key) + .await + .expect("principal A session remains"); + assert_eq!(unchanged.harness_type, HarnessType::Codex); + assert_eq!(unchanged.iron_control_principal.as_deref(), Some("prn_a")); + } + + fn idle_row( + metadata: serde_json::Value, + completed_at: OffsetDateTime, + ) -> IdleSandboxCandidateRow { + IdleSandboxCandidateRow { + thread_key: "test:idle-row".to_owned(), + sandbox_id: "sbx-idle-row".to_owned(), + execution_id: "exe-idle-row".to_owned(), + completed_at, + metadata, + } + } + + #[test] + fn idle_candidate_uses_persisted_timeout_deadline() { + let now = OffsetDateTime::now_utc(); + let candidate = super::idle_candidate_from_row( + idle_row( + json!({"idle_timeout_ms": 1000}), + now - TimeDuration::seconds(2), + ), + Duration::from_secs(3600), + now, + ) + .unwrap() + .expect("candidate should use persisted timeout"); + + assert_eq!(candidate.idle_timeout, Duration::from_secs(1)); + } + + #[test] + fn idle_candidate_waits_for_persisted_timeout_even_when_backstop_elapsed() { + let now = OffsetDateTime::now_utc(); + let candidate = super::idle_candidate_from_row( + idle_row( + json!({"idle_timeout_ms": 10_000}), + now - TimeDuration::seconds(2), + ), + Duration::from_secs(1), + now, + ) + .unwrap(); + + assert!(candidate.is_none()); + } + + #[test] + fn idle_candidate_falls_back_to_backstop_for_missing_or_invalid_timeout() { + let now = OffsetDateTime::now_utc(); + let candidate = super::idle_candidate_from_row( + idle_row( + json!({"idle_timeout_ms": "not-a-number"}), + now - TimeDuration::seconds(2), + ), + Duration::from_secs(1), + now, + ) + .unwrap() + .expect("candidate should use backstop"); + + assert_eq!(candidate.idle_timeout, Duration::from_secs(1)); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn idle_candidates_use_persisted_execution_idle_timeout() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = ThreadKey::parse(format!("test:idle-cleanup-{}", Uuid::new_v4())).unwrap(); + let sandbox_id = format!("sbx-idle-{}", Uuid::new_v4()); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some(&sandbox_id)) + .await + .expect("set sandbox id"); + let execution_id = store + .create_execution(&thread_key, None, json!({"idle_timeout_ms": 1000})) + .await + .expect("create execution") + .execution + .execution_id; + store + .complete_execution(&execution_id) + .await + .expect("complete execution"); + sqlx::query( + r#" + update session_executions + set completed_at = now() - interval '2 seconds', updated_at = now() + where execution_id = $1 + "#, + ) + .bind(&execution_id) + .execute(store.pool()) + .await + .expect("age execution"); + + let candidates = store + .list_idle_sandbox_candidates(Duration::from_secs(3600)) + .await + .expect("list idle sandbox candidates"); + let candidate = candidates + .iter() + .find(|candidate| candidate.thread_key == thread_key) + .expect("candidate should use execution idle timeout, not backstop"); + + assert_eq!(candidate.sandbox_id, sandbox_id); + assert_eq!(candidate.execution_id, execution_id); + assert_eq!(candidate.idle_timeout, Duration::from_secs(1)); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn stdout_owner_fences_output_and_terminal_updates() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = ThreadKey::parse(format!("test:stdout-owner-{}", Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark running"); + + assert!( + store + .claim_stdout_owner(&execution_id, "owner-a", Duration::from_millis(25)) + .await + .expect("owner-a claims stdout") + ); + assert!( + store + .append_event_if_stdout_owner( + &thread_key, + &execution_id, + "owner-a", + Duration::from_millis(25), + "session.output.line", + json!("line-from-owner-a"), + ) + .await + .expect("owner-a appends") + .is_some() + ); + assert!( + store + .append_event_if_stdout_owner( + &thread_key, + &execution_id, + "owner-b", + Duration::from_millis(25), + "session.output.line", + json!("line-from-stale-owner-b"), + ) + .await + .expect("owner-b append is fenced") + .is_none() + ); + assert!( + store + .complete_execution_if_active_and_stdout_owner(&execution_id, "owner-b") + .await + .expect("owner-b terminal update is fenced") + .is_none() + ); + + tokio::time::sleep(Duration::from_millis(40)).await; + assert!( + store + .claim_expired_stdout_owner(&execution_id, "owner-b", Duration::from_secs(5)) + .await + .expect("owner-b claims after lease expiry") + ); + assert!( + store + .append_event_if_stdout_owner( + &thread_key, + &execution_id, + "owner-a", + Duration::from_secs(5), + "session.output.line", + json!("line-from-expired-owner-a"), + ) + .await + .expect("expired owner-a append is fenced") + .is_none() + ); + let completed = store + .complete_execution_if_active_and_stdout_owner(&execution_id, "owner-b") + .await + .expect("owner-b completes") + .expect("completion should be recorded"); + assert_eq!( + completed.status, + centaur_session_core::ExecutionStatus::Completed + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn canonical_release_requires_cancellation_and_fences_the_old_stdout_owner() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = ThreadKey::parse(format!("test:release-{}", Uuid::new_v4())).unwrap(); + let sandbox_id = format!("sbx-release-{}", Uuid::new_v4().simple()); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some(&sandbox_id)) + .await + .expect("assign sandbox"); + store + .update_harness_thread_id(&thread_key, Some("codex-thread-old")) + .await + .expect("assign harness thread"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark execution running"); + assert!( + store + .claim_stdout_owner(&execution_id, "old-owner", Duration::from_secs(60)) + .await + .expect("claim stdout owner") + ); + + let rejected = store + .release_session_if_sandbox_matches( + &thread_key, + Some(&sandbox_id), + false, + "release requested", + ) + .await + .expect("release decision"); + assert!(matches!(rejected, ReleaseSessionResult::ActiveExecution(_))); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("session after rejected release") + .sandbox_id + .as_deref(), + Some(sandbox_id.as_str()) + ); + + let released = store + .release_session_if_sandbox_matches( + &thread_key, + Some(&sandbox_id), + true, + "release requested", + ) + .await + .expect("release session"); + let ReleaseSessionResult::Released { + session, + cancelled_execution, + } = released + else { + panic!("expected released session"); + }; + assert_eq!(session.sandbox_id, None); + assert_eq!(session.harness_thread_id, None); + assert_eq!(session.status, centaur_session_core::SessionStatus::Idle); + assert_eq!( + cancelled_execution + .as_ref() + .map(|execution| &execution.status), + Some(¢aur_session_core::ExecutionStatus::Cancelled) + ); + + assert!( + store + .append_event_if_stdout_owner( + &thread_key, + &execution_id, + "old-owner", + Duration::from_secs(60), + "session.output.line", + json!("stale output"), + ) + .await + .expect("stale owner append is fenced") + .is_none() + ); + assert!( + store + .complete_execution_if_active_and_stdout_owner(&execution_id, "old-owner") + .await + .expect("stale owner completion is fenced") + .is_none() + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn release_and_stdout_append_share_session_then_execution_lock_order() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = ThreadKey::parse(format!("test:release-race-{}", Uuid::new_v4())).unwrap(); + let sandbox_id = format!("sbx-release-race-{}", Uuid::new_v4().simple()); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .update_sandbox_id(&thread_key, Some(&sandbox_id)) + .await + .expect("assign sandbox"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark running"); + assert!( + store + .claim_stdout_owner(&execution_id, "race-owner", Duration::from_secs(60)) + .await + .expect("claim owner") + ); + + let barrier = Arc::new(tokio::sync::Barrier::new(2)); + let append_store = store.clone(); + let append_thread = thread_key.clone(); + let append_execution = execution_id.clone(); + let append_barrier = barrier.clone(); + let append = async move { + append_barrier.wait().await; + for sequence in 0..32 { + if append_store + .append_event_if_stdout_owner( + &append_thread, + &append_execution, + "race-owner", + Duration::from_secs(60), + "session.output.line", + json!({"sequence": sequence}), + ) + .await? + .is_none() + { + break; + } + } + Ok::<_, SessionStoreError>(()) + }; + let release_store = store.clone(); + let release_thread = thread_key.clone(); + let release_sandbox = sandbox_id.clone(); + let release = async move { + barrier.wait().await; + release_store + .release_session_if_sandbox_matches( + &release_thread, + Some(&release_sandbox), + true, + "concurrent release", + ) + .await + }; + + let (append_result, release_result) = tokio::time::timeout(Duration::from_secs(5), async { + tokio::join!(append, release) + }) + .await + .expect("release/stdout append must not deadlock"); + append_result.expect("append result"); + assert!(matches!( + release_result.expect("release result"), + ReleaseSessionResult::Released { .. } + )); + } + + #[tokio::test] + async fn cancelled_execution_cannot_assign_a_sandbox_after_null_release() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = + ThreadKey::parse(format!("test:release-before-bind-{}", Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark execution running"); + assert!( + store + .claim_stdout_owner(&execution_id, "released-owner", Duration::from_secs(60)) + .await + .expect("claim stdout owner") + ); + + assert!(matches!( + store + .release_session_if_sandbox_matches( + &thread_key, + None, + true, + "release before sandbox bind", + ) + .await + .expect("release session"), + ReleaseSessionResult::Released { .. } + )); + + let assigned = store + .assign_sandbox_to_active_execution( + &thread_key, + &execution_id, + "released-owner", + None, + "sbx-too-late", + None, + &SandboxCapabilities::default_enabled(), + ) + .await + .expect("fenced assignment"); + assert!(assigned.is_none()); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("session after fenced assignment") + .sandbox_id, + None + ); + let status = sqlx::query_scalar::<_, String>( + "select status from session_executions where execution_id = $1", + ) + .bind(&execution_id) + .fetch_one(store.pool()) + .await + .expect("cancelled execution status"); + assert_eq!(status, ExecutionStatus::Cancelled.as_ref()); + } + + #[tokio::test] + async fn stale_stdout_owner_cannot_assign_or_clear_after_lease_takeover() { + let Some(store) = test_store().await else { + return; + }; + let thread_key = + ThreadKey::parse(format!("test:owner-before-bind-{}", Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark execution running"); + assert!( + store + .claim_stdout_owner(&execution_id, "owner-a", Duration::from_secs(60)) + .await + .expect("claim owner A") + ); + sqlx::query( + "update session_executions set stdout_owner_lease_expires_at = now() - interval '1 second' where execution_id = $1", + ) + .bind(&execution_id) + .execute(store.pool()) + .await + .expect("expire owner A lease"); + assert!( + store + .claim_stdout_owner(&execution_id, "owner-b", Duration::from_secs(60)) + .await + .expect("claim owner B") + ); + + assert!( + store + .assign_sandbox_to_active_execution( + &thread_key, + &execution_id, + "owner-a", + None, + "sbx-stale-owner", + None, + &SandboxCapabilities::default_enabled(), + ) + .await + .expect("stale owner assignment") + .is_none() + ); + assert!( + store + .assign_sandbox_to_active_execution( + &thread_key, + &execution_id, + "owner-b", + None, + "sbx-current-owner", + None, + &SandboxCapabilities::default_enabled(), + ) + .await + .expect("current owner assignment") + .is_some() + ); + assert!( + store + .clear_sandbox_from_active_execution( + &thread_key, + &execution_id, + "owner-a", + "sbx-current-owner", + ) + .await + .expect("stale owner clear") + .is_none() + ); + assert!( + store + .clear_sandbox_from_active_execution( + &thread_key, + &execution_id, + "owner-b", + "sbx-current-owner", + ) + .await + .expect("current owner clear") + .is_some() + ); + assert_eq!( + store + .get_session(&thread_key) + .await + .expect("session after current owner clear") + .sandbox_id, + None + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn releases_all_stdout_leases_held_by_one_owner() { + let Some(store) = test_store().await else { + return; + }; + let owner = format!("owner-{}", Uuid::new_v4().simple()); + let peer = format!("peer-{}", Uuid::new_v4().simple()); + let mut owned = Vec::new(); + for label in ["a", "b"] { + let thread_key = + ThreadKey::parse(format!("test:handoff-{label}-{}", Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + let execution_id = store + .create_execution(&thread_key, None, json!({})) + .await + .expect("create execution") + .execution + .execution_id; + store + .mark_execution_running(&execution_id) + .await + .expect("mark running"); + assert!( + store + .claim_stdout_owner(&execution_id, &owner, Duration::from_secs(60)) + .await + .expect("claim stdout owner") + ); + owned.push((execution_id, thread_key)); + } + // A bystander owner's lease must survive the release untouched. + let bystander_thread = + ThreadKey::parse(format!("test:handoff-bystander-{}", Uuid::new_v4())).unwrap(); + store + .create_or_get_session(&bystander_thread, &HarnessType::Codex, None, json!({})) + .await + .expect("create bystander session"); + let bystander_execution = store + .create_execution(&bystander_thread, None, json!({})) + .await + .expect("create bystander execution") + .execution + .execution_id; + store + .mark_execution_running(&bystander_execution) + .await + .expect("mark bystander running"); + let bystander = format!("bystander-{}", Uuid::new_v4().simple()); + assert!( + store + .claim_stdout_owner(&bystander_execution, &bystander, Duration::from_secs(60)) + .await + .expect("claim bystander lease") + ); + assert_eq!( + store + .count_executions_with_stdout_owner(&owner) + .await + .expect("count owned"), + 2 + ); + + let released = store + .release_stdout_owned_executions(&owner) + .await + .expect("release owned leases"); + assert_eq!(released.len(), 2); + for (execution_id, thread_key) in &owned { + assert!( + released.iter().any(|execution| { + execution.execution_id == *execution_id && execution.thread_key == *thread_key + }), + "released set must include {execution_id}" + ); + } + assert_eq!( + store + .count_executions_with_stdout_owner(&owner) + .await + .expect("count after release"), + 0 + ); + + // Released leases are immediately claimable by a peer, without + // waiting for expiry. + assert!( + store + .claim_stdout_owner(&owned[0].0, &peer, Duration::from_secs(60)) + .await + .expect("peer claims released lease") + ); + + assert_eq!( + store + .count_executions_with_stdout_owner(&bystander) + .await + .expect("count bystander"), + 1, + "release must be scoped to the requested owner" + ); + store + .fail_execution_if_active(&bystander_execution, "test cleanup") + .await + .expect("terminalize bystander"); + + // Terminal executions are never part of a release, even if a lease + // column is still populated. + for (execution_id, _) in &owned { + store + .fail_execution_if_active(execution_id, "test cleanup") + .await + .expect("terminalize execution"); + } + assert!( + store + .release_stdout_owned_executions(&peer) + .await + .expect("release for peer") + .is_empty() + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn warm_eviction_reservation_blocks_later_claims() { + let Some(store) = test_store().await else { + return; + }; + let sandbox_id = format!("sbx-warm-evict-{}", Uuid::new_v4()); + let workload_key = format!("workload-warm-evict-{}", Uuid::new_v4()); + store + .insert_ready_warm_sandbox(&sandbox_id, &workload_key) + .await + .expect("insert warm sandbox"); + sqlx::query( + r#" + update session_warm_sandboxes + set created_at = now() - interval '100 years' + where sandbox_id = $1 + "#, + ) + .bind(&sandbox_id) + .execute(store.pool()) + .await + .expect("age warm sandbox"); + + let reserved = store + .reserve_ready_warm_sandboxes_for_eviction(1) + .await + .expect("reserve warm sandbox"); + + assert_eq!(reserved, vec![sandbox_id.clone()]); + assert_eq!( + store + .claim_ready_warm_sandbox(&workload_key, "test-thread") + .await + .expect("claim after reservation"), + None + ); + assert!( + store + .list_referenced_sandbox_ids() + .await + .expect("list referenced sandboxes") + .contains(&sandbox_id) + ); + + store + .mark_warm_sandbox_failed(&sandbox_id, "test cleanup") + .await + .expect("mark reserved warm sandbox failed"); + assert!( + !store + .list_referenced_sandbox_ids() + .await + .expect("list referenced sandboxes") + .contains(&sandbox_id) + ); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn stale_ready_failure_cannot_overwrite_a_concurrent_claim() { + let Some(store) = test_store().await else { + return; + }; + let sandbox_id = format!("sbx-warm-claim-race-{}", Uuid::new_v4()); + let workload_key = format!("workload-warm-claim-race-{}", Uuid::new_v4()); + let thread_key = ThreadKey::parse(format!("test:warm-claim-race-{}", Uuid::new_v4())) + .expect("thread key"); + store + .create_or_get_session(&thread_key, &HarnessType::Codex, None, json!({})) + .await + .expect("create session"); + store + .insert_ready_warm_sandbox(&sandbox_id, &workload_key) + .await + .expect("insert warm sandbox"); + assert_eq!( + store + .claim_ready_warm_sandbox(&workload_key, thread_key.as_str()) + .await + .expect("claim warm sandbox"), + Some(sandbox_id.clone()) + ); + + assert!( + !store + .mark_ready_warm_sandbox_failed_if_unclaimed(&sandbox_id, "stale backend status",) + .await + .expect("conditional stale failure") + ); + let status = sqlx::query_scalar::<_, String>( + "select status from session_warm_sandboxes where sandbox_id = $1", + ) + .bind(&sandbox_id) + .fetch_one(store.pool()) + .await + .expect("load warm status"); + assert_eq!(status, "claimed"); + + store + .mark_warm_sandbox_failed(&sandbox_id, "test cleanup") + .await + .expect("cleanup warm sandbox"); + } } diff --git a/services/api-rs/crates/centaur-session-sqlx/tests/etl_context_rls.rs b/services/api-rs/crates/centaur-session-sqlx/tests/etl_context_rls.rs index 186290dac..0562722b0 100644 --- a/services/api-rs/crates/centaur-session-sqlx/tests/etl_context_rls.rs +++ b/services/api-rs/crates/centaur-session-sqlx/tests/etl_context_rls.rs @@ -18,6 +18,8 @@ const DROP_SLACK_CONTEXT_ADMIN_CHANNELS_SQL: &str = include_str!("../migrations/0023_drop_slack_context_rls_admin_channels.sql"); const CENTAUR_READONLY_RLS_POLICIES_SQL: &str = include_str!("../migrations/0024_centaur_readonly_rls_policies.sql"); +const SLACK_PRIVATE_CHANNELS_SQL: &str = + include_str!("../migrations/0039_slack_private_channels.sql"); const RLS_TABLES: &[&str] = &[ "slack_sync_channels", @@ -84,11 +86,16 @@ async fn run_rls_assertions(conn: &mut PgConnection, schema: &str) -> Result<(), execute_migration(conn, ETL_CONTEXT_RLS_SQL).await?; execute_migration(conn, DROP_SLACK_CONTEXT_ADMIN_CHANNELS_SQL).await?; execute_migration(conn, CENTAUR_READONLY_RLS_POLICIES_SQL).await?; + insert_privacy_backfill_rows(conn).await?; + execute_migration(conn, SLACK_PRIVATE_CHANNELS_SQL).await?; + assert_privacy_backfill(conn).await?; + clear_privacy_backfill_rows(conn).await?; grant_schema_usage(conn, schema).await?; assert_rls_enabled(conn).await?; assert_expected_policies(conn).await?; assert_legacy_admin_state_is_removed(conn).await?; + assert_public_context_helper_acl(conn).await?; insert_fixture_rows(conn).await?; @@ -158,6 +165,16 @@ async fn run_rls_assertions(conn: &mut PgConnection, schema: &str) -> Result<(), let unset_channel = visible_rows(conn, schema, "centaur_slack_reader", None).await?; assert_eq!(unset_channel, public_company_context_rows()); + let dm_channel = visible_rows(conn, schema, "centaur_slack_reader", Some("D_DM")).await?; + assert_eq!(dm_channel, public_company_context_rows()); + + let other_channel = visible_rows(conn, schema, "centaur_slack_reader", Some("C_OTHER")).await?; + assert_eq!(other_channel, public_company_context_rows()); + + let private_channel = + visible_rows(conn, schema, "centaur_slack_reader", Some("C_PRIVATE")).await?; + assert_eq!(private_channel, private_current_channel_rows()); + let formerly_admin_channel = visible_rows(conn, schema, "centaur_slack_reader", Some("C_ADMIN")).await?; assert_eq!( @@ -189,7 +206,11 @@ async fn run_rls_assertions(conn: &mut PgConnection, schema: &str) -> Result<(), ); let readonly_role = visible_rows(conn, schema, "centaur_readonly", None).await?; - assert_eq!(readonly_role, all_visible_rows()); + assert_eq!(readonly_role, public_visible_rows()); + + let readonly_private_channel = + visible_rows(conn, schema, "centaur_readonly", Some("C_PRIVATE")).await?; + assert_eq!(readonly_private_channel, public_and_private_visible_rows()); Ok(()) } @@ -256,13 +277,116 @@ async fn execute_migration(conn: &mut PgConnection, sql: &str) -> Result<(), sql Ok(()) } +async fn insert_privacy_backfill_rows(conn: &mut PgConnection) -> Result<(), sqlx::Error> { + sqlx::raw_sql( + r#" + insert into slack_sync_channels (channel_id, channel_name, raw_payload) values + ('C_BACKFILL_PUBLIC', 'known public', '{"is_private": false}'), + ('C_BACKFILL_PRIVATE', 'known private', '{"is_private": true}'), + ('C_BACKFILL_UNKNOWN', 'unknown privacy', '{}'), + ('C_BACKFILL_MALFORMED', 'malformed privacy', '{"is_private": "false"}'); + "#, + ) + .execute(&mut *conn) + .await?; + Ok(()) +} + +async fn assert_privacy_backfill(conn: &mut PgConnection) -> Result<(), sqlx::Error> { + let rows: Vec<(String, bool)> = sqlx::query_as( + r#" + select channel_id, is_private + from slack_sync_channels + where channel_id like 'C_BACKFILL_%' + order by channel_id + "#, + ) + .fetch_all(&mut *conn) + .await?; + assert_eq!( + rows, + vec![ + ("C_BACKFILL_MALFORMED".to_owned(), true), + ("C_BACKFILL_PRIVATE".to_owned(), true), + ("C_BACKFILL_PUBLIC".to_owned(), false), + ("C_BACKFILL_UNKNOWN".to_owned(), true), + ] + ); + + let default_is_private: bool = sqlx::query_scalar( + "insert into slack_sync_channels (channel_id, channel_name) values ('C_DEFAULT', 'default') returning is_private", + ) + .fetch_one(&mut *conn) + .await?; + assert!( + default_is_private, + "unknown privacy must default to private" + ); + Ok(()) +} + +async fn clear_privacy_backfill_rows(conn: &mut PgConnection) -> Result<(), sqlx::Error> { + sqlx::query("delete from slack_sync_channels where channel_id like 'C_BACKFILL_%' or channel_id = 'C_DEFAULT'") + .execute(&mut *conn) + .await?; + Ok(()) +} + +async fn assert_public_context_helper_acl(conn: &mut PgConnection) -> Result<(), sqlx::Error> { + let row = sqlx::query( + r#" + select + p.prosecdef as security_definer, + coalesce(p.proconfig @> array['search_path=pg_catalog']::text[], false) + as locked_search_path, + has_function_privilege( + 'centaur_slack_reader', + p.oid, + 'EXECUTE' + ) as reader_can_execute, + exists ( + select 1 + from aclexplode(coalesce(p.proacl, acldefault('f', p.proowner))) acl + where acl.grantee = 0 + and acl.privilege_type = 'EXECUTE' + ) as public_can_execute + from pg_proc p + join pg_namespace n on n.oid = p.pronamespace + where n.nspname = current_schema() + and p.proname = 'centaur_slack_channel_is_public_syncable' + "#, + ) + .fetch_one(&mut *conn) + .await?; + + assert!(row.get::("security_definer")); + assert!(row.get::("locked_search_path")); + assert!(row.get::("reader_can_execute")); + assert!(!row.get::("public_can_execute")); + + let legacy_helper_count: i64 = sqlx::query_scalar( + r#" + select count(*) + from pg_proc p + join pg_namespace n on n.oid = p.pronamespace + where n.nspname = current_schema() + and p.proname = 'centaur_slack_channel_is_syncable' + "#, + ) + .fetch_one(&mut *conn) + .await?; + assert_eq!(legacy_helper_count, 0); + Ok(()) +} + async fn create_minimal_etl_tables(conn: &mut PgConnection) -> Result<(), sqlx::Error> { sqlx::raw_sql( r#" create table slack_sync_channels ( channel_id text primary key, channel_name text not null default '', - is_syncable boolean not null default false + is_syncable boolean not null default false, + raw_payload jsonb not null default '{}'::jsonb ); create table slack_sync_users ( @@ -544,28 +668,36 @@ async fn assert_legacy_admin_state_is_removed(conn: &mut PgConnection) -> Result async fn insert_fixture_rows(conn: &mut PgConnection) -> Result<(), sqlx::Error> { sqlx::raw_sql( r#" - insert into slack_sync_channels (channel_id, channel_name, is_syncable) values - ('C_ALPHA', 'alpha', false), - ('C_BETA', 'beta', true), - ('C_ADMIN', 'admin', false); + insert into slack_sync_channels + (channel_id, channel_name, is_syncable, is_private) + values + ('C_ALPHA', 'alpha', false, false), + ('C_BETA', 'beta', true, false), + ('C_ADMIN', 'admin', false, false), + ('C_PRIVATE', 'private', true, true); insert into slack_sync_users (user_id, user_name) values ('U_ALPHA', 'alpha user'), - ('U_BETA', 'beta user'); + ('U_BETA', 'beta user'), + ('U_PRIVATE', 'private user'); insert into slack_sync_messages (channel_id, message_ts, text) values ('C_ALPHA', '1000.000001', 'alpha channel message'), - ('C_BETA', '1000.000002', 'beta channel message'); + ('C_BETA', '1000.000002', 'beta channel message'), + ('C_PRIVATE', '1000.000003', 'private channel message'); insert into slack_sync_message_attachments (channel_id, message_ts, slack_file_id, name) values ('C_ALPHA', '1000.000001', 'F_ALPHA', 'alpha.pdf'), - ('C_BETA', '1000.000002', 'F_BETA', 'beta.pdf'); + ('C_BETA', '1000.000002', 'F_BETA', 'beta.pdf'), + ('C_PRIVATE', '1000.000003', 'F_PRIVATE', 'private.pdf'); insert into company_context_documents (document_id, source, source_type, metadata) values ('doc_slack_alpha', 'slack', 'slack_thread', '{"channel_id": "C_ALPHA"}'), ('doc_slack_beta', 'slack', 'slack_thread', '{"channel_id": "C_BETA"}'), + ('doc_slack_private', 'slack', 'slack_thread', '{"channel_id": "C_PRIVATE"}'), + ('doc_slack_unknown', 'slack', 'slack_thread', '{}'), ('doc_gdrive', 'google_drive', 'google_doc', '{}'), ('doc_gcal', 'google_calendar', 'calendar_event', '{}'), ('doc_linear', 'linear', 'linear_issue', '{}'); @@ -704,14 +836,34 @@ fn public_company_context_rows() -> VisibleRows { } } -fn all_visible_rows() -> VisibleRows { +fn private_current_channel_rows() -> VisibleRows { + VisibleRows { + slack_channels: vec!["C_PRIVATE".to_owned()], + slack_messages: vec!["C_PRIVATE:1000.000003".to_owned()], + slack_attachments: vec!["C_PRIVATE:1000.000003:F_PRIVATE".to_owned()], + context_docs: vec![ + "doc_gcal".to_owned(), + "doc_gdrive".to_owned(), + "doc_linear".to_owned(), + "doc_slack_beta".to_owned(), + "doc_slack_private".to_owned(), + ], + ..empty_visible_rows() + } +} + +fn public_visible_rows() -> VisibleRows { VisibleRows { slack_channels: vec![ "C_ADMIN".to_owned(), "C_ALPHA".to_owned(), "C_BETA".to_owned(), ], - slack_users: vec!["U_ALPHA".to_owned(), "U_BETA".to_owned()], + slack_users: vec![ + "U_ALPHA".to_owned(), + "U_BETA".to_owned(), + "U_PRIVATE".to_owned(), + ], slack_messages: vec![ "C_ALPHA:1000.000001".to_owned(), "C_BETA:1000.000002".to_owned(), @@ -741,3 +893,13 @@ fn all_visible_rows() -> VisibleRows { linear_checkpoints: 1, } } + +fn public_and_private_visible_rows() -> VisibleRows { + let mut rows = public_visible_rows(); + rows.slack_channels.push("C_PRIVATE".to_owned()); + rows.slack_messages.push("C_PRIVATE:1000.000003".to_owned()); + rows.slack_attachments + .push("C_PRIVATE:1000.000003:F_PRIVATE".to_owned()); + rows.context_docs.push("doc_slack_private".to_owned()); + rows +} diff --git a/services/api-rs/crates/centaur-session-sqlx/tests/slack_dm_context_rls.rs b/services/api-rs/crates/centaur-session-sqlx/tests/slack_dm_context_rls.rs index e215310bb..8d10f0d23 100644 --- a/services/api-rs/crates/centaur-session-sqlx/tests/slack_dm_context_rls.rs +++ b/services/api-rs/crates/centaur-session-sqlx/tests/slack_dm_context_rls.rs @@ -12,6 +12,8 @@ const SLACK_DM_CONTEXT_DOCUMENTS_SQL: &str = include_str!("../migrations/0029_slack_dm_context_documents.sql"); const SLACK_DM_CONVERSATION_CONTEXT_DOCUMENTS_SQL: &str = include_str!("../migrations/0030_slack_dm_conversation_context_documents.sql"); +const READONLY_DM_RLS_SQL: &str = + include_str!("../migrations/0042_centaur_readonly_slack_dm_rls.sql"); const RLS_TABLES: &[&str] = &[ "slack_dm_sync_conversations", @@ -58,6 +60,7 @@ async fn run_rls_assertions(conn: &mut PgConnection, schema: &str) -> Result<(), execute_migration(conn, SLACK_DM_SYNC_SQL).await?; execute_slack_dm_context_documents_migration(conn).await?; execute_slack_dm_conversation_context_documents_migration(conn).await?; + execute_migration(conn, READONLY_DM_RLS_SQL).await?; grant_schema_usage(conn, schema).await?; assert_rls_enabled(conn).await?; @@ -178,7 +181,11 @@ async fn run_rls_assertions(conn: &mut PgConnection, schema: &str) -> Result<(), Some("U_A"), ) .await?; - assert_eq!(readonly, empty_visible_dm_rows()); + assert_eq!(readonly, user_a); + + let readonly_missing_user = + visible_rows(conn, schema, "centaur_readonly", Some("T_HOME"), None).await?; + assert_eq!(readonly_missing_user, empty_visible_dm_rows()); Ok(()) } diff --git a/services/api-rs/crates/centaur-workflows/Cargo.toml b/services/api-rs/crates/centaur-workflows/Cargo.toml index 8b0e255c1..c700cb826 100644 --- a/services/api-rs/crates/centaur-workflows/Cargo.toml +++ b/services/api-rs/crates/centaur-workflows/Cargo.toml @@ -7,6 +7,7 @@ repository.workspace = true [dependencies] absurd-sdk.workspace = true +base64.workspace = true centaur-session-core.workspace = true centaur-session-runtime.workspace = true centaur-session-sqlx.workspace = true @@ -16,9 +17,11 @@ chrono.workspace = true chrono-tz.workspace = true cron.workspace = true futures-util.workspace = true +hmac.workspace = true reqwest.workspace = true serde.workspace = true serde_json.workspace = true +sha2.workspace = true sqlx.workspace = true thiserror.workspace = true time.workspace = true diff --git a/services/api-rs/crates/centaur-workflows/src/lib.rs b/services/api-rs/crates/centaur-workflows/src/lib.rs index 739c14e97..51be071b2 100644 --- a/services/api-rs/crates/centaur-workflows/src/lib.rs +++ b/services/api-rs/crates/centaur-workflows/src/lib.rs @@ -3,7 +3,10 @@ use std::{ env, path::PathBuf, str::FromStr, - sync::{Arc, RwLock}, + sync::{ + Arc, Mutex as StdMutex, RwLock, + atomic::{AtomicBool, Ordering}, + }, time::Duration, }; @@ -11,6 +14,7 @@ use absurd::{ Client, ClientOptions, CreateQueueOptions, RetryKind, RetryStrategy, SpawnOptions, StepHandle, TaskContext, TaskRegistrationOptions, Worker, WorkerOptions, }; +use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD}; use centaur_sandbox_core::SandboxSpec; use centaur_session_core::{HarnessType, MessageRole, SessionMessageInput, ThreadKey}; use centaur_session_runtime::{ @@ -21,15 +25,18 @@ use centaur_session_sqlx::PgSessionStore; use chrono::{DateTime, Utc}; use chrono_tz::Tz; use cron::Schedule; -use futures_util::{TryStreamExt, pin_mut}; +use futures_util::{TryStreamExt, future::join_all, pin_mut}; +use hmac::{Hmac, Mac}; use serde::{Deserialize, Serialize}; use serde_json::{Value, json}; +use sha2::Sha256; use sqlx::Row; use thiserror::Error; use time::OffsetDateTime; use tokio::{ io::{AsyncBufReadExt, AsyncRead, AsyncWrite, AsyncWriteExt, BufReader}, process::Command, + sync::Notify, task::JoinHandle, }; use tracing::{info, warn}; @@ -72,6 +79,155 @@ const WORKFLOW_ETL_BACKFILL_WORKER_CONCURRENCY_ENV: &str = const DEFAULT_WORKFLOW_ETL_BACKFILL_WORKER_CONCURRENCY: usize = 1; const WORKFLOW_SCHEDULE_WORKER_CONCURRENCY_ENV: &str = "WORKFLOW_SCHEDULE_WORKER_CONCURRENCY"; const DEFAULT_WORKFLOW_SCHEDULE_WORKER_CONCURRENCY: usize = 1; +const WORKFLOW_WORKER_CLOSE_TIMEOUT: Duration = Duration::from_secs(5); +const WORKFLOW_TASK_TOKEN_TTL: Duration = Duration::from_secs(60 * 60); +const WORKFLOW_TASK_TOKEN_VERSION: u8 = 1; +const WORKFLOW_TASK_TOKEN_ENV: &str = "CENTAUR_WORKFLOW_TASK_TOKEN"; +const WORKFLOW_API_KEY_ENV: &str = "WORKFLOW_API_KEY"; +const CONTROL_API_KEY_ENV: &str = "CENTAUR_CONTROL_API_KEY"; +/// Local workflow-host mode is a developer compatibility path, not a process +/// sandbox. Still remove every known ambient control/service credential so a +/// child workflow must use its scoped task capability instead of inheriting an +/// api-rs administrator lane. +const LOCAL_WORKFLOW_HOST_DENIED_ENVS: &[&str] = &[ + CONTROL_API_KEY_ENV, + WORKFLOW_API_KEY_ENV, + "CENTAUR_API_KEY", + "CENTAUR_JWT_SIGNING_SECRET", + "CENTAUR_CONSOLE_CENTAUR_API_KEY", + "IRON_CONTROL_API_KEY", + "IRON_CONTROL_INITIAL_API_KEY", + "SLACKBOT_API_KEY", + "GITHUBBOT_API_KEY", + "LINEARBOT_API_KEY", + "DISCORDBOT_API_KEY", + "TEAMSBOT_API_KEY", + "SLACK_FEEDBACK_API_KEY", + "SLACK_BOT_TOKEN", + "SLACK_BOT_TOKEN_OVERRIDE", + "GITHUB_TOKEN", + "OPENAI_API_KEY", +]; + +type HmacSha256 = Hmac; + +#[derive(Debug, Deserialize, Serialize)] +struct WorkflowTaskTokenClaims { + version: u8, + run_id: String, + task_id: String, + expires_at: i64, +} + +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct WorkflowTaskTokenIdentity { + pub run_id: String, + pub task_id: String, +} + +pub fn mint_workflow_task_token( + signing_key: &[u8], + run_id: &str, + task_id: &str, + expires_at: i64, +) -> Result { + if signing_key.is_empty() || run_id.trim().is_empty() || task_id.trim().is_empty() { + return Err(WorkflowRuntimeError::Internal( + "workflow task token requires a signing key, run id, and task id".to_owned(), + )); + } + let payload = serde_json::to_vec(&WorkflowTaskTokenClaims { + version: WORKFLOW_TASK_TOKEN_VERSION, + run_id: run_id.to_owned(), + task_id: task_id.to_owned(), + expires_at, + })?; + let encoded = URL_SAFE_NO_PAD.encode(payload); + let mut mac = HmacSha256::new_from_slice(signing_key) + .map_err(|error| WorkflowRuntimeError::Internal(error.to_string()))?; + mac.update(encoded.as_bytes()); + let signature = URL_SAFE_NO_PAD.encode(mac.finalize().into_bytes()); + Ok(format!("{encoded}.{signature}")) +} + +pub fn verify_workflow_task_token( + signing_key: &[u8], + token: &str, + expected_run_id: &str, + expected_task_id: &str, + now_unix: i64, +) -> bool { + decode_workflow_task_token(signing_key, token, now_unix).is_some_and(|identity| { + identity.run_id == expected_run_id && identity.task_id == expected_task_id + }) +} + +pub fn decode_workflow_task_token( + signing_key: &[u8], + token: &str, + now_unix: i64, +) -> Option { + let (encoded, signature) = token.split_once('.')?; + if signature.contains('.') { + return None; + } + let Ok(signature) = URL_SAFE_NO_PAD.decode(signature) else { + return None; + }; + let Ok(mut mac) = HmacSha256::new_from_slice(signing_key) else { + return None; + }; + mac.update(encoded.as_bytes()); + if mac.verify_slice(&signature).is_err() { + return None; + } + let Ok(payload) = URL_SAFE_NO_PAD.decode(encoded) else { + return None; + }; + let Ok(claims) = serde_json::from_slice::(&payload) else { + return None; + }; + if claims.version != WORKFLOW_TASK_TOKEN_VERSION || claims.expires_at < now_unix { + return None; + } + Some(WorkflowTaskTokenIdentity { + run_id: claims.run_id, + task_id: claims.task_id, + }) +} + +fn workflow_task_token(run_id: &str, task_id: &str) -> Result { + let key = workflow_task_signing_key_from_env()?; + let expires_at = OffsetDateTime::now_utc() + .checked_add(time::Duration::seconds( + WORKFLOW_TASK_TOKEN_TTL.as_secs() as i64 + )) + .ok_or_else(|| WorkflowRuntimeError::Internal("workflow token expiry overflow".to_owned()))? + .unix_timestamp(); + mint_workflow_task_token(&key, run_id, task_id, expires_at) +} + +fn select_workflow_task_signing_key( + workflow_key: Option, + control_key: Option, +) -> Option> { + workflow_key + .filter(|key| !key.trim().is_empty()) + .or_else(|| control_key.filter(|key| !key.trim().is_empty())) + .map(String::into_bytes) +} + +pub fn workflow_task_signing_key_from_env() -> Result, WorkflowRuntimeError> { + select_workflow_task_signing_key( + env::var(WORKFLOW_API_KEY_ENV).ok(), + env::var(CONTROL_API_KEY_ENV).ok(), + ) + .ok_or_else(|| { + WorkflowRuntimeError::Internal(format!( + "{WORKFLOW_API_KEY_ENV} or {CONTROL_API_KEY_ENV} is required to sign workflow task authorization" + )) + }) +} struct WorkflowTaskHeartbeatGuard { task: JoinHandle<()>, @@ -93,11 +249,11 @@ struct WorkflowRuntimeInner { slack_live_client: Client, etl_client: Client, etl_backfill_client: Client, - _worker: Worker, - _slack_live_worker: Worker, - _etl_worker: Worker, - _etl_backfill_worker: Worker, - _schedule_worker: Worker, + workers: StdMutex>>, + metadata_reconciler: StdMutex>>, + draining: AtomicBool, + workers_closed: AtomicBool, + workers_close_notify: Notify, webhook_registry: Arc>>, schedule_registry: Arc>>, } @@ -207,6 +363,13 @@ impl WorkflowHostSandboxRuntime { pub fn new(runtime: SandboxRuntime, spec: SandboxSpec) -> Self { Self { runtime, spec } } + + /// Reuse api-rs's process-wide sandbox manager so the deployment drain + /// inventories workflow-host sandboxes as well as session sandboxes. + pub fn with_runtime(mut self, runtime: SandboxRuntime) -> Self { + self.runtime = runtime; + self + } } #[derive(Clone, Debug, Deserialize, Serialize)] @@ -236,8 +399,6 @@ pub struct WorkflowRun { pub run_id: String, pub task_id: String, pub workflow_name: String, - #[serde(skip_serializing_if = "Option::is_none")] - pub thread_key: Option, pub status: String, pub input: Value, pub result: Option, @@ -249,16 +410,6 @@ pub struct WorkflowRun { pub updated_at: OffsetDateTime, } -#[derive(Clone, Debug, Deserialize, Serialize)] -pub struct WorkflowCheckpoint { - pub checkpoint_name: String, - pub state: Value, - #[serde(skip_serializing_if = "Option::is_none")] - pub owner_run_id: Option, - #[serde(with = "time::serde::rfc3339")] - pub updated_at: OffsetDateTime, -} - #[derive(Clone, Debug, Deserialize, Serialize)] pub struct RegisteredWorkflowWebhook { pub workflow_name: String, @@ -630,7 +781,7 @@ impl WorkflowRuntime { "started absurd workflow schedule worker" ); - if let Some(interval) = workflow_reconcile_interval() { + let metadata_reconciler = workflow_reconcile_interval().map(|interval| { spawn_workflow_metadata_reconciler( schedule_client.clone(), WorkflowQueueClients { @@ -642,8 +793,8 @@ impl WorkflowRuntime { webhook_registry.clone(), schedule_registry.clone(), interval, - ); - } + ) + }); Ok(Self { inner: Arc::new(WorkflowRuntimeInner { @@ -651,11 +802,17 @@ impl WorkflowRuntime { slack_live_client, etl_client, etl_backfill_client, - _worker: worker, - _slack_live_worker: slack_live_worker, - _etl_worker: etl_worker, - _etl_backfill_worker: etl_backfill_worker, - _schedule_worker: schedule_worker, + workers: StdMutex::new(Some(vec![ + worker, + slack_live_worker, + etl_worker, + etl_backfill_worker, + schedule_worker, + ])), + metadata_reconciler: StdMutex::new(metadata_reconciler), + draining: AtomicBool::new(false), + workers_closed: AtomicBool::new(false), + workers_close_notify: Notify::new(), webhook_registry, schedule_registry, }), @@ -666,6 +823,7 @@ impl WorkflowRuntime { &self, request: CreateWorkflowRunRequest, ) -> Result { + self.ensure_accepting_work()?; let workflow_name = request.workflow_name.trim(); if workflow_name.is_empty() { return Err(WorkflowRuntimeError::BadRequest( @@ -698,79 +856,32 @@ impl WorkflowRuntime { }) } - pub async fn list_runs(&self, limit: i64) -> Result, WorkflowRuntimeError> { - let limit = limit.clamp(1, 200); - let mut runs = Vec::new(); - runs.extend(self.list_runs_for_queue(WORKFLOW_QUEUE, limit).await?); - runs.extend( - self.list_runs_for_queue(WORKFLOW_SLACK_LIVE_QUEUE, limit) - .await?, - ); - runs.extend(self.list_runs_for_queue(WORKFLOW_ETL_QUEUE, limit).await?); - runs.extend( - self.list_runs_for_queue(WORKFLOW_ETL_BACKFILL_QUEUE, limit) - .await?, - ); - runs.sort_by(|a, b| { - b.created_at - .cmp(&a.created_at) - .then(b.task_id.cmp(&a.task_id)) - }); - runs.truncate(limit as usize); - Ok(runs) - } - - pub async fn list_runs_filtered( + pub async fn list_runs( &self, limit: i64, workflow_name: Option<&str>, thread_key: Option<&str>, - status: Option<&str>, - parent_run_id: Option<&str>, ) -> Result, WorkflowRuntimeError> { let limit = limit.clamp(1, 200); let mut runs = Vec::new(); runs.extend( - self.list_runs_for_queue_filtered( - WORKFLOW_QUEUE, - limit, - workflow_name, - thread_key, - status, - parent_run_id, - ) - .await?, + self.list_runs_for_queue(WORKFLOW_QUEUE, limit, workflow_name, thread_key) + .await?, ); runs.extend( - self.list_runs_for_queue_filtered( - WORKFLOW_SLACK_LIVE_QUEUE, - limit, - workflow_name, - thread_key, - status, - parent_run_id, - ) - .await?, + self.list_runs_for_queue(WORKFLOW_SLACK_LIVE_QUEUE, limit, workflow_name, thread_key) + .await?, ); runs.extend( - self.list_runs_for_queue_filtered( - WORKFLOW_ETL_QUEUE, - limit, - workflow_name, - thread_key, - status, - parent_run_id, - ) - .await?, + self.list_runs_for_queue(WORKFLOW_ETL_QUEUE, limit, workflow_name, thread_key) + .await?, ); runs.extend( - self.list_runs_for_queue_filtered( + self.list_runs_for_queue( WORKFLOW_ETL_BACKFILL_QUEUE, limit, workflow_name, thread_key, - status, - parent_run_id, ) .await?, ); @@ -787,42 +898,8 @@ impl WorkflowRuntime { &self, queue_name: &str, limit: i64, - ) -> Result, WorkflowRuntimeError> { - let (task_table, run_table) = absurd_queue_tables(queue_name)?; - let rows = sqlx::query(&format!( - r#" - select - r.run_id::text as run_id, - t.task_id::text as task_id, - t.task_name, - t.params, - t.state, - t.attempts, - t.completed_payload, - r.failure_reason, - t.enqueue_at as created_at, - greatest(t.enqueue_at, coalesce(r.available_at, t.enqueue_at)) as updated_at - from {task_table} t - join {run_table} r on r.run_id = t.last_attempt_run - order by t.enqueue_at desc, t.task_id desc - limit $1 - "#, - )) - .bind(limit) - .fetch_all(self.inner.client.pool()) - .await?; - - rows.into_iter().map(workflow_run_from_row).collect() - } - - async fn list_runs_for_queue_filtered( - &self, - queue_name: &str, - limit: i64, workflow_name: Option<&str>, thread_key: Option<&str>, - status: Option<&str>, - parent_run_id: Option<&str>, ) -> Result, WorkflowRuntimeError> { let (task_table, run_table) = absurd_queue_tables(queue_name)?; let rows = sqlx::query(&format!( @@ -842,8 +919,6 @@ impl WorkflowRuntime { join {run_table} r on r.run_id = t.last_attempt_run where ($2::text is null or t.params->>'workflow_name' = $2) and ($3::text is null or t.params->'input'->>'thread_key' = $3) - and ($4::text is null or t.state = $4) - and ($5::text is null or t.params->'input'->'metadata'->>'parent_run_id' = $5) order by t.enqueue_at desc, t.task_id desc limit $1 "#, @@ -851,8 +926,6 @@ impl WorkflowRuntime { .bind(limit) .bind(workflow_name) .bind(thread_key) - .bind(status) - .bind(parent_run_id) .fetch_all(self.inner.client.pool()) .await?; @@ -918,55 +991,12 @@ impl WorkflowRuntime { Err(WorkflowRuntimeError::NotFound(run_id.to_owned())) } - pub async fn get_run_checkpoints( - &self, - run_id: &str, - ) -> Result, WorkflowRuntimeError> { - for queue_name in [ - WORKFLOW_QUEUE, - WORKFLOW_SLACK_LIVE_QUEUE, - WORKFLOW_ETL_QUEUE, - WORKFLOW_ETL_BACKFILL_QUEUE, - ] { - if let Some(run) = self.get_run_for_queue(queue_name, run_id).await? { - return self - .get_run_checkpoints_for_queue(queue_name, &run.task_id) - .await; - } - } - Err(WorkflowRuntimeError::NotFound(run_id.to_owned())) - } - - async fn get_run_checkpoints_for_queue( - &self, - queue_name: &str, - task_id: &str, - ) -> Result, WorkflowRuntimeError> { - let checkpoint_table = absurd_checkpoint_table(queue_name)?; - let rows = sqlx::query(&format!( - r#" - select - checkpoint_name, - state, - owner_run_id::text as owner_run_id, - updated_at - from {checkpoint_table} - where task_id = $1::uuid - order by updated_at asc, checkpoint_name asc - "#, - )) - .bind(task_id) - .fetch_all(self.inner.client.pool()) - .await?; - - rows.into_iter().map(workflow_checkpoint_from_row).collect() - } - pub async fn emit_event( &self, event_name: &str, payload: Value, ) -> Result<(), WorkflowRuntimeError> { + self.ensure_accepting_work()?; self.inner .client .emit_event(event_name, payload.clone(), Some(WORKFLOW_QUEUE)) @@ -986,6 +1016,70 @@ impl WorkflowRuntime { Ok(()) } + fn ensure_accepting_work(&self) -> Result<(), WorkflowRuntimeError> { + if self.inner.draining.load(Ordering::SeqCst) { + return Err(WorkflowRuntimeError::Disabled( + "workflow runtime is draining".to_owned(), + )); + } + Ok(()) + } + + /// Stop schedule reconciliation and every queue worker, waiting for tasks + /// already claimed by this process to finish before the session runtime's + /// irreversible sandbox drain fence is raised. This prevents a draining + /// replica from consuming retries merely to fail sandbox allocation. + pub async fn close_workers(&self) -> Result<(), WorkflowRuntimeError> { + self.inner.draining.store(true, Ordering::SeqCst); + let reconciler = self + .inner + .metadata_reconciler + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + .take(); + if let Some(reconciler) = reconciler { + reconciler.abort(); + let _ = reconciler.await; + } + let workers = self + .inner + .workers + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + .take(); + let Some(workers) = workers else { + let notified = self.inner.workers_close_notify.notified(); + if self.inner.workers_closed.load(Ordering::SeqCst) { + return Ok(()); + } + let _ = tokio::time::timeout(WORKFLOW_WORKER_CLOSE_TIMEOUT, notified).await; + return Ok(()); + }; + let results = join_all( + workers + .into_iter() + .map(|worker| worker.close_with_timeout(WORKFLOW_WORKER_CLOSE_TIMEOUT)), + ) + .await; + let mut result = Ok(()); + for worker_result in results { + match worker_result { + Ok(true) => {} + Ok(false) => warn!( + timeout_seconds = WORKFLOW_WORKER_CLOSE_TIMEOUT.as_secs(), + "aborted workflow worker and active handlers after close timeout" + ), + Err(error) => { + result = Err(error.into()); + break; + } + } + } + self.inner.workers_closed.store(true, Ordering::SeqCst); + self.inner.workers_close_notify.notify_waiters(); + result + } + pub fn get_webhook(&self, slug: &str) -> Option { self.inner .webhook_registry @@ -1083,19 +1177,6 @@ fn absurd_queue_tables( } } -fn absurd_checkpoint_table(queue_name: &str) -> Result<&'static str, WorkflowRuntimeError> { - match queue_name { - WORKFLOW_QUEUE => Ok("absurd.c_centaur_workflows"), - WORKFLOW_SLACK_LIVE_QUEUE => Ok("absurd.c_centaur_workflows_slack_live"), - WORKFLOW_ETL_QUEUE => Ok("absurd.c_centaur_workflows_etl"), - WORKFLOW_ETL_BACKFILL_QUEUE => Ok("absurd.c_centaur_workflows_etl_backfill"), - WORKFLOW_SCHEDULE_QUEUE => Ok("absurd.c_centaur_workflow_schedules"), - other => Err(WorkflowRuntimeError::Internal(format!( - "unknown workflow queue {other:?}" - ))), - } -} - fn build_webhook_registry( discovery: &PythonWorkflowMetadata, enablement: &WorkflowEnablement, @@ -1822,7 +1903,7 @@ fn spawn_workflow_metadata_reconciler( webhook_registry: Arc>>, schedule_registry: Arc>>, interval: Duration, -) { +) -> JoinHandle<()> { tokio::spawn(async move { let mut ticker = tokio::time::interval(interval); let mut reaper = RemovedWorkflowReaper::from_env(); @@ -1863,7 +1944,7 @@ fn spawn_workflow_metadata_reconciler( Err(error) => warn!(%error, "failed to reconcile workflow metadata"), } } - }); + }) } async fn reconcile_workflow_metadata_once( @@ -2689,6 +2770,51 @@ struct WorkflowSandboxCleanupGuard { workflow_run_id: String, } +struct WorkflowHostSandboxCleanupGuard { + runtime: Option, + sandbox_id: centaur_sandbox_core::SandboxId, +} + +impl WorkflowHostSandboxCleanupGuard { + fn new(runtime: SandboxRuntime, sandbox_id: centaur_sandbox_core::SandboxId) -> Self { + Self { + runtime: Some(runtime), + sandbox_id, + } + } + + async fn cleanup(&mut self) { + let Some(runtime) = self.runtime.take() else { + return; + }; + if let Err(error) = runtime.stop_sandbox(&self.sandbox_id).await { + warn!( + sandbox_id = %self.sandbox_id.as_str(), + %error, + "failed to stop workflow host sandbox" + ); + } + } +} + +impl Drop for WorkflowHostSandboxCleanupGuard { + fn drop(&mut self) { + let Some(runtime) = self.runtime.take() else { + return; + }; + let sandbox_id = self.sandbox_id.clone(); + tokio::spawn(async move { + if let Err(error) = runtime.stop_sandbox(&sandbox_id).await { + warn!( + sandbox_id = %sandbox_id.as_str(), + %error, + "failed to stop dropped workflow host sandbox" + ); + } + }); + } +} + impl WorkflowSandboxCleanupGuard { fn new(session_runtime: SessionRuntime, workflow_run_id: String) -> Self { Self { @@ -2775,6 +2901,7 @@ async fn run_python_workflow_host_local( session_runtime: SessionRuntime, ) -> Result { let host_path = python_workflow_host_path(); + let task_token = workflow_task_token(ctx.run_id(), ctx.task_id())?; let mut command = Command::new( env::var(PYTHON_HOST_INTERPRETER_ENV).unwrap_or_else(|_| "python3".to_owned()), ); @@ -2782,10 +2909,16 @@ async fn run_python_workflow_host_local( .arg(&host_path) .stdin(std::process::Stdio::piped()) .stdout(std::process::Stdio::piped()) - .stderr(std::process::Stdio::piped()); + .stderr(std::process::Stdio::piped()) + .kill_on_drop(true) + .env("WORKFLOW_RUN_ID", ctx.run_id()) + .env("WORKFLOW_TASK_ID", ctx.task_id()) + .env(WORKFLOW_TASK_TOKEN_ENV, task_token) + .env("WORKFLOW_NAME", &input.workflow_name); if env::var_os("WORKFLOW_DIRS").is_none() { command.env("WORKFLOW_DIRS", default_workflow_dirs()); } + remove_local_workflow_host_credentials(&mut command); let mut child = command.spawn().map_err(|error| { WorkflowRuntimeError::Internal(format!( @@ -2814,34 +2947,91 @@ async fn run_python_workflow_host_local( collected.join("\n") }); - let result = run_python_workflow_host_protocol( - input, - ctx, - session_runtime, + write_host_message( &mut stdin, - stdout, - stderr_task, + &json!({ + "type": "workflow.start", + "run_id": ctx.run_id(), + "task_id": ctx.task_id(), + "workflow_name": input.workflow_name, + "input": input.input, + }), ) - .await; - drop(stdin); - if result.is_err() { - cleanup_local_python_workflow_host(&mut child).await; - } else { - let _ = child.wait().await; - } - result -} + .await?; -async fn cleanup_local_python_workflow_host(child: &mut tokio::process::Child) { - match child.try_wait() { - Ok(Some(_)) => return, - Ok(None) => {} - Err(error) => { - warn!(%error, "failed to inspect local Python workflow host before cleanup"); + let mut lines = BufReader::new(stdout).lines(); + while let Some(line) = lines.next_line().await? { + if line.trim().is_empty() { + continue; + } + let message: Value = serde_json::from_str(&line)?; + match message.get("type").and_then(Value::as_str) { + Some("workflow.result") => { + drop(stdin); + let _ = child.wait().await; + return Ok(message.get("result").cloned().unwrap_or(Value::Null)); + } + Some("workflow.error") | Some("host.error") => { + let stderr = stderr_task.await.unwrap_or_default(); + return Err(WorkflowRuntimeError::Internal(format!( + "Python workflow host error: {}{}{}", + message + .get("message") + .and_then(Value::as_str) + .unwrap_or("unknown error"), + if stderr.is_empty() { "" } else { "\nstderr:\n" }, + stderr, + ))); + } + Some("ctx.log") => { + let workflow_log = message + .get("message") + .and_then(|value| value.as_str()) + .unwrap_or("workflow_log"); + info!( + workflow_log = %workflow_log, + fields = %message.get("fields").cloned().unwrap_or_else(|| json!({})), + task_id = ctx.task_id(), + run_id = ctx.run_id(), + "python workflow log" + ); + } + Some("ctx.metric") => { + record_python_workflow_metric(&message); + } + Some(message_type) if message_type.starts_with("ctx.") => { + let response = + match handle_python_context_request(&message, &ctx, &session_runtime, &input) + .await + { + Ok(response) => response, + Err(error) => { + drop(stdin); + let _ = child.start_kill(); + let _ = child.wait().await; + return Err(error); + } + }; + write_host_message(&mut stdin, &response).await?; + } + other => { + return Err(WorkflowRuntimeError::Internal(format!( + "unexpected Python workflow host message type {other:?}: {message}" + ))); + } } } - if let Err(error) = child.kill().await { - warn!(%error, "failed to kill local Python workflow host after protocol error"); + + let status = child.wait().await?; + let stderr = stderr_task.await.unwrap_or_default(); + Err(WorkflowRuntimeError::Internal(format!( + "Python workflow host exited before workflow.result: status={status}, stderr={stderr}" + ))) +} + +fn remove_local_workflow_host_credentials(command: &mut Command) { + for name in LOCAL_WORKFLOW_HOST_DENIED_ENVS { + command.env_remove(name); } } @@ -2851,10 +3041,12 @@ async fn run_python_workflow_host_in_sandbox( session_runtime: SessionRuntime, sandbox: WorkflowHostSandboxRuntime, ) -> Result { + let task_token = workflow_task_token(ctx.run_id(), ctx.task_id())?; let mut spec = sandbox.spec.clone(); spec = spec .env("WORKFLOW_RUN_ID", ctx.run_id()) .env("WORKFLOW_TASK_ID", ctx.task_id()) + .env(WORKFLOW_TASK_TOKEN_ENV, task_token) .env("WORKFLOW_NAME", input.workflow_name.clone()); if env::var_os("WORKFLOW_DIRS").is_none() && !sandbox_spec_has_env(&spec, "WORKFLOW_DIRS") { spec = spec.env("WORKFLOW_DIRS", default_workflow_dirs()); @@ -2864,7 +3056,14 @@ async fn run_python_workflow_host_in_sandbox( { spec = spec.env("DATABASE_URL", database_url); } + // Cross the same allocation barrier as normal session sandboxes. Drain + // sets an irreversible fence, waits for this permit, and only then takes + // its backend inventory, so a workflow retry cannot escape the drain. + let allocation_permit = session_runtime.acquire_sandbox_allocation_permit().await?; let (sandbox_id, io) = sandbox.runtime.create_running_io(spec).await?; + drop(allocation_permit); + let mut sandbox_cleanup = + WorkflowHostSandboxCleanupGuard::new(sandbox.runtime.clone(), sandbox_id.clone()); let mut stdin = io.stdin; let stderr_task = tokio::spawn(async move { let _guard = io.guard; @@ -2885,9 +3084,7 @@ async fn run_python_workflow_host_in_sandbox( ) .await; drop(stdin); - if let Err(error) = sandbox.runtime.stop_sandbox(&sandbox_id).await { - warn!(sandbox_id = %sandbox_id.as_str(), %error, "failed to stop workflow host sandbox"); - } + sandbox_cleanup.cleanup().await; result } @@ -2959,9 +3156,7 @@ where } Some(message_type) if message_type.starts_with("ctx.") => { let response = - handle_python_context_request(&message, &ctx, &session_runtime, &input) - .await - .map_err(absurd_to_workflow_error)?; + handle_python_context_request(&message, &ctx, &session_runtime, &input).await?; write_host_message(stdin, &response).await?; } other => { @@ -3091,7 +3286,7 @@ async fn handle_python_context_request( ctx: &TaskContext, session_runtime: &SessionRuntime, input: &WorkflowTaskInput, -) -> absurd::Result { +) -> Result { let request_id = message .get("request_id") .and_then(Value::as_str) @@ -3138,50 +3333,29 @@ async fn handle_python_context_request( } } } - Some("ctx.sleep_until") => { + Some("ctx.sleep") => { let step = message .get("step") .and_then(Value::as_str) .unwrap_or("sleep"); - let wake_at = message - .get("wake_at") - .and_then(Value::as_str) - .ok_or_else(|| "ctx.sleep_until missing wake_at".to_owned()) - .and_then(|value| { - DateTime::parse_from_rfc3339(value) - .map(|parsed| parsed.with_timezone(&Utc)) - .map_err(|error| format!("invalid ctx.sleep_until wake_at: {error}")) - }); - match wake_at { - Ok(wake_at) => match ctx.sleep_until(step, wake_at).await { - Ok(()) => Ok(json!({ "slept": true })), - Err(absurd::Error::Suspend) => return Err(absurd::Error::Suspend), + match parse_python_duration_seconds(message) { + Ok(duration) => match ctx.sleep_for(step, duration).await { + Ok(()) => Ok(json!({"slept": true})), + Err(absurd::Error::Suspend) => return Err(WorkflowRuntimeError::Suspend), Err(error) => Err(error.to_string()), }, Err(error) => Err(error), } } - Some("ctx.sleep_for") => { + Some("ctx.sleep_until") => { let step = message .get("step") .and_then(Value::as_str) - .unwrap_or("sleep"); - let seconds = message - .get("seconds") - .and_then(Value::as_f64) - .ok_or_else(|| "ctx.sleep_for missing seconds".to_owned()) - .and_then(|value| { - if !value.is_finite() || value < 0.0 { - return Err( - "ctx.sleep_for seconds must be a non-negative finite number".to_owned() - ); - } - Ok(Duration::from_secs_f64(value)) - }); - match seconds { - Ok(duration) => match ctx.sleep_for(step, duration).await { - Ok(()) => Ok(json!({ "slept": true })), - Err(absurd::Error::Suspend) => return Err(absurd::Error::Suspend), + .unwrap_or("sleep_until"); + match parse_python_wake_at(message) { + Ok(wake_at) => match ctx.sleep_until(step, wake_at).await { + Ok(()) => Ok(json!({"slept": true})), + Err(absurd::Error::Suspend) => return Err(WorkflowRuntimeError::Suspend), Err(error) => Err(error.to_string()), }, Err(error) => Err(error), @@ -3208,27 +3382,41 @@ async fn handle_python_context_request( } other => Err(format!("unsupported context request type {other:?}")), }; - match result { - Ok(value) => Ok(json!({ + Ok(match result { + Ok(value) => json!({ "type": "ctx.response", "request_id": request_id, "ok": true, "value": value, - })), - Err(error) => Ok(json!({ + }), + Err(error) => json!({ "type": "ctx.response", "request_id": request_id, "ok": false, "error": error, - })), - } + }), + }) } -fn absurd_to_workflow_error(error: absurd::Error) -> WorkflowRuntimeError { - match error { - absurd::Error::Suspend => WorkflowRuntimeError::Suspended, - other => WorkflowRuntimeError::Internal(other.to_string()), +fn parse_python_duration_seconds(message: &Value) -> Result { + let seconds = message + .get("duration_seconds") + .and_then(Value::as_f64) + .ok_or_else(|| "ctx.sleep missing numeric duration_seconds".to_owned())?; + if !seconds.is_finite() || seconds < 0.0 { + return Err("ctx.sleep duration_seconds must be a finite non-negative number".to_owned()); } + Ok(Duration::from_secs_f64(seconds)) +} + +fn parse_python_wake_at(message: &Value) -> Result, String> { + let raw = message + .get("wake_at") + .and_then(Value::as_str) + .ok_or_else(|| "ctx.sleep_until missing wake_at".to_owned())?; + DateTime::parse_from_rfc3339(raw) + .map(|value| value.with_timezone(&Utc)) + .map_err(|error| format!("ctx.sleep_until invalid wake_at: {error}")) } async fn run_python_agent_turn( @@ -3447,13 +3635,11 @@ async fn call_python_workflow_tool(message: &Value) -> Result String { fn workflow_run_from_row(row: sqlx::postgres::PgRow) -> Result { let params: Value = row.try_get("params")?; let input = params.get("input").cloned().unwrap_or(Value::Null); - let thread_key = input - .get("thread_key") - .and_then(Value::as_str) - .map(ToOwned::to_owned); let workflow_name = params .get("workflow_name") .and_then(Value::as_str) @@ -3759,7 +3941,6 @@ fn workflow_run_from_row(row: sqlx::postgres::PgRow) -> Result Result Result { - Ok(WorkflowCheckpoint { - checkpoint_name: row.try_get("checkpoint_name")?, - state: row.try_get("state")?, - owner_run_id: row.try_get("owner_run_id")?, - updated_at: row.try_get("updated_at")?, - }) -} - fn absurd_error(error: WorkflowRuntimeError) -> absurd::Error { match error { - WorkflowRuntimeError::Suspended => absurd::Error::Suspend, + WorkflowRuntimeError::Suspend => absurd::Error::Suspend, other => absurd::Error::TaskFailed(Box::new(other)), } } #[derive(Debug, Error)] pub enum WorkflowRuntimeError { + #[error("workflow suspended")] + Suspend, /// The caller supplied an invalid request or workflow configuration. /// Maps to HTTP 400. #[error("{0}")] @@ -3800,8 +3972,6 @@ pub enum WorkflowRuntimeError { Disabled(String), #[error("workflow run not found: {0}")] NotFound(String), - #[error("workflow suspended")] - Suspended, /// Server-side failure (workflow host spawn/protocol, internal dispatch). /// Maps to HTTP 500. #[error("{0}")] @@ -3833,6 +4003,81 @@ mod tests { use super::*; use chrono::TimeZone; + #[test] + fn workflow_task_tokens_are_signed_scoped_and_expiring() { + let token = mint_workflow_task_token(b"a-distinct-workflow-key", "run-1", "task-1", 200) + .expect("mint task token"); + assert!(verify_workflow_task_token( + b"a-distinct-workflow-key", + &token, + "run-1", + "task-1", + 100, + )); + assert_eq!( + decode_workflow_task_token(b"a-distinct-workflow-key", &token, 100), + Some(WorkflowTaskTokenIdentity { + run_id: "run-1".to_owned(), + task_id: "task-1".to_owned(), + }) + ); + assert!(!verify_workflow_task_token( + b"a-distinct-workflow-key", + &token, + "run-2", + "task-1", + 100, + )); + assert!(!verify_workflow_task_token( + b"a-distinct-workflow-key", + &token, + "run-1", + "task-1", + 201, + )); + assert!(!verify_workflow_task_token( + b"wrong-key", + &token, + "run-1", + "task-1", + 100, + )); + } + + #[test] + fn workflow_task_signing_key_prefers_dedicated_key_and_falls_back_to_control() { + assert_eq!( + select_workflow_task_signing_key( + Some("workflow-key".to_owned()), + Some("control-key".to_owned()) + ), + Some(b"workflow-key".to_vec()) + ); + assert_eq!( + select_workflow_task_signing_key(None, Some("control-key".to_owned())), + Some(b"control-key".to_vec()) + ); + } + + #[test] + fn local_workflow_host_explicitly_removes_ambient_service_credentials() { + let mut command = Command::new("workflow-host-test"); + for name in LOCAL_WORKFLOW_HOST_DENIED_ENVS { + command.env(name, "must-not-reach-child"); + } + + remove_local_workflow_host_credentials(&mut command); + + let configured = command.as_std().get_envs().collect::>(); + for name in LOCAL_WORKFLOW_HOST_DENIED_ENVS { + assert!( + configured + .iter() + .any(|(key, value)| { *key == std::ffi::OsStr::new(name) && value.is_none() }) + ); + } + } + #[test] fn parse_worker_concurrency_uses_override_or_default() { // Override wins. @@ -4047,7 +4292,6 @@ mod tests { run_id: "run".to_owned(), task_id: "task".to_owned(), workflow_name: "workflow".to_owned(), - thread_key: None, status: "completed".to_owned(), input: json!({}), result: None, diff --git a/services/api-rs/rfcs/0004-console-mcp-jwt-auth.md b/services/api-rs/rfcs/0004-console-mcp-jwt-auth.md new file mode 100644 index 000000000..2760693b6 --- /dev/null +++ b/services/api-rs/rfcs/0004-console-mcp-jwt-auth.md @@ -0,0 +1,745 @@ +# RFC 0004: Console OAuth for MCP Auth + +Status: Draft +Owner: TBD +Target: `services/console`, `services/api-rs` + +## Summary + +Make Centaur's remote MCP endpoint use the MCP HTTP authorization flow, with +console acting as the OAuth authorization server and api-rs acting as the MCP +protected resource server. + +The user should not need to copy a JWT from a console page into Amp. Instead: + +1. A harness connects to `POST /mcp` without a token. +2. api-rs returns `401 Unauthorized` with a `WWW-Authenticate: Bearer ...` + challenge that points at MCP Protected Resource Metadata. +3. The harness fetches the Protected Resource Metadata and learns that console + is the authorization server. +4. The harness discovers console's OAuth metadata. +5. The harness registers as an OAuth client, or uses preconfigured client + metadata. +6. The harness opens a browser to console's authorization endpoint with PKCE. +7. The user signs in with the normal console login/SSO flow. +8. Console ensures the signed-in user has an iron-control principal and returns + an authorization code to the harness. +9. The harness exchanges the code for a bearer access token. +10. The harness calls `POST /mcp` with `Authorization: Bearer `. +11. api-rs verifies the token and uses the encoded principal for MCP tool + execution. + +This matches the MCP authorization model used by HTTP-based MCP clients and +harnesses, while keeping console as the identity and permission UX. + +## Motivation + +The current MCP branch exposes the HTTP MCP transport and persistent tool +runners without a user-facing auth flow. That keeps the transport work small, +but MCP clients still need a standard way to sign in and bind tool execution to +the right console principal. + +Remote MCP clients already know how to follow an OAuth-style authorization +flow. We should use that instead of asking users to paste bearer tokens. + +Console already owns: + +- user login and SSO +- user approval/disable state +- principals, roles, grants, and effective permissions +- the operator UI where users can understand what identity they are using + +MCP auth should use that surface. + +The important product property is that MCP permissions are controlled by the +principal. The access token should identify the principal. It should not copy +the principal's current grants into the token. If an operator changes the +principal's roles or grants, the next per-user tool runner/proxy sync should see +the updated permissions without reissuing the token. + +## Goals + +- Use MCP-standard HTTP authorization so Amp and other harnesses can start auth + themselves. +- Keep the existing MCP endpoint as `POST /mcp`. +- Keep api-rs as the MCP protected resource server. +- Make console the OAuth authorization server for Centaur MCP. +- Use console login/SSO as the user authentication ceremony. +- Return bearer access tokens from a token endpoint, not from a copy-token page. +- Encode the iron-control principal id in the access token. +- Keep MCP authorization based on live principal grants in iron-control. +- Keep the current per-principal persistent MCP tool runner model. +- Support Dynamic Client Registration initially, since generic MCP clients may + not be pre-registered with our console. + +## Non-Goals + +- Implement a local bridge. +- Make Slackbot the long-term issuer of MCP credentials. +- Put live credential grants or secret names inside access tokens. +- Build a full general-purpose OAuth provider for arbitrary third-party apps. +- Support every OAuth client authentication method in the first version. +- Require Tailscale identity for MCP auth. + +## Current State + +The MCP branch has: + +- `POST /mcp` in api-rs. +- No MCP token issuer. +- An unauthenticated transport path used as the base for this authorization + work. +- Persistent tool runners keyed by `principal_id`. + +The persistent runner already wants the iron-control principal id. For a +proxied tool, api-rs creates or reuses a runner whose sandbox spec carries: + +```text +iron_control_principal = +CENTAUR_MCP_PRINCIPAL_ID = +``` + +That means the access token should encode the iron-control principal id, not +only the console user id or email. + +## Protocol Design + +### Roles + +Centaur maps MCP/OAuth roles as follows: + +| Role | Centaur Component | +|------|-------------------| +| MCP protected resource server | api-rs `/mcp` | +| OAuth authorization server | console | +| OAuth client | Amp, Codex, VS Code, or another MCP harness/client | +| Resource owner | signed-in console user | + +### Discovery Flow + +When a harness calls `/mcp` without a valid token, api-rs returns: + +```http +HTTP/1.1 401 Unauthorized +WWW-Authenticate: Bearer realm="mcp", + resource_metadata="https://api.example.com/.well-known/oauth-protected-resource/mcp", + scope="mcp:tools" +``` + +api-rs serves Protected Resource Metadata at both: + +```text +/.well-known/oauth-protected-resource +/.well-known/oauth-protected-resource/mcp +``` + +Example: + +```json +{ + "resource": "https://api.example.com/mcp", + "authorization_servers": ["https://console.example.com"], + "scopes_supported": ["mcp:tools"] +} +``` + +The `resource` value must be the canonical externally visible MCP endpoint. +For local preview dogfooding this can be `http://localhost:3000/mcp`; for +production it should be the public HTTPS MCP URL. + +The harness then fetches authorization server metadata from console. + +Console should serve OAuth Authorization Server Metadata at: + +```text +/.well-known/oauth-authorization-server +``` + +Optionally, console can also serve: + +```text +/.well-known/openid-configuration +``` + +Example metadata: + +```json +{ + "issuer": "https://console.example.com", + "authorization_endpoint": "https://console.example.com/mcp/oauth/authorize", + "token_endpoint": "https://console.example.com/mcp/oauth/token", + "registration_endpoint": "https://console.example.com/mcp/oauth/register", + "response_types_supported": ["code"], + "grant_types_supported": ["authorization_code", "refresh_token"], + "code_challenge_methods_supported": ["S256"], + "scopes_supported": ["mcp:tools"], + "token_endpoint_auth_methods_supported": ["none"], + "resource_indicators_supported": true +} +``` + +### Client Registration + +First version should support Dynamic Client Registration because generic MCP +harnesses may not have a pre-registered Centaur client id. + +Console endpoint: + +```text +POST /mcp/oauth/register +``` + +Allowed registration shape: + +```json +{ + "client_name": "Amp", + "redirect_uris": ["http://127.0.0.1:49152/callback"], + "grant_types": ["authorization_code", "refresh_token"], + "response_types": ["code"], + "token_endpoint_auth_method": "none" +} +``` + +Console returns: + +```json +{ + "client_id": "mcp_client_...", + "client_name": "Amp", + "redirect_uris": ["http://127.0.0.1:49152/callback"], + "grant_types": ["authorization_code", "refresh_token"], + "response_types": ["code"], + "token_endpoint_auth_method": "none", + "client_id_issued_at": 1782749000 +} +``` + +Registration constraints: + +- public clients only in v1 (`token_endpoint_auth_method = "none"`) +- require PKCE S256 at authorize/token time +- allow loopback redirect URIs (`http://127.0.0.1`, `http://localhost`, `[::1]`) +- allow HTTPS redirect URIs +- reject wildcard redirect URIs +- reject non-loopback plain HTTP redirect URIs +- store only client metadata and timestamps, not user authorization + +Future versions can add OAuth Client ID Metadata Documents. Do not advertise +`client_id_metadata_document_supported` until console actually validates those +documents. + +### Authorization Endpoint + +Console endpoint: + +```text +GET /mcp/oauth/authorize +``` + +Required parameters: + +```text +response_type=code +client_id= +redirect_uri= +code_challenge= +code_challenge_method=S256 +resource= +scope=mcp:tools +state= +``` + +Behavior: + +- If signed out, redirect through existing console login/SSO and return to the + authorize request. +- If the console user is pending or disabled, deny authorization. +- Validate `client_id`, `redirect_uri`, `scope`, `resource`, and PKCE params. +- Ensure the signed-in user has an MCP principal. +- Optionally show a compact consent/confirmation page. +- Create a short-lived one-time authorization code. +- Redirect to the client `redirect_uri` with `code` and original `state`. + +The authorization code stores: + +- `client_id` +- `redirect_uri` +- `code_challenge` +- `code_challenge_method` +- `resource` +- `scope` +- `user_id` +- `principal_id` +- expiration timestamp, suggested 5 minutes +- consumed timestamp, initially null + +### Token Endpoint + +Console endpoint: + +```text +POST /mcp/oauth/token +``` + +Authorization code exchange request: + +```text +grant_type=authorization_code +code= +redirect_uri= +client_id= +code_verifier= +resource= +``` + +Console validates: + +- code exists, is unexpired, and is unused +- code belongs to `client_id` +- `redirect_uri` matches the code +- `resource` matches the code +- PKCE verifier matches the stored S256 challenge +- user is still active +- principal still exists + +Console returns: + +```json +{ + "access_token": "", + "token_type": "Bearer", + "expires_in": 3600, + "refresh_token": "cmcpr_...", + "scope": "mcp:tools" +} +``` + +The access token is a JWT signed by console with the shared Centaur signing +secret. Refresh tokens are opaque random strings stored hashed by console. + +Refresh token request: + +```text +grant_type=refresh_token +refresh_token= +client_id= +resource= +scope=mcp:tools +``` + +Refresh behavior: + +- validate refresh token hash, client id, user status, principal, resource, and + scope +- rotate refresh token on each use +- return a fresh access token + +### Access Token Claims + +Example JWT payload: + +```json +{ + "iss": "https://console.example.com", + "aud": "https://api.example.com/mcp", + "sub": "usr_abc123", + "jti": "mcp_at_018f...", + "iat": 1782749000, + "nbf": 1782749000, + "exp": 1782752600, + "scope": "mcp:tools", + "client_id": "mcp_client_abc123", + "principal_id": "prn_abc123", + "principal_foreign_id": "console-user-alice-example-com", + "principal_namespace": "default", + "email": "alice@example.com", + "name": "Alice Example" +} +``` + +Claim semantics: + +| Claim | Purpose | +|-------|---------| +| `iss` | Console issuer URL from authorization server metadata. | +| `aud` | Canonical MCP resource URI from the authorization request. | +| `sub` | Console user oid. Useful for audit/debugging. | +| `jti` | Access token instance id. Used as `token_id` in MCP whoami/logs. | +| `iat`/`nbf`/`exp` | Token lifetime. | +| `scope` | MCP protocol scopes, not credential grants. | +| `client_id` | Registered OAuth client id. | +| `principal_id` | iron-control principal oid used for tool runner/proxy binding. | +| `principal_foreign_id` | Human/debug identifier. Not authoritative for proxy binding. | +| `principal_namespace` | Human/debug namespace. | +| `email`/`name` | Display only. | + +The JWT must not include: + +- secret ids +- role ids +- grant details +- actual credential values +- provider refresh tokens + +### Principal Model + +Console creates or finds one MCP principal per active console user. + +Default mapping: + +```text +namespace: +foreign_id: console-user- +name: Console User +labels: + managed-by: centaur + principal-kind: console-user + console-user-id: + email: +``` + +The access token includes the resulting principal oid, for example `prn_...`. + +The oid is what api-rs uses to bind the per-sandbox iron-proxy. The foreign id +and email are included for diagnostics only. + +This gives us the permissions flexibility we want: + +- grant tools/secrets directly to the user's console principal +- assign roles to the user's console principal +- later change grants/roles without changing MCP tokens +- later support group/team based assignment through console policy without + changing MCP transport + +### Signing Secret + +Add one general Centaur JWT signing secret instead of an MCP-specific secret: + +```text +CENTAUR_JWT_SIGNING_SECRET +``` + +This should live in the shared infra Secret and be mounted into both console and +api-rs. + +Why not reuse Rails `SECRET_KEY_BASE`? + +- It is tied to Rails cookies and framework internals. +- Rotating it has Rails-specific blast radius. +- api-rs should not need to treat Rails session signing material as an API auth + root. +- A general Centaur JWT secret can support other future service-issued JWTs with + issuer/audience separation. + +The first version can use HS256 with this shared secret. + +JWT verification in api-rs must require: + +- known algorithm: `HS256` +- trusted issuer matching console metadata +- audience matching the canonical MCP resource URI +- `exp` in the future +- `nbf` absent or not in the future +- `iat` not unreasonably in the future +- required `principal_id` +- required `scope` containing `mcp:tools` or `mcp:*` + +Future rotation can add: + +```text +CENTAUR_JWT_SIGNING_KID +CENTAUR_JWT_VERIFYING_SECRETS +``` + +where the verifying env var is a JSON map of `kid -> secret`. + +### MCP Endpoint Auth + +api-rs should accept only console-issued OAuth access JWTs. + +```text +Authorization: Bearer + +verify as console OAuth MCP access token +``` + +The verified identity should normalize into the existing MCP principal shape: + +```text +McpAuthenticatedPrincipal { + token_id: + principal_id: + name: + scopes: ["mcp:tools"] + expires_at: +} +``` + +Everything after auth should stay the same: + +- `tools/list` checks `mcp:tools`. +- `tools/call` checks `mcp:tools`. +- proxied tools use persistent runner keyed by `principal_id`. +- the runner sandbox uses that same `principal_id` for iron-proxy. + +## Deployment Config + +Add the shared secret to the infra Secret: + +```text +CENTAUR_JWT_SIGNING_SECRET= +``` + +`just bootstrap-secrets` should generate it if absent and never rotate it in +place. + +Chart wiring: + +- api-rs already imports the shared infra Secret with `envFrom`, so it can read + `CENTAUR_JWT_SIGNING_SECRET`. +- console should explicitly mount `CENTAUR_JWT_SIGNING_SECRET` from the shared + infra Secret, because console intentionally does not use `envFrom`. +- api-rs needs a canonical MCP public URL for Protected Resource Metadata. +- console needs the same canonical MCP public URL for OAuth resource validation. +- console needs its own public issuer URL. + +Suggested env vars: + +```text +CENTAUR_JWT_SIGNING_SECRET +CENTAUR_MCP_PUBLIC_URL +CENTAUR_CONSOLE_PUBLIC_URL +CENTAUR_MCP_ACCESS_TOKEN_TTL_SECONDS +CENTAUR_MCP_REFRESH_TOKEN_TTL_SECONDS +CENTAUR_MCP_PRINCIPAL_NAMESPACE +``` + +`CENTAUR_JWT_SIGNING_SECRET` is intentionally general. The other env vars are +MCP-specific policy/display knobs. + +## Security Considerations + +### Bearer Token Risk + +The OAuth access token is a bearer token. Anyone who obtains it can use the +encoded principal's MCP permissions until it expires. + +Mitigations: + +- short access token TTL, suggested 1 hour +- refresh tokens stored hashed by console +- refresh token rotation +- no access token values in logs +- no access token persistence in console DB +- no token values in Slack messages +- no grants embedded in access tokens +- future `jti` denylist if needed + +### Disabled Users + +Console checks user status when authorizing and refreshing. + +Because access tokens are stateless, disabling a user does not automatically +invalidate already-issued access tokens until they expire. Short access token +TTL limits this window. Refresh tokens must stop working immediately for +disabled users. + +### Permission Changes + +Permission changes should not require new access tokens. + +The token identifies `principal_id`; iron-control remains the live source of +truth for grants. If a role is revoked from the principal, the next proxy sync +should remove that credential from the user's runner. + +### Audience and Resource Binding + +The API must require `aud` to match the canonical MCP resource URI. Console must +issue access tokens only for the `resource` value supplied by the client and +accepted by console policy. + +Do not accept generic audiences like `api` or `centaur`. + +### DCR Abuse + +Unauthenticated Dynamic Client Registration can be abused if unconstrained. + +Initial constraints: + +- only public clients +- loopback or HTTPS redirect URIs only +- no wildcard redirects +- no custom schemes initially +- rate limit registration +- audit client registrations +- optionally prune unused clients + +### Secret Rotation + +Initial implementation can use one shared signing secret. + +Before production reliance, add a `kid` strategy: + +- console signs with active `kid` +- api-rs verifies against active plus previous keys +- old keys stay in verify-only mode until all access tokens signed with them + expire + +## Alternatives Considered + +### Manual Copyable JWT Page + +Pros: + +- simplest to implement +- no OAuth client registration, auth codes, or token endpoint + +Cons: + +- not the MCP-supported HTTP auth flow harnesses are built around +- poor UX for Amp and other clients +- users manually handle long bearer secrets +- harder to refresh tokens cleanly + +This RFC replaces the copy-token page with MCP OAuth. + +### Keep Opaque DB Tokens + +Pros: + +- simple revocation +- simple for a Slack-first prototype + +Cons: + +- api-rs remains an issuer +- Slackbot remains an issuance UX +- every non-Slack surface needs another token flow +- console login/SSO is not the source of user identity +- not the harness-native MCP auth path + +### External OAuth Provider Only + +We could point the MCP Protected Resource Metadata directly at Okta, Google, or +another IdP. + +Pros: + +- mature OAuth implementation +- less auth code in console + +Cons: + +- the access token still needs Centaur principal claims +- we still need a principal mapping layer +- group/role policy becomes split between IdP and iron-control +- local/dev and preview flows are harder + +Console can still federate login to Okta/Google while issuing the Centaur MCP +access token itself. + +### Tailscale MCP Auth + +Pros: + +- strong device/user identity on a tailnet + +Cons: + +- not every user is on the same tailnet +- does not solve Discord or external users +- still need to map tailnet identity to iron-control principals + +### Reuse Rails `SECRET_KEY_BASE` + +Pros: + +- already exists +- console and api-rs can be wired to read it + +Cons: + +- wrong blast radius +- tied to Rails cookies/framework behavior +- not obviously safe to expose as a general API signing root + +Use `CENTAUR_JWT_SIGNING_SECRET` instead. + +## Rollout Plan + +1. Add this RFC. +2. Add `CENTAUR_JWT_SIGNING_SECRET` bootstrap and chart wiring. +3. Add api-rs Protected Resource Metadata with console authorization server URL. +4. Add console OAuth authorization server metadata. +5. Add console Dynamic Client Registration. +6. Add console authorization code + PKCE flow. +7. Add console token endpoint with JWT access tokens and opaque refresh tokens. +8. Add console principal resolution for signed-in users. +9. Add api-rs JWT access token verification for MCP bearer auth. +10. Replace the unauthenticated MCP path with JWT bearer verification. +11. Dogfood with Amp using a local port-forwarded preview API. + +## Test Plan + +api-rs tests: + +- missing bearer returns `401` with `WWW-Authenticate` containing + `resource_metadata` and `scope="mcp:tools"` +- Protected Resource Metadata returns the configured resource and console + authorization server +- expired JWT is rejected +- wrong issuer is rejected +- wrong audience/resource is rejected +- missing `principal_id` is rejected +- missing `mcp:tools` scope is rejected +- valid JWT authenticates and `centaur_whoami` reports principal and `jti` + +Console tests: + +- authorization metadata includes required endpoints and supported capabilities +- DCR accepts loopback redirect URIs +- DCR rejects wildcard and non-loopback HTTP redirect URIs +- signed-out authorize request redirects to login and resumes +- pending user cannot authorize +- disabled user cannot authorize +- active user can authorize +- authorization code is one-time-use +- wrong PKCE verifier is rejected +- token endpoint returns bearer JWT with expected issuer, audience, subject, + principal, scope, and expiration +- refresh token is stored hashed and rotates on use +- disabled users cannot refresh +- raw signing secret is never rendered + +Chart/script tests: + +- `just bootstrap-secrets` creates `CENTAUR_JWT_SIGNING_SECRET` only when absent +- console deployment receives `CENTAUR_JWT_SIGNING_SECRET` +- api-rs receives the same secret +- Helm lint/template pass + +Manual dogfood: + +- port-forward preview api-rs +- configure Amp with the preview MCP URL +- verify Amp opens browser auth automatically +- sign in through console +- complete PKCE code exchange +- call `centaur_whoami` +- call a non-secret tool +- call a proxied tool granted to the console principal +- revoke a grant and verify subsequent proxied calls lose access + +## Open Questions + +- Do we need refresh tokens in v1, or will access-token-only be acceptable for + the harnesses we care about? +- Should we support OAuth Client ID Metadata Documents in v1, or is DCR enough + for Amp/Codex/VS Code? +- Default access token TTL: 1 hour, 8 hours, or environment-specific? +- Default refresh token TTL: 7 days, 30 days, or environment-specific? +- Should the console principal namespace default to `default` or a dedicated + namespace like `mcp`? +- Do we want a first-version access-token `jti` denylist, or is short TTL + enough? diff --git a/services/console/AGENTS.md b/services/console/AGENTS.md new file mode 100644 index 000000000..c77f672c0 --- /dev/null +++ b/services/console/AGENTS.md @@ -0,0 +1,50 @@ +# Console Guide + +## Role + +The console is a Rails application that provides the operator UI and the +credential-control JSON API. It manages principals, roles, grants, encrypted +secret records, proxy synchronization, broker credentials, console login, and +MCP OAuth flows. Its Threads surface reads Centaur session data and is not a +second session control plane. + +Use `README.md` and `docs/API.md` for the supported behavior and API shapes. + +## Invariants + +- Keep cookie-backed console sessions, bearer-authenticated operator APIs, + proxy sync authentication, and OAuth/MCP tokens as separate trust boundaries. +- Secret plaintext may be accepted for creation or rotation but must never be + returned, logged, rendered, inspected in tests, or stored outside encrypted + model attributes and configured providers. +- Apply authorization in controllers and service objects before loading or + mutating scoped resources. Test disabled users, non-admin users, namespace + isolation, token replay, and ownership checks where relevant. +- OAuth/MCP changes must cover redirect validation, consent, PKCE, refresh-token + family rotation and replay, revocation, account disablement, and identity + reconciliation. A connected UI state alone is not proof of usable access. +- The Threads UI is an observer of durable session data. Do not make it write + chat messages or bypass the session API. +- Put business logic in models or `app/services`, keep controllers thin, and + preserve JSON error and pagination contracts. +- Generate migrations with Rails and commit the resulting `db/schema.rb` change. + Do not hand-edit the schema dump. +- Use local fixtures or synthetic snapshots for UI work; do not make tests + depend on a remote database. + +## Validation + +From `services/console`: + +```bash +bundle install +bin/rails db:prepare +bin/rails test +bin/rubocop +bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error +``` + +`bin/ci` runs the full local CI sequence, including dependency audits, tests, +and seed validation. Add focused controller/model/service tests with each +behavior change. For visible UI changes, run `just dev`, exercise the affected +flow in a browser, and check narrow and wide layouts plus keyboard/focus states. diff --git a/services/console/Gemfile.lock b/services/console/Gemfile.lock index d18751d60..15fdfa8b6 100644 --- a/services/console/Gemfile.lock +++ b/services/console/Gemfile.lock @@ -352,7 +352,7 @@ GEM bindex (>= 0.4.0) railties (>= 8.0.0) websocket (1.2.11) - websocket-driver (0.8.1) + websocket-driver (0.8.2) base64 websocket-extensions (>= 0.1.0) websocket-extensions (0.1.5) @@ -369,6 +369,7 @@ PLATFORMS arm-linux-musl arm64-darwin-23 arm64-darwin-24 + arm64-darwin-25 x86_64-linux x86_64-linux-gnu x86_64-linux-musl diff --git a/services/console/README.md b/services/console/README.md index fabdc0dc3..82aba7f82 100644 --- a/services/console/README.md +++ b/services/console/README.md @@ -30,6 +30,43 @@ Operators manage credentials, principals, roles, and grants through the API or t All of the console's environment variables use the `CENTAUR_CONSOLE_` prefix. For backwards compatibility, every variable also resolves from the legacy `IRON_CONTROL_` name when the `CENTAUR_CONSOLE_` one is unset, so existing deployments keep working until they migrate. The `CENTAUR_CONSOLE_` name wins when both are set. +The Threads tab reads api-rs session rows from the Centaur API database. Set +`CENTAUR_CONSOLE_CENTAUR_DATABASE_URL` to that database URL when it differs from +the console's primary database. In the Helm chart this is sourced from the +shared `DATABASE_URL` secret key. + +For local development, sign in through the normal login form at +`http://localhost:3000/login` with the seeded initial user's +`CENTAUR_CONSOLE_INITIAL_USER_EMAIL` / `CENTAUR_CONSOLE_INITIAL_USER_PASSWORD` +credentials, the same as every other environment. + +To build the Threads UX against production-shaped data without connecting the +Console to production, create a bounded local snapshot: + +```bash +export CENTAUR_PROD_DATABASE_URL=postgresql://readonly:...@.../ai_v2 +bash scripts/mirror-prod-threads-snapshot.sh all +``` + +The script exports recent `sessions`, `session_messages`, +`session_executions`, terminal `session_events` plus reasoning +`session.output.line` events (capped by `THINKING_EVENT_LIMIT_PER_THREAD`, +default 200 per thread), and referenced `slack_sync_users` rows with the source +connection forced read-only, then imports them into the local `ai_v2` database +used by the Console dev container. The Threads surface is read-only: it does +not render a composer and rejects POSTs server-side. + +Threads extras beyond the Slack surface: + +- Thinking traces: reasoning items the harness streamed over stdout are + persisted by api-rs as `session.output.line` events; the transcript renders + each completed reasoning block as a collapsed "Thinking" disclosure. +- Split view: Cmd/Ctrl-click a sidebar thread to open it alongside the current + one, up to four threads in a grid. The `thread` param carries the open keys + comma-separated (`?thread=,,,`), primary first. + All keys resolve through the same owner scope as a single thread, and each + panel has a close control. + ## First Boot The console requires an authenticated user and API key before any API endpoint will respond. To bootstrap a fresh deployment without a console, set the following environment variables on startup: @@ -59,7 +96,7 @@ The operator console always supports email and password sign-in. To add Google o | `CENTAUR_CONSOLE_GOOGLE_CLIENT_SECRET` | for Google | Google OAuth client secret for console login. | | `CENTAUR_CONSOLE_SLACK_CLIENT_ID` | for Slack | Slack OpenID Connect client ID for console login. | | `CENTAUR_CONSOLE_SLACK_CLIENT_SECRET` | for Slack | Slack OpenID Connect client secret for console login. | -| `CENTAUR_CONSOLE_BOOTSTRAP_ADMINS` | no | Comma- or whitespace-separated email allowlist. Matching users become active admins on first SSO login. Other new SSO users are created as pending users. | +| `CENTAUR_CONSOLE_BOOTSTRAP_ADMINS` | no | Comma- or whitespace-separated email allowlist. Matching users become active admins on first SSO login. Other SSO users become active non-admin operators and land on the console directly -- the deployment's network boundary is the access control, there is no approval queue. | Register these callback URLs with the provider: diff --git a/services/console/app/assets/tailwind/application.css b/services/console/app/assets/tailwind/application.css index 9f2af7945..ce1059d30 100644 --- a/services/console/app/assets/tailwind/application.css +++ b/services/console/app/assets/tailwind/application.css @@ -44,6 +44,30 @@ @apply mt-1 text-sm text-zinc-500; } + .console-page-header { + @apply mb-6 flex min-h-11 items-start justify-between gap-4; + } + + .console-page-heading { + @apply flex min-w-0 items-start gap-3; + } + + .console-page-icon { + @apply mt-0.5 grid size-7 shrink-0 place-items-center rounded-lg bg-ink-850/75 text-zinc-400; + } + + .console-page-title-copy { + @apply min-w-0; + } + + .console-page-actions { + @apply flex shrink-0 items-center gap-2; + } + + .console-page-meta { + @apply pt-1 text-xs text-zinc-500; + } + .back-link { @apply text-xs text-zinc-500 hover:text-centaur-400; } @@ -57,7 +81,11 @@ } .console-table { - @apply min-w-full divide-y divide-ink-600 text-sm; + @apply min-w-full text-sm; + } + + .console-table > thead { + @apply border-b border-ink-600; } .console-table-head { diff --git a/services/console/app/controllers/api/v1/principals_controller.rb b/services/console/app/controllers/api/v1/principals_controller.rb index 13b1e31e3..4adfedee6 100644 --- a/services/console/app/controllers/api/v1/principals_controller.rb +++ b/services/console/app/controllers/api/v1/principals_controller.rb @@ -1,8 +1,12 @@ module Api module V1 class PrincipalsController < Api::BaseController + InvalidSlackChannelPermissions = Class.new(StandardError) + + rescue_from InvalidSlackChannelPermissions, with: :render_slack_channel_permissions_error + def index - records, meta = paginated_label_search(Principal.all) + records, meta = paginated_label_search(Principal.includes(:slack_channel_permissions)) render json: { data: records.map { |p| record_payload(p) }, meta: meta } end @@ -24,8 +28,12 @@ def lookup def create principal = Principal.new(namespace: upsert_namespace, foreign_id: data_params[:foreign_id], created_by: current_user) - principal.assign_attributes(principal_params) - principal.save! + ActiveRecord::Base.transaction do + principal.assign_attributes(principal_params) + principal.apply_default_sandbox_capabilities!(principal_params) + principal.save! + replace_slack_channel_permissions!(principal) if data_params.key?(:slack_channel_permissions) + end render status: :created, json: { data: record_payload(principal) } rescue ActiveRecord::RecordInvalid => e render_validation_error(e.record) @@ -37,8 +45,12 @@ def create def update principal = resolve_for_upsert(Principal) was_new = principal.new_record? - principal.assign_attributes(principal_params) - principal.save! + ActiveRecord::Base.transaction do + principal.assign_attributes(principal_params) + principal.apply_default_sandbox_capabilities!(principal_params) if was_new + principal.save! + replace_slack_channel_permissions!(principal) if data_params.key?(:slack_channel_permissions) + end render status: (was_new ? :created : :ok), json: { data: record_payload(principal) } rescue ActiveRecord::RecordInvalid => e render_validation_error(e.record) @@ -65,6 +77,26 @@ def effective_config render json: body end + # POST /api/v1/principals/:id/slack_channel_permissions + # + # Upserts one Slack channel permission row without replacing the rest of + # the principal's operator-managed Slack permissions. + def upsert_slack_channel_permission + principal = Principal.find_by_oid!(params[:id]) + attrs = upsert_slack_channel_permission_params + attrs[:channel_id] = attrs[:channel_id].to_s.strip.upcase + permission, was_new = save_slack_channel_permission!(principal, attrs) + + render status: (was_new ? :created : :ok), json: { data: permission.as_permission_json } + rescue ActiveRecord::RecordNotUnique + permission = principal.slack_channel_permissions.find_by!(channel_id: attrs[:channel_id]) + permission.assign_attributes(attrs) + permission.save! + render status: :ok, json: { data: permission.as_permission_json } + rescue ActiveRecord::RecordInvalid => e + render_validation_error(e.record) + end + private def record_payload(principal) @@ -73,9 +105,15 @@ def record_payload(principal) namespace: principal.namespace, foreign_id: principal.foreign_id, name: principal.name, - labels: principal.labels, - sandbox_repo_cache_enabled: principal.sandbox_repo_cache_enabled, + labels: principal.labels_with_sandbox_capabilities, + slack_channel_permissions: principal.slack_channel_permissions_payload, + sandbox_repo_cache: principal.sandbox_repo_cache, + # Transitional expand/contract field for the pre-enum api-rs kept + # live during the Console-first rollout. Remove only after every old + # runtime generation is retired. + sandbox_repo_cache_enabled: principal.sandbox_repo_cache == "all", sandbox_observability_enabled: principal.sandbox_observability_enabled, + sandbox_api_server_enabled: principal.sandbox_api_server_enabled, created_at: principal.created_at, updated_at: principal.updated_at } @@ -84,11 +122,70 @@ def record_payload(principal) def principal_params data_params.permit( :name, - :sandbox_repo_cache_enabled, + :sandbox_repo_cache, :sandbox_observability_enabled, + :sandbox_api_server_enabled, labels: {} ) end + + def replace_slack_channel_permissions!(principal) + SlackChannelPermission.replace_for_principal!( + principal, + slack_channel_permission_params + ) + end + + def save_slack_channel_permission!(principal, attrs) + permission = principal.slack_channel_permissions.find_or_initialize_by( + channel_id: attrs[:channel_id] + ) + was_new = permission.new_record? + permission.assign_attributes(attrs) + permission.save! + [ permission, was_new ] + end + + def slack_channel_permission_params + raw = data_params[:slack_channel_permissions] + unless raw.nil? || raw.is_a?(Array) + raise InvalidSlackChannelPermissions, "slack_channel_permissions must be an array" + end + + rows = data_params.permit( + slack_channel_permissions: %i[ + channel_id + channel_name + upload_enabled + download_enabled + history_enabled + ] + ).fetch(:slack_channel_permissions, []) + + if raw.present? && rows.length != raw.length + raise InvalidSlackChannelPermissions, "slack_channel_permissions rows must be objects" + end + + rows + end + + def upsert_slack_channel_permission_params + @upsert_slack_channel_permission_params ||= data_params.permit( + :channel_id, + :channel_name, + :upload_enabled, + :download_enabled, + :history_enabled + ).tap do |attrs| + attrs[:upload_enabled] = true unless attrs.key?(:upload_enabled) + attrs[:download_enabled] = true unless attrs.key?(:download_enabled) + attrs[:history_enabled] = true unless attrs.key?(:history_enabled) + end + end + + def render_slack_channel_permissions_error(error) + render_error(status: :unprocessable_entity, message: error.message) + end end end end diff --git a/services/console/app/controllers/api/v1/proxy_sync_controller.rb b/services/console/app/controllers/api/v1/proxy_sync_controller.rb index b70ffd484..286fb8de1 100644 --- a/services/console/app/controllers/api/v1/proxy_sync_controller.rb +++ b/services/console/app/controllers/api/v1/proxy_sync_controller.rb @@ -7,13 +7,12 @@ module V1 # hash (no payload), so the proxy skips re-applying. Otherwise we return the # full `secrets` and `transforms` payload. # - # `proxy` carries managed proxy runtime settings. `secrets` populates the - # proxy's `secrets` transform. `transforms` carries whole transforms the proxy - # splices into its pipeline: one gcp_auth, gcp_id_token, hmac_sign, or - # aws_auth transform per granted secret, and one bundled oauth_token - # transform. `postgres` carries one upstream-DSN entry per granted - # PgDsnSecret, keyed by foreign_id; the proxy's locally-defined listeners - # bind to these by foreign_id. + # `secrets` populates the proxy's `secrets` transform. `transforms` carries + # whole transforms the proxy splices into its pipeline: one gcp_auth, + # gcp_id_token, hmac_sign, or aws_auth transform per granted secret, and one + # bundled oauth_token transform. `postgres` carries one upstream-DSN + # entry per granted PgDsnSecret, keyed by foreign_id; the proxy's + # locally-defined listeners bind to these by foreign_id. # # The top-level `rules`, `mcp`, and `ingest_token` fields the proxy also # understands are intentionally omitted: centaur-console has no models for them @@ -34,7 +33,6 @@ def create config_hash: current_hash, status: current_proxy.status, principal_id: current_proxy.principal&.oid, - proxy: config["proxy"], secrets: config["secrets"], transforms: config["transforms"], postgres: config["postgres"] diff --git a/services/console/app/controllers/application_controller.rb b/services/console/app/controllers/application_controller.rb index 2093da1eb..71581ff7c 100644 --- a/services/console/app/controllers/application_controller.rb +++ b/services/console/app/controllers/application_controller.rb @@ -9,7 +9,7 @@ class ApplicationController < ActionController::Base # controllers don't each hand-roll a rescue. Mirrors Api::BaseController. rescue_from ActiveRecord::RecordNotFound, with: :render_not_found - helper_method :current_user + helper_method :current_user, :acting_admin?, :descoped? helper_method :public_base_url, :oauth_callback_redirect_uri # The public origin the console is reached at. Derived from the request by @@ -37,6 +37,25 @@ def oauth_callback_redirect_uri(slug) # and pending controllers skip this so pending users can reach the holding page # and sign out. before_action :require_active_account + # The sidebar thread list is global chrome (rendered by layouts/console.html.erb + # on every page), but populating it issues several queries against the api-rs + # ai_v2 sessions DB, including an unindexed sequential scan + sort of the + # sessions table. Running that in every console request blocked pages that only + # render the empty-state list (principals, roles, secrets, ...). Instead we + # initialize the ivars empty here and load the real list lazily via a Turbo + # Frame (Console::ThreadsController#sidebar), so the cross-database work happens + # once, out of band, and never blocks the primary page render. + before_action :init_console_sidebar_threads + + CONSOLE_SIDEBAR_THREAD_LIMIT = 30 + CONSOLE_SIDEBAR_SLACK_PROVIDER = Oauth::Providers::Slack::KEY + CONSOLE_SIDEBAR_SLACK_THREAD_OWNER_METADATA_KEYS = %w[slack_user_id actor_user_id user_id].freeze + CONSOLE_SIDEBAR_SLACK_THREAD_TEAM_METADATA_KEYS = %w[slack_team_id team_id home_team_id].freeze + CONSOLE_SIDEBAR_SLACK_CREDENTIAL_USER_LABEL_KEYS = %w[slack_user_id].freeze + CONSOLE_SIDEBAR_SLACK_CREDENTIAL_EMAIL_LABEL_KEYS = %w[email slack_email].freeze + CONSOLE_SIDEBAR_SLACK_TEAM_LABEL = "slack_team_id".freeze + CONSOLE_SIDEBAR_THREAD_OWNER_METADATA_KEYS = %w[actor_email user_email].freeze + ConsoleSidebarSlackThreadOwner = Struct.new(:user_id, :team_id, keyword_init: true) private @@ -46,6 +65,24 @@ def current_user @current_user ||= User.find_by(id: session[:user_id]) if session[:user_id] end + # Whether this admin has temporarily descoped themselves to operator + # permissions ("view as operator"). Self-healing: the flag is dropped if the + # user is no longer an admin, so it can never outlive the privileges it pauses. + def descoped? + return false unless session[:descoped] + return true if current_user&.admin? + + session.delete(:descoped) + false + end + + # The permission check console gates use instead of current_user.admin?: a + # real admin who is not currently descoped. Keeping current_user untouched + # means audit trails and data displays still see the true account. + def acting_admin? + current_user&.admin? && !descoped? + end + # before_action gate for console pages: bounce anonymous requests to the login # form rather than rendering the page. def require_login @@ -64,15 +101,48 @@ def require_active_account end end - # Guard for admin-only controllers (e.g. user management). Not a global gate. + # Guard for admin-only controllers (the Control and Data Sync sections, user + # management). Not a global gate. Bounces non-admins to their only available + # section. Keep this redirect silent: direct/admin-default URLs are not + # actionable errors for non-admin operators, especially on a fresh visit. def require_admin - redirect_to root_path, alert: "That page is restricted to admins." unless current_user&.admin? + redirect_to console_threads_path unless acting_admin? + end + + # Where a signed-in user lands when no explicit destination applies: admins get + # the Control section, everyone else the threads view (their only section). + def default_console_landing_path + acting_admin? ? console_principals_path : console_threads_path + end + + # Cheap default so every page renders the empty sidebar list without touching + # the sessions DB. The real list is filled in by #load_console_sidebar_threads, + # invoked only from the lazy sidebar Turbo Frame. + def init_console_sidebar_threads + @console_sidebar_threads = [] + @console_sidebar_latest_messages = {} + end + + def load_console_sidebar_threads + @console_sidebar_threads = [] + @console_sidebar_latest_messages = {} + return unless current_user&.active? + + threads = console_sidebar_visible_thread_scope + .recent_first + .limit(CONSOLE_SIDEBAR_THREAD_LIMIT) + .to_a + threads = console_sidebar_threads_with_direct_selection(threads) + @console_sidebar_threads = threads + @console_sidebar_latest_messages = console_sidebar_latest_messages_for(threads.map(&:thread_key)) + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.debug("console_sidebar_threads_unavailable error=#{e.class}: #{e.message}") end # Establishes the console cookie session and sends the user to the right # post-login page. Password login re-renders for disabled accounts; SSO login # redirects because it is returning from an external provider. - def sign_in_console_user(user, disabled: :redirect) + def sign_in_console_user(user, disabled: :redirect, destination: nil) if user.disabled? if disabled == :render flash.now[:alert] = "Your account is disabled." @@ -82,16 +152,231 @@ def sign_in_console_user(user, disabled: :redirect) return redirect_to login_path, alert: "Your account is disabled." end + return_to = session[:return_to] reset_session session[:user_id] = user.id + session[:return_to] = return_to if return_to.present? if user.active? - redirect_to console_principals_path, notice: "Signed in as #{user.email}." + redirect_to(destination.presence || post_login_redirect_path, notice: "Signed in as #{user.email}.") else redirect_to pending_path, notice: "Your account is awaiting approval." end end + def post_login_redirect_path + path = session.delete(:return_to).to_s + return default_console_landing_path unless path.start_with?("/") && !path.start_with?("//") + path + end + + def safe_console_return_path(default: default_console_landing_path) + raw = params[:return_to].presence || params[:next].presence + return default if raw.blank? + + uri = URI.parse(raw.to_s) + return default if uri.scheme.present? || uri.host.present? + + path = uri.path.presence + return default unless path == "/" || path&.start_with?("/console") + + uri.to_s + rescue URI::InvalidURIError + default + end + def render_not_found(e) render plain: e.message, status: :not_found end + + def console_sidebar_visible_thread_scope + slack_owners = console_sidebar_slack_thread_owners_for_current_user + conditions = [ + console_sidebar_console_thread_owner_sql, + (console_sidebar_slack_thread_owner_sql(slack_owners) if slack_owners.any?) + ].compact + + return CentaurSession.where("1=0") if conditions.empty? + + CentaurSession.where(conditions.map { |condition| "(#{condition})" }.join(" OR ")) + end + + def console_sidebar_threads_with_direct_selection(threads) + selected = console_sidebar_direct_selected_threads(threads) + selected.any? ? [ *selected, *threads ] : threads + end + + def console_sidebar_direct_selected_threads(threads) + thread_keys = console_sidebar_selected_thread_keys - threads.map(&:thread_key) + return [] if thread_keys.empty? + + # Resolve through the owner scope, not a raw find_by, so a directly linked + # thread only surfaces in the sidebar when the current user started it. This + # mirrors Console::ThreadsController#selected_session. + console_sidebar_visible_thread_scope.where(thread_key: thread_keys).to_a + end + + # The thread param carries up to PANEL_LIMIT comma-separated keys when the + # split view is open; every open thread should surface and highlight. + def console_sidebar_selected_thread_keys + return [] unless params[:controller] == "console/threads" + + params[:thread].to_s.split(",").map(&:strip).reject(&:blank?).uniq + .first(Console::ThreadsController::PANEL_LIMIT) + end + + def console_sidebar_console_thread_owner_sql + email = console_sidebar_normalize_email(current_user&.email) + return if email.blank? + + console_source = [ + "thread_key LIKE 'console:%'", + "metadata ->> 'platform' = 'console'", + "metadata ->> 'source' = 'console'" + ].join(" OR ") + owner_clauses = CONSOLE_SIDEBAR_THREAD_OWNER_METADATA_KEYS.map do |key| + "lower(metadata ->> #{console_sidebar_sql_quote(key)}) = #{console_sidebar_sql_quote(email)}" + end + + "(#{console_source}) AND (#{owner_clauses.join(" OR ")})" + end + + def console_sidebar_slack_thread_owners_for_current_user + @console_sidebar_slack_thread_owners_for_current_user ||= begin + subjects = console_sidebar_slack_identity_subjects_for_current_user + emails = console_sidebar_slack_identity_emails_for_current_user + + if subjects.empty? && emails.empty? + [] + else + credentials = BrokerCredential + .joins(:oauth_app) + .includes(:oauth_app) + .where(oauth_apps: { provider: CONSOLE_SIDEBAR_SLACK_PROVIDER }) + .where(console_sidebar_slack_oauth_credential_owner_sql(subjects: subjects, emails: emails)) + + credential_owners = credentials.filter_map do |credential| + user_id = console_sidebar_first_present( + credential.provider_subject, + *CONSOLE_SIDEBAR_SLACK_CREDENTIAL_USER_LABEL_KEYS.map { |key| credential.labels&.[](key) } + ) + next if user_id.blank? + + ConsoleSidebarSlackThreadOwner.new( + user_id: user_id, + team_id: console_sidebar_first_present( + credential.labels&.[](CONSOLE_SIDEBAR_SLACK_TEAM_LABEL), + credential.oauth_app&.labels&.[](CONSOLE_SIDEBAR_SLACK_TEAM_LABEL) + ) + ) + end + + # A Slack OIDC sign-in stores the workspace user id (U…) as the + # identity subject — the same id slackbotv2 writes into session + # metadata — so SSO alone owns those threads even when the user has + # not minted a broker credential through the connect flow. + identity_owners = subjects.map do |subject| + ConsoleSidebarSlackThreadOwner.new(user_id: subject, team_id: nil) + end + + (credential_owners + identity_owners) + .uniq { |owner| [ console_sidebar_normalize_key(owner.user_id), console_sidebar_normalize_key(owner.team_id) ] } + end + end + end + + def console_sidebar_slack_identity_subjects_for_current_user + current_user.user_identities + .where(provider: CONSOLE_SIDEBAR_SLACK_PROVIDER) + .pluck(:subject) + .filter_map { |value| console_sidebar_normalize_key(value) } + .uniq + end + + def console_sidebar_slack_identity_emails_for_current_user + ([ current_user.email ] + current_user.user_identities.where(provider: CONSOLE_SIDEBAR_SLACK_PROVIDER).pluck(:email)) + .filter_map { |value| console_sidebar_normalize_email(value) } + .uniq + end + + def console_sidebar_slack_oauth_credential_owner_sql(subjects:, emails:) + clauses = [] + if subjects.any? + subject_values = console_sidebar_sql_list(subjects) + clauses << "lower(broker_credentials.provider_subject) IN (#{subject_values})" + CONSOLE_SIDEBAR_SLACK_CREDENTIAL_USER_LABEL_KEYS.each do |key| + clauses << "lower(broker_credentials.labels ->> #{console_sidebar_sql_quote(key)}) IN (#{subject_values})" + end + end + + if emails.any? + email_values = console_sidebar_sql_list(emails) + clauses << "lower(broker_credentials.provider_email) IN (#{email_values})" + CONSOLE_SIDEBAR_SLACK_CREDENTIAL_EMAIL_LABEL_KEYS.each do |key| + clauses << "lower(broker_credentials.labels ->> #{console_sidebar_sql_quote(key)}) IN (#{email_values})" + end + end + + clauses.join(" OR ") + end + + def console_sidebar_slack_thread_owner_sql(owners) + slack_source = [ + "thread_key LIKE 'slack:%'", + "metadata ->> 'platform' = 'slack'", + "metadata ->> 'source' = 'slackbotv2'" + ].join(" OR ") + + owner_clauses = owners.map do |owner| + user_id = console_sidebar_normalize_key(owner.user_id) + user_clauses = CONSOLE_SIDEBAR_SLACK_THREAD_OWNER_METADATA_KEYS.map do |key| + "lower(metadata ->> #{console_sidebar_sql_quote(key)}) = #{console_sidebar_sql_quote(user_id)}" + end + owner_clause = "(#{user_clauses.join(" OR ")})" + + # Team scoping narrows the match only when the credential exposes a team; + # see Console::ThreadsController#slack_thread_owner_sql. + if owner.team_id.present? + team_id = console_sidebar_normalize_key(owner.team_id) + team_clauses = CONSOLE_SIDEBAR_SLACK_THREAD_TEAM_METADATA_KEYS.map do |key| + "lower(metadata ->> #{console_sidebar_sql_quote(key)}) = #{console_sidebar_sql_quote(team_id)}" + end + team_clauses << "lower(split_part(thread_key, ':', 2)) = #{console_sidebar_sql_quote(team_id)}" + owner_clause = "(#{owner_clause} AND (#{team_clauses.join(" OR ")}))" + end + + owner_clause + end + + "(#{slack_source}) AND (#{owner_clauses.join(" OR ")})" + end + + def console_sidebar_latest_messages_for(keys) + return {} if keys.empty? + + CentaurSessionMessage + .where(thread_key: keys) + .select("distinct on (thread_key) session_messages.*") + .order(Arel.sql("thread_key, created_at desc, message_id desc")) + .index_by(&:thread_key) + end + + def console_sidebar_first_present(*values) + values.find(&:present?) + end + + def console_sidebar_normalize_key(value) + value.to_s.strip.downcase.presence + end + + def console_sidebar_normalize_email(value) + value.to_s.strip.downcase.presence + end + + def console_sidebar_sql_list(values) + values.map { |value| console_sidebar_sql_quote(value) }.join(", ") + end + + def console_sidebar_sql_quote(value) + ActiveRecord::Base.connection.quote(value.to_s) + end end diff --git a/services/console/app/controllers/console/base_secrets_controller.rb b/services/console/app/controllers/console/base_secrets_controller.rb index e9c31a7b0..21e9f25d4 100644 --- a/services/console/app/controllers/console/base_secrets_controller.rb +++ b/services/console/app/controllers/console/base_secrets_controller.rb @@ -10,6 +10,7 @@ class BaseSecretsController < ApplicationController layout "console" + before_action :require_admin before_action :assign_kind before_action :set_secret, only: %i[edit update destroy] diff --git a/services/console/app/controllers/console/broker_credentials_controller.rb b/services/console/app/controllers/console/broker_credentials_controller.rb index 5051f4b1b..08c9c4be1 100644 --- a/services/console/app/controllers/console/broker_credentials_controller.rb +++ b/services/console/app/controllers/console/broker_credentials_controller.rb @@ -10,6 +10,7 @@ class BrokerCredentialsController < ApplicationController layout "console" + before_action :require_admin before_action :set_credential, only: %i[edit update destroy] def new @@ -69,6 +70,7 @@ def assign_form(credential) secret = credential_params[:client_secret] if secret.present? + # Active Record encryption protects this write-only attribute at rest. credential.client_secret = secret reset_refresh_state(credential) if credential.grant == BrokerCredential::GITHUB_APP_INSTALLATION end diff --git a/services/console/app/controllers/console/descopes_controller.rb b/services/console/app/controllers/console/descopes_controller.rb new file mode 100644 index 000000000..e9e136acd --- /dev/null +++ b/services/console/app/controllers/console/descopes_controller.rb @@ -0,0 +1,28 @@ +module Console + # Admin "view as operator" support: an admin can temporarily pause their own + # admin permissions to see the console as a regular operator would. The flag + # lives in the cookie session; acting_admin? (the check every admin gate uses) + # is false while it's set, and ApplicationController drops it automatically if + # the user is no longer an admin. + # + # create is admin-gated. destroy is deliberately not: while descoped, the user + # fails require_admin, but they must always be able to restore themselves. + class DescopesController < ApplicationController + before_action :require_admin, only: :create + + def create + session[:descoped] = true + Rails.logger.info("console_descope_started admin=#{current_user.email}") + # No flash: the persistent descope banner already announces the state. + redirect_to console_threads_path + end + + def destroy + return redirect_to default_console_landing_path unless descoped? + + session.delete(:descoped) + Rails.logger.info("console_descope_stopped admin=#{current_user.email}") + redirect_to console_principals_path, notice: "Admin permissions restored." + end + end +end diff --git a/services/console/app/controllers/console/etls_controller.rb b/services/console/app/controllers/console/etls_controller.rb index bdc9296d4..98525845f 100644 --- a/services/console/app/controllers/console/etls_controller.rb +++ b/services/console/app/controllers/console/etls_controller.rb @@ -1,6 +1,8 @@ class Console::EtlsController < ApplicationController layout "console" + before_action :require_admin + class_attribute :client_factory, default: -> { CentaurApiClient.new } def index diff --git a/services/console/app/controllers/console/integrations_controller.rb b/services/console/app/controllers/console/integrations_controller.rb new file mode 100644 index 000000000..cbc098257 --- /dev/null +++ b/services/console/app/controllers/console/integrations_controller.rb @@ -0,0 +1,25 @@ +# The user-facing Integrations page: every enabled OauthApp with its public +# consent start link (/oauth//start), so any signed-in team member can +# connect an integration without an operator sharing the link by hand. +# +# Deliberately not admin-gated (unlike ConsoleController): the whole point of +# the well-known consent links is that regular team members click them. Only +# non-sensitive fields are shown -- slug, provider, description -- never the +# client id/secret or minted credentials. +class Console::IntegrationsController < ApplicationController + layout "console" + + def index + @oauth_apps = OauthApp.where(enabled: true).order(:slug) + # The user's existing connections: credentials they minted while signed in + # (created_by, recorded by the consent callback) plus any whose IdP-reported + # email matches their console login -- the fallback for consents made + # without a console session. Newest wins if several match one app. + mine = BrokerCredential.where(created_by: current_user) + .or(BrokerCredential.where(provider_email: current_user.email)) + @credentials_by_app_id = mine + .where(oauth_app_id: @oauth_apps.select(:id)) + .order(:updated_at) + .index_by(&:oauth_app_id) + end +end diff --git a/services/console/app/controllers/console/oauth_apps_controller.rb b/services/console/app/controllers/console/oauth_apps_controller.rb index 7294bb3d9..33e73432a 100644 --- a/services/console/app/controllers/console/oauth_apps_controller.rb +++ b/services/console/app/controllers/console/oauth_apps_controller.rb @@ -9,6 +9,7 @@ class OauthAppsController < ApplicationController layout "console" + before_action :require_admin before_action :set_app, only: %i[edit update] def new @@ -49,6 +50,7 @@ def assign_form(app) app.labels = label_params secret = app_params[:client_secret] + # Active Record encryption protects this write-only attribute at rest. app.client_secret = secret if secret.present? end diff --git a/services/console/app/controllers/console/principals_controller.rb b/services/console/app/controllers/console/principals_controller.rb index fe8e25c58..c14c02008 100644 --- a/services/console/app/controllers/console/principals_controller.rb +++ b/services/console/app/controllers/console/principals_controller.rb @@ -4,22 +4,53 @@ module Console # controller only handles the POST/DELETE actions wired from that page. Gated by # the app-wide require_login (not admin -- mirrors the secret/credential forms). class PrincipalsController < ApplicationController + include KvRowParams include SecretKinds layout "console" - before_action :set_principal + before_action :require_admin + before_action :set_principal, except: %i[new create] + + def new + @principal = Principal.new(namespace: "default") + end + + def create + @principal = Principal.new(created_by: current_user) + assign_form(@principal) + @principal.apply_default_sandbox_capabilities! + if @principal.save + redirect_to console_principal_path(@principal.oid), notice: "Principal created." + else + render :new, status: :unprocessable_entity + end + end + + def destroy + label = principal_label(@principal) + @principal.destroy! + redirect_to console_principals_path, notice: "Deleted principal #{label}." + end def update_sandbox_access @principal.update!( - sandbox_repo_cache_enabled: ActiveModel::Type::Boolean.new.cast(params[:sandbox_repo_cache_enabled]), - sandbox_observability_enabled: ActiveModel::Type::Boolean.new.cast(params[:sandbox_observability_enabled]) + sandbox_repo_cache: params[:sandbox_repo_cache], + sandbox_observability_enabled: ActiveModel::Type::Boolean.new.cast(params[:sandbox_observability_enabled]), + sandbox_api_server_enabled: ActiveModel::Type::Boolean.new.cast(params[:sandbox_api_server_enabled]) ) redirect_to console_principal_path(@principal.oid), notice: "Updated sandbox access." rescue ActiveRecord::RecordInvalid => e redirect_to console_principal_path(@principal.oid), alert: e.record.errors.full_messages.to_sentence end + def update_slack_channel_permissions + @principal.update!(slack_channel_permission_params) + redirect_to console_principal_path(@principal.oid), notice: "Updated Slack channel permissions." + rescue ActiveRecord::RecordInvalid => e + redirect_to console_principal_path(@principal.oid), alert: e.record.errors.full_messages.to_sentence + end + def assign_role role = Role.find_by_oid!(params[:role_id]) @principal.principal_roles.find_or_create_by!(role: role) @@ -63,6 +94,32 @@ def revoke_grant private + def assign_form(principal) + fields = principal_params.permit(:namespace, :foreign_id, :name) + fields[:namespace] = fields[:namespace].presence || "default" + fields[:foreign_id] = fields[:foreign_id].presence + principal.assign_attributes(fields) + principal.labels = label_params + end + + def principal_params + params.fetch(:principal, ActionController::Parameters.new) + end + + def slack_channel_permission_params + params.require(:principal).permit( + slack_channel_permissions_attributes: %i[ + id + channel_id + channel_name + upload_enabled + download_enabled + history_enabled + _destroy + ] + ) + end + # Parse the ":" value from the grant dropdown into a secret record. # Returns nil for a blank/unknown selection so the action can flash and bail. def resolve_grantable(value) @@ -86,6 +143,10 @@ def secret_label(secret) secret.try(:name).presence || secret.foreign_id.presence || secret.oid end + def principal_label(principal) + principal.name.presence || principal.foreign_id.presence || principal.oid + end + def set_principal @principal = Principal.find_by_oid!(params[:id]) end diff --git a/services/console/app/controllers/console/roles_controller.rb b/services/console/app/controllers/console/roles_controller.rb index 6ab6635ef..f9dc7bc03 100644 --- a/services/console/app/controllers/console/roles_controller.rb +++ b/services/console/app/controllers/console/roles_controller.rb @@ -7,6 +7,7 @@ class RolesController < ApplicationController layout "console" + before_action :require_admin before_action :set_role, only: %i[show edit update grant_secret revoke_grant] def index diff --git a/services/console/app/controllers/console/secrets_controller.rb b/services/console/app/controllers/console/secrets_controller.rb index 9b1ed0649..6f9009cd5 100644 --- a/services/console/app/controllers/console/secrets_controller.rb +++ b/services/console/app/controllers/console/secrets_controller.rb @@ -7,6 +7,7 @@ class SecretsController < ApplicationController layout "console" + before_action :require_admin before_action :set_secret def grant_role diff --git a/services/console/app/controllers/console/system_settings_controller.rb b/services/console/app/controllers/console/system_settings_controller.rb new file mode 100644 index 000000000..1a069f0f9 --- /dev/null +++ b/services/console/app/controllers/console/system_settings_controller.rb @@ -0,0 +1,33 @@ +module Console + class SystemSettingsController < ApplicationController + layout "console" + + before_action :require_admin + before_action :set_system_setting + + def edit + end + + def update + if @system_setting.update(system_setting_params) + redirect_to edit_console_system_settings_path, notice: "System settings updated." + else + render :edit, status: :unprocessable_entity + end + end + + private + + def set_system_setting + @system_setting = SystemSetting.current + end + + def system_setting_params + params.require(:system_setting).permit( + :default_sandbox_repo_cache, + :default_sandbox_observability_enabled, + :default_sandbox_api_server_enabled + ) + end + end +end diff --git a/services/console/app/controllers/console/threads_controller.rb b/services/console/app/controllers/console/threads_controller.rb new file mode 100644 index 000000000..a7538c192 --- /dev/null +++ b/services/console/app/controllers/console/threads_controller.rb @@ -0,0 +1,1705 @@ +class Console::ThreadsController < ApplicationController + layout "console" + + # Injectable for tests, mirroring Console::WorkflowsController. + class_attribute :client_factory, default: -> { CentaurApiClient.new } + + THREAD_LIMIT = 250 + MESSAGE_LIMIT = 80 + EXECUTION_LIMIT = 8 + TRANSCRIPT_EVENT_LIMIT = 80 + PANEL_LIMIT = 4 + THINKING_EVENT_LIMIT = 200 + ACTIVITY_SUMMARY_EVENT_LIMIT = 200 + RAW_TRACE_OUTPUT_LINE_PATTERNS = %w[ + reasoning + thinking + tooluse + tool_use + toolresult + tool_result + ].freeze + COMPLETED_TRACE_METHOD_PATTERNS = %w[ + item/completed + item.completed + ].freeze + COMPLETED_TRACE_ITEM_PATTERNS = %w[ + commandexecution + command_execution + mcptoolcall + mcp_tool_call + toolcall + tool_call + tooluse + tool_use + functioncall + function_call + filechange + file_change + ].freeze + TOOL_TRACE_ITEM_TYPES = %w[ + commandExecution + command_execution + mcpToolCall + mcp_tool_call + toolCall + tool_call + toolUse + tool_use + functionCall + function_call + fileChange + file_change + ].freeze + # Messages and thinking precede the terminal event for a same-timestamp tie. + TRANSCRIPT_SOURCE_ORDER = { message: 0, thinking: 1, event: 2 }.freeze + SLACK_PROVIDER = Oauth::Providers::Slack::KEY + SLACK_THREAD_OWNER_METADATA_KEYS = %w[slack_user_id actor_user_id user_id].freeze + SLACK_THREAD_TEAM_METADATA_KEYS = %w[slack_team_id team_id home_team_id].freeze + SLACK_CREDENTIAL_USER_LABEL_KEYS = %w[slack_user_id].freeze + SLACK_CREDENTIAL_EMAIL_LABEL_KEYS = %w[email slack_email].freeze + SLACK_TEAM_LABEL = "slack_team_id" + CONSOLE_THREAD_OWNER_METADATA_KEYS = %w[actor_email user_email].freeze + SLACK_USER_ID_PATTERN = /\A[UW][A-Z0-9]+\z/.freeze + SLACK_MENTION_PATTERN = /<@([UW][A-Z0-9]+)(?:\|([^>]+))?>|@([UW][A-Z0-9]+)/.freeze + # Deploy-time default-model overrides: the same env vars deployers set in + # sandbox.extraEnv to change the harness model, mirrored onto the Console by + # the chart. Amp has no fixed default model, so it is intentionally absent. + HARNESS_DEFAULT_MODEL_ENVS = { + "claudecode" => "CLAUDE_MODEL", + "codex" => "CODEX_MODEL" + }.freeze + # Harness config files carrying each harness's baked-in default model, used + # when no env override is set. Resolved against CENTAUR_HARNESS_CONFIG_DIR + # (the sandbox entrypoint's variable) or the repo checkout's harness/ + # directory; absent files (e.g. in the production image, whose build context + # is services/console) simply yield no default. + HARNESS_CONFIG_FILES = { + "claudecode" => "claude/settings.json", + "codex" => "codex/config.toml" + }.freeze + + SlackThreadOwner = Struct.new(:user_id, :team_id, keyword_init: true) + + # Pseudo thread key that opens a new-chat composer pane in the split view. + NEW_PANE_KEY = "new".freeze + + # The composer's model selector, in display order. Each entry pins the + # harness the choice runs on (wire values match api-rs's HarnessType enum, + # serde lowercase); the model ids are the ones the bots' --model flags + # expand to (services/slackbotv2/src/overrides.ts). Amp appears as a plain + # entry with no model: it picks its own model per turn. `efforts` are the + # per-turn reasoning efforts the harness accepts for the model (codex only — + # harness-server discards `reasoning` for claude/amp; enum per + # crates/harness-server/src/codex.rs, `max` being 5.6-specific). + ComposerAgent = Struct.new(:value, :label, :harness, :model, :efforts, keyword_init: true) + CODEX_EFFORTS = [ + %w[minimal Minimal], + %w[low Low], + %w[medium Medium], + %w[high High], + [ "xhigh", "Extra High" ] + ].freeze + # First entry doubles as the default pick (unless the deploy's default-model + # resolution for its harness names another listed model). + COMPOSER_AGENTS = [ + ComposerAgent.new(value: "gpt-5.6-sol", label: "GPT-5.6 Sol", + harness: "codex", model: "gpt-5.6-sol", + efforts: CODEX_EFFORTS + [ %w[max Max] ]), + ComposerAgent.new(value: "gpt-5.5", label: "GPT-5.5", + harness: "codex", model: "gpt-5.5", + efforts: CODEX_EFFORTS), + ComposerAgent.new(value: "claude-opus-4-8", label: "Claude Opus 4.8", + harness: "claudecode", model: "claude-opus-4-8", efforts: []), + ComposerAgent.new(value: "claude-sonnet-4-6", label: "Claude Sonnet 4.6", + harness: "claudecode", model: "claude-sonnet-4-6", efforts: []), + ComposerAgent.new(value: "claude-haiku-4-5", label: "Claude Haiku 4.5", + harness: "claudecode", model: "claude-haiku-4-5", efforts: []), + ComposerAgent.new(value: "claude-fable-5", label: "Claude Fable 5", + harness: "claudecode", model: "claude-fable-5", efforts: []), + ComposerAgent.new(value: "amp", label: "Amp", + harness: "amp", model: nil, efforts: []) + ].freeze + + helper_method :thread_title, + :thread_source_icon, + :thread_source_label, + :thread_harness_label, + :thread_model_label, + :thread_user_label, + :thread_message_text, + :thread_text_preview, + :thread_status_classes, + :composer_agent_choices, + :composer_default_agent_value, + :composer_agents_json, + :thread_execution_active? + + def index + @query = params[:q].to_s.strip + requested_keys = requested_thread_keys + # "new" is a sentinel pane key: Cmd-clicking the sidebar's New chat adds a + # composer pane to the split view the same way thread keys add threads. On + # its own it is just the full-page new-chat screen. + @new_chat_pane_index = requested_keys.index(NEW_PANE_KEY) if requested_keys.size > 1 + thread_keys = requested_keys - [ NEW_PANE_KEY ] + @selected_thread_key = thread_keys.first.to_s + @pane_thread_keys = thread_keys.drop(1) + @starting_new_thread = params[:new].present? || requested_keys == [ NEW_PANE_KEY ] + @thread_db_unavailable = false + @thread_not_found = false + + load_threads + if @thread_not_found + render status: :not_found + return + end + redirect_to_first_thread if auto_select_first_thread? + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.warn("console_threads_load_failed error=#{e.class}: #{e.message}") + empty_thread_state + @thread_db_unavailable = true + end + + # Composer submit: no thread_key starts a new chat (create session + first + # message + execute), a thread_key sends a follow-up into an existing chat. + # Both paths talk to api-rs through CentaurApiClient; the transcript itself + # is still read from the sessions DB by #index after the redirect. + def create + thread_key = params[:thread_key].to_s.strip.presence + + prompt = params[:prompt].to_s.strip + if prompt.blank? + redirect_to( + thread_key ? console_threads_path(thread: reply_redirect_keys(thread_key)) : console_threads_path(new: 1), + alert: "Type a message first." + ) + return + end + + thread_key ? reply_to_thread(thread_key, prompt) : start_thread(prompt) + end + + # Lazily-loaded sidebar thread list, requested by the Turbo Frame in + # layouts/console.html.erb. Runs the cross-database sessions query out of band + # so it never blocks the primary page render. Renders only the frame partial + # (no layout). DB errors leave the list empty via load_console_sidebar_threads. + def sidebar + load_console_sidebar_threads + render partial: "console/threads/sidebar_threads", layout: false + end + + private + + def api_client + @api_client ||= client_factory.call + end + + # Whether the thread's newest execution is still running — drives the + # transcript's thinking indicator and the while-running auto-refresh. + def thread_execution_active?(thread_key) + execution = @latest_executions&.[](thread_key) + execution.present? && %w[queued running executing].include?(execution.status.to_s) + end + + # Selector options as [label, value] pairs, the deploy's default model + # first (pre-checked in the menu). The default comes from the same + # env/config resolution the thread header uses, so the composer never + # claims a default the sandbox would not actually run. + def composer_agent_choices + default_value = composer_default_agent_value + COMPOSER_AGENTS + .sort_by.with_index { |agent, index| agent.value == default_value ? -1 : index } + .map { |agent| [ agent.label, agent.value ] } + end + + def composer_default_agent_value + default = default_model_for_harness(COMPOSER_AGENTS.first.harness) + COMPOSER_AGENTS.find { |agent| agent.value == default }&.value || + COMPOSER_AGENTS.first.value + end + + # Per-agent metadata the picker script needs to rebuild the effort submenu + # when the model changes: { value => { label:, efforts: [[value, label]] } }. + def composer_agents_json + COMPOSER_AGENTS.to_h do |agent| + [ agent.value, { label: agent.label, efforts: agent.efforts } ] + end.to_json + end + + def composer_effort_param(agent) + effort = params[:effort].to_s.strip + return nil if effort.blank? + + agent.efforts.map(&:first).include?(effort) ? effort : nil + end + + def composer_agent_for(raw) + value = raw.to_s.strip + value = composer_default_agent_value if value.blank? + COMPOSER_AGENTS.find { |agent| agent.value == value } + end + + def start_thread(prompt) + agent = composer_agent_for(params[:model]) + if agent.nil? + redirect_to console_threads_path(new: 1), + alert: "Unknown model #{params[:model].to_s.inspect}." + return + end + + thread_key = "console:#{SecureRandom.uuid}" + api_client.create_session( + thread_key: thread_key, + harness_type: agent.harness, + metadata: console_actor_metadata.merge(agent.model.present? ? { model: agent.model } : {}) + ) + send_prompt(thread_key, prompt, model: agent.model, effort: composer_effort_param(agent)) + # A new-chat pane in a split view swaps the sentinel for the created + # thread so the other panes stay open. + open_keys = params[:open_threads].to_s.split(",").map(&:strip).reject(&:blank?) + redirect_keys = open_keys.include?(NEW_PANE_KEY) ? + open_keys.map { |key| key == NEW_PANE_KEY ? thread_key : key } : [ thread_key ] + redirect_to console_threads_path(thread: redirect_keys.uniq.first(PANEL_LIMIT).join(",")) + rescue CentaurApiClient::Error => e + redirect_to console_threads_path(new: 1), alert: "Could not start the chat: #{e.message}" + end + + def reply_to_thread(thread_key, prompt) + # Resolve through the owner scope so a crafted thread_key cannot post into + # another user's chat — same rule the read side applies to ?thread=. + session = visible_thread_scope.where(thread_key: thread_key).first + if session.nil? + redirect_to console_threads_path, alert: "Chat not found." + return + end + + send_prompt(session.thread_key, prompt, model: reply_model_for(session)) + redirect_to console_threads_path(thread: reply_redirect_keys(session.thread_key)) + rescue CentaurApiClient::Error => e + redirect_to console_threads_path(thread: reply_redirect_keys(thread_key)), + alert: "Could not send the message: #{e.message}" + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.warn("console_threads_reply_lookup_failed error=#{e.class}: #{e.message}") + redirect_to console_threads_path, alert: "Chat database is unavailable." + end + + # Append persists the turn in conversation history; execute runs it. The + # shared client_message_id lets api-rs dedupe the copy of the message the + # harness echoes back. + def send_prompt(thread_key, prompt, model: nil, effort: nil) + message_id = SecureRandom.uuid + + api_client.append_session_messages( + thread_key: thread_key, + messages: [ + { + client_message_id: message_id, + role: "user", + parts: [ { type: "text", text: prompt } ], + metadata: console_actor_metadata + } + ] + ) + + execute_metadata = console_actor_metadata.merge(action: "execute") + execute_metadata[:model] = model if model.present? + execute_metadata[:reasoning] = effort if effort.present? + api_client.execute_session( + thread_key: thread_key, + idempotency_key: SecureRandom.uuid, + metadata: execute_metadata, + input_lines: [ + composer_input_line( + thread_key, prompt, + model: model, effort: effort, client_message_id: message_id + ) + ] + ) + end + + # One blocks-protocol user line, the shape harness-server parses from + # execute's input_lines. `model` is honored by every harness; omitted (e.g. + # for Amp) the harness runs its own default. `reasoning` is the per-turn + # codex effort; other harnesses discard it, and validation upstream only + # accepts it for codex models anyway. + def composer_input_line(thread_key, prompt, model:, effort:, client_message_id:) + line = { + type: "user", + thread_key: thread_key, + client_user_message_id: client_message_id, + trace_metadata: { action: "execute", source: "console" }, + message: { role: "user", content: [ { type: "text", text: prompt } ] } + } + line[:model] = model if model.present? + line[:reasoning] = effort if effort.present? + line.to_json + end + + # Follow-ups reuse the model the chat has been running on (mirrors the + # display resolution in thread_model_label, minus the upcasing): last + # execution's recorded model, session metadata, then the deploy default. + def reply_model_for(session) + recorded_model(latest_executions_for([ session.thread_key ])[session.thread_key]&.metadata) || + recorded_model(session.metadata_hash) || + default_model_for_harness(session.harness_type.to_s) + end + + # Keeps split-view panes open across a composer submit: the form carries the + # page's full ?thread= list, and the redirect re-orders it so the posted + # thread stays primary. Unowned keys are filtered again by #index on render. + def reply_redirect_keys(thread_key) + open_keys = params[:open_threads].to_s.split(",").map(&:strip).reject(&:blank?) + ([ thread_key ] + open_keys).uniq.first(PANEL_LIMIT).join(",") + end + + def console_actor_metadata + email = current_user&.email.to_s + { + platform: "console", + source: "console", + user_email: email, + actor_email: email + } + end + + def load_threads + session_scope = visible_thread_scope + base_sessions = session_scope.recent_first.limit(THREAD_LIMIT).to_a + keys = base_sessions.map(&:thread_key).uniq + + @latest_messages = latest_messages_for(keys) + @latest_executions = latest_executions_for(keys) + @message_counts = count_records(CentaurSessionMessage, keys) + @execution_counts = count_records(CentaurSessionExecution, keys) + + @sessions = base_sessions.select { |session| matches_query?(session) } + @selected_session = selected_session(session_scope, base_sessions) + if @thread_not_found + @pane_sessions = [] + @thread_panels = [] + @selected_messages = [] + @selected_executions = [] + @selected_events = [] + @selected_transcript_items = [] + return + end + @pane_sessions = resolve_pane_sessions(session_scope, base_sessions) + load_selected_session_summaries(keys) + @selected_thread_key = @selected_session&.thread_key.to_s + @thread_panels = build_thread_panels + @selected_transcript_items = @thread_panels.first&.dig(:transcript_items) || [] + end + + def empty_thread_state + @thread_not_found = false + @sessions = [] + @selected_session = nil + @pane_sessions = [] + @thread_panels = [] + @selected_messages = [] + @selected_executions = [] + @selected_events = [] + @selected_transcript_items = [] + @latest_messages = {} + @latest_executions = {} + @message_counts = {} + @execution_counts = {} + end + + def matches_query?(session) + return true if @query.blank? + + needle = @query.downcase + [ + session.thread_key, + thread_title(session), + thread_source_label(session), + thread_user_label(session), + thread_text_preview(@latest_messages[session.thread_key]) + ].any? { |value| value.to_s.downcase.include?(needle) } + end + + def selected_session(session_scope, base_sessions) + return nil if @starting_new_thread + + if @selected_thread_key.present? + selected = base_sessions.find { |session| session.thread_key == @selected_thread_key } + # Resolve the key through the owner scope so a directly linked chat only + # loads when the current user started it. base_sessions is capped at + # THREAD_LIMIT, so this also recovers an owned thread beyond that window. + selected ||= session_scope.where(thread_key: @selected_thread_key).first + # A directly requested key outside the owner scope renders as 404 rather + # than silently falling back to another chat, so nonexistent and + # inaccessible chats are indistinguishable to the viewer. + @thread_not_found = selected.nil? + return selected + end + @sessions.first + end + + def auto_select_first_thread? + params[:thread].blank? && !@starting_new_thread && @query.blank? && @selected_session.present? + end + + # The thread param carries up to PANEL_LIMIT comma-separated thread keys; the + # first is the primary thread and the rest are extra split-view panes + # (Cmd/Ctrl-click on a sidebar thread appends its key). + def requested_thread_keys + params[:thread].to_s.split(",").map(&:strip).reject(&:blank?).uniq.first(PANEL_LIMIT) + end + + # Extra split-view panes resolve through the same owner scope as the primary + # thread, so a crafted ?thread= list cannot surface another user's thread. + # Unowned keys are dropped silently. + def resolve_pane_sessions(session_scope, base_sessions) + keys = @pane_thread_keys - [ @selected_session&.thread_key ] + keys.filter_map do |key| + base_sessions.find { |session| session.thread_key == key } || + session_scope.where(thread_key: key).first + end + end + + def build_thread_panels + sessions = ([ @selected_session ] + Array(@pane_sessions)).compact + .uniq(&:thread_key) + .first(PANEL_LIMIT) + panels = if sessions.empty? + [] + else + # Build the primary panel last so the @selected_* thread state (used by + # the page header and mention-resolution memos) ends on the primary + # thread. + extra_panels = sessions.drop(1).map { |session| thread_panel_for(session) } + [ thread_panel_for(sessions.first) ] + extra_panels + end + + if @new_chat_pane_index && panels.any? + panels.insert( + [ @new_chat_pane_index, panels.size ].min, + { new_chat: true, thread_key: NEW_PANE_KEY, session: nil, transcript_items: [] } + ) + end + panels + end + + def thread_panel_for(session) + @selected_session = session + @selected_messages = selected_messages + @selected_executions = selected_executions + @selected_events = selected_events + reset_selected_thread_memos + + { + session: session, + thread_key: session.thread_key, + transcript_items: selected_transcript_items + } + end + + # Mention labels and inferred bot ids are memoized off the selected thread's + # messages and events, so they must be recomputed per panel. + def reset_selected_thread_memos + @slack_mention_labels_by_id = nil + @slack_bot_user_ids = nil + end + + def redirect_to_first_thread + redirect_to console_threads_path(thread: @selected_session.thread_key) + end + + def load_selected_session_summaries(loaded_keys) + missing_keys = ([ @selected_session ] + Array(@pane_sessions)).compact + .map(&:thread_key) + .uniq + .reject { |key| loaded_keys.include?(key) } + return if missing_keys.empty? + + @latest_messages.merge!(latest_messages_for(missing_keys)) + @latest_executions.merge!(latest_executions_for(missing_keys)) + @message_counts.merge!(count_records(CentaurSessionMessage, missing_keys)) + @execution_counts.merge!(count_records(CentaurSessionExecution, missing_keys)) + end + + def visible_thread_scope + slack_owners = slack_thread_owners_for_current_user + conditions = [ + console_thread_owner_sql, + (slack_thread_owner_sql(slack_owners) if slack_owners.any?) + ].compact + + return CentaurSession.where("1=0") if conditions.empty? + + CentaurSession.where(conditions.map { |condition| "(#{condition})" }.join(" OR ")) + end + + def console_thread_owner_sql + email = normalize_email(current_user&.email) + return if email.blank? + + console_source = [ + "thread_key LIKE 'console:%'", + "metadata ->> 'platform' = 'console'", + "metadata ->> 'source' = 'console'" + ].join(" OR ") + owner_clauses = CONSOLE_THREAD_OWNER_METADATA_KEYS.map do |key| + "lower(metadata ->> #{sql_quote(key)}) = #{sql_quote(email)}" + end + + "(#{console_source}) AND (#{owner_clauses.join(" OR ")})" + end + + def slack_thread_owners_for_current_user + @slack_thread_owners_for_current_user ||= begin + if current_user + subjects = slack_identity_subjects_for_current_user + emails = slack_identity_emails_for_current_user + + if subjects.empty? && emails.empty? + [] + else + credentials = BrokerCredential + .joins(:oauth_app) + .includes(:oauth_app) + .where(oauth_apps: { provider: SLACK_PROVIDER }) + .where(slack_oauth_credential_owner_sql(subjects: subjects, emails: emails)) + + credential_owners = credentials.filter_map do |credential| + user_id = first_present( + credential.provider_subject, + *SLACK_CREDENTIAL_USER_LABEL_KEYS.map { |key| credential.labels&.[](key) } + ) + next if user_id.blank? + + SlackThreadOwner.new( + user_id: user_id, + team_id: first_present( + credential.labels&.[](SLACK_TEAM_LABEL), + credential.oauth_app&.labels&.[](SLACK_TEAM_LABEL) + ) + ) + end + + # A Slack OIDC sign-in stores the workspace user id (U…) as the + # identity subject — the same id slackbotv2 writes into session + # metadata — so SSO alone owns those threads even when the user has + # not minted a broker credential through the connect flow. + identity_owners = subjects.map do |subject| + SlackThreadOwner.new(user_id: subject, team_id: nil) + end + + (credential_owners + identity_owners) + .uniq { |owner| [ normalize_key(owner.user_id), normalize_key(owner.team_id) ] } + end + else + [] + end + end + end + + def slack_identity_subjects_for_current_user + current_user.user_identities + .where(provider: SLACK_PROVIDER) + .pluck(:subject) + .filter_map { |value| normalize_key(value) } + .uniq + end + + def slack_identity_emails_for_current_user + ([ current_user.email ] + current_user.user_identities.where(provider: SLACK_PROVIDER).pluck(:email)) + .filter_map { |value| normalize_email(value) } + .uniq + end + + def slack_oauth_credential_owner_sql(subjects:, emails:) + clauses = [] + if subjects.any? + subject_values = sql_list(subjects) + clauses << "lower(broker_credentials.provider_subject) IN (#{subject_values})" + SLACK_CREDENTIAL_USER_LABEL_KEYS.each do |key| + clauses << "lower(broker_credentials.labels ->> #{sql_quote(key)}) IN (#{subject_values})" + end + end + + if emails.any? + email_values = sql_list(emails) + clauses << "lower(broker_credentials.provider_email) IN (#{email_values})" + SLACK_CREDENTIAL_EMAIL_LABEL_KEYS.each do |key| + clauses << "lower(broker_credentials.labels ->> #{sql_quote(key)}) IN (#{email_values})" + end + end + + clauses.join(" OR ") + end + + def slack_thread_owner_sql(owners) + slack_source = [ + "thread_key LIKE 'slack:%'", + "metadata ->> 'platform' = 'slack'", + "metadata ->> 'source' = 'slackbotv2'" + ].join(" OR ") + + owner_clauses = owners.map do |owner| + user_id = normalize_key(owner.user_id) + user_clauses = SLACK_THREAD_OWNER_METADATA_KEYS.map do |key| + "lower(metadata ->> #{sql_quote(key)}) = #{sql_quote(user_id)}" + end + owner_clause = "(#{user_clauses.join(" OR ")})" + + # Team scoping narrows the match only when the owning credential exposes a + # team. slackbotv2 uses slack:CHANNEL:TS thread keys and does not record a + # slack_team_id, so requiring a team would hide otherwise-owned threads. + if owner.team_id.present? + team_id = normalize_key(owner.team_id) + team_clauses = SLACK_THREAD_TEAM_METADATA_KEYS.map do |key| + "lower(metadata ->> #{sql_quote(key)}) = #{sql_quote(team_id)}" + end + team_clauses << "lower(split_part(thread_key, ':', 2)) = #{sql_quote(team_id)}" + owner_clause = "(#{owner_clause} AND (#{team_clauses.join(" OR ")}))" + end + + owner_clause + end + + "(#{slack_source}) AND (#{owner_clauses.join(" OR ")})" + end + + def first_present(*values) + values.find(&:present?) + end + + def normalize_key(value) + value.to_s.strip.downcase.presence + end + + def normalize_email(value) + value.to_s.strip.downcase.presence + end + + def sql_list(values) + values.map { |value| sql_quote(value) }.join(", ") + end + + def sql_quote(value) + ActiveRecord::Base.connection.quote(value.to_s) + end + + def selected_messages + return [] unless @selected_session + + # Fetch the newest MESSAGE_LIMIT messages, then reverse for oldest-first + # display. Ordering ascending before LIMIT would return the OLDEST N and + # drop the newest for long threads (mirrors selected_events below). + CentaurSessionMessage + .where(thread_key: @selected_session.thread_key) + .order(created_at: :desc, message_id: :desc) + .limit(MESSAGE_LIMIT) + .to_a + .reverse + end + + def selected_executions + return [] unless @selected_session + + CentaurSessionExecution + .where(thread_key: @selected_session.thread_key) + .order(created_at: :desc, execution_id: :desc) + .limit(EXECUTION_LIMIT) + .to_a + end + + def selected_events + return [] unless @selected_session + + CentaurSessionEvent + .where(thread_key: @selected_session.thread_key) + .where(event_type: %w[ + session.execution_completed + session.execution_failed + session.execution_cancelled + ]) + .order(event_id: :desc) + .limit(TRANSCRIPT_EVENT_LIMIT) + .to_a + .reverse + end + + def selected_transcript_items + message_items = @selected_messages.map { |message| transcript_item_for_message(message) } + + event_items = @selected_events.filter_map { |event| transcript_item_for_event(event) } + + thinking_items = selected_thinking_items + + (message_items + thinking_items + event_items).sort_by do |item| + [ item[:created_at] || Time.zone.at(0), TRANSCRIPT_SOURCE_ORDER[item[:source]] || 0 ] + end + end + + # The api-rs stdout pump persists every harness output line verbatim as a + # session.output.line event whose payload is a JSON-encoded string. Codex + # reasoning arrives as item/completed notifications with item.type == + # "reasoning" carrying the full accumulated thinking text; tool activity + # arrives as completed command/tool items. Claude Code stream-json persists + # each assistant API message whose content can include "thinking" and + # "tool_use" blocks. The SQL LIKE filter keeps the query from paging through + # the whole firehose; exact matching happens here. + def selected_thinking_items + return [] unless @selected_session + + items = CentaurSessionEvent + .where(thread_key: @selected_session.thread_key) + .where(event_type: "session.output.line") + .where(trace_output_line_filter_sql, *trace_output_line_filter_values) + .order(event_id: :desc) + .limit(THINKING_EVENT_LIMIT) + .to_a + .reverse + .filter_map { |event| thinking_transcript_item(event) } + + apply_activity_summaries(compact_trace_items(items)) + end + + # api-rs's activity-summary worker condenses harness output into short + # first-person status lines persisted as session.activity_summary events, + # each pointing at the output-line event that triggered it via + # source_event_id. A summary belongs to the latest trace item at or before + # its source line, so each disclosure's collapsed preview shows the newest + # status generated during that block; items no summary covers keep the + # raw-text fallback rendered by the transcript partial. + def apply_activity_summaries(items) + anchored = items.select { |item| item[:event_id] } + return items if anchored.empty? + + selected_activity_summaries.each do |event| + payload = event.payload_hash + summary = payload["summary"].to_s.strip + source_event_id = payload["source_event_id"] + next if summary.blank? || source_event_id.nil? + + item = anchored.reverse_each.find { |candidate| candidate[:event_id] <= source_event_id.to_i } + item[:summary] = summary if item + end + items + end + + def selected_activity_summaries + return [] unless @selected_session + + CentaurSessionEvent + .where(thread_key: @selected_session.thread_key) + .where(event_type: "session.activity_summary") + .order(event_id: :desc) + .limit(ACTIVITY_SUMMARY_EVENT_LIMIT) + .to_a + .reverse + end + + def thinking_transcript_item(event) + line = event.payload + return nil unless line.is_a?(String) + + value = JSON.parse(line) + return nil unless value.is_a?(Hash) + + trace = reasoning_trace(value) || claude_thinking_trace(value) || tool_trace(value) + return nil unless trace + + { + role: "thinking", + label: trace[:label], + align: :start, + text: trace[:text], + trace_kind: trace[:kind] || "thinking", + commands: trace[:commands], + tools: trace[:tools], + execution_id: event.execution_id, + event_id: event.event_id, + created_at: event.created_at, + source: :thinking + } + rescue JSON::ParserError + nil + end + + def compact_trace_items(items) + grouped = [] + command_group = [] + + flush_command_group = lambda do + grouped << command_trace_group(command_group) if command_group.any? + command_group = [] + end + + items.each do |item| + if item[:trace_kind] == "command" && + (command_group.empty? || same_trace_group?(command_group.last, item)) + command_group << item + else + flush_command_group.call + item[:trace_kind] == "command" ? command_group << item : grouped << item + end + end + + flush_command_group.call + grouped + end + + def same_trace_group?(left, right) + left_execution = left[:execution_id].presence + right_execution = right[:execution_id].presence + return left_execution == right_execution if left_execution && right_execution + + # Older imported fixtures can lack execution ids. Keep immediately adjacent + # command traces together, but avoid merging activity from distinct turns. + left_time = left[:created_at] + right_time = right[:created_at] + left_time.present? && right_time.present? && (right_time - left_time).abs <= 5.minutes + end + + def command_trace_group(items) + commands = items.flat_map { |item| Array(item[:commands]) } + failed_count = commands.count { |command| command[:failed] } + command_count = commands.length + + { + role: "thinking", + label: "Ran #{pluralized_count(command_count, "command")}", + failed_label: failed_count.positive? ? "#{failed_count} failed" : nil, + align: :start, + text: command_group_text(commands), + trace_kind: "commands", + commands: commands, + execution_id: items.first[:execution_id], + event_id: items.first[:event_id], + created_at: items.first[:created_at], + source: :thinking + } + end + + def command_group_text(commands) + commands.map do |command| + [ + "$ #{command[:command]}", + ("Status: #{command[:status]}" if command[:status].present?), + ("Exit code: #{command[:exit_code]}" if command[:exit_code].present?), + command[:output] + ].compact.join("\n") + end.join("\n\n").strip + end + + def pluralized_count(count, singular) + "#{count} #{singular}#{count == 1 ? "" : "s"}" + end + + def trace_output_line_filter_sql + @trace_output_line_filter_sql ||= begin + raw = RAW_TRACE_OUTPUT_LINE_PATTERNS.map { "lower(payload::text) LIKE ?" }.join(" OR ") + completed_methods = + COMPLETED_TRACE_METHOD_PATTERNS.map { "lower(payload::text) LIKE ?" }.join(" OR ") + completed_items = + COMPLETED_TRACE_ITEM_PATTERNS.map { "lower(payload::text) LIKE ?" }.join(" OR ") + "(#{raw}) OR ((#{completed_methods}) AND (#{completed_items}))" + end + end + + def trace_output_line_filter_values + @trace_output_line_filter_values ||= begin + patterns = + RAW_TRACE_OUTPUT_LINE_PATTERNS + + COMPLETED_TRACE_METHOD_PATTERNS + + COMPLETED_TRACE_ITEM_PATTERNS + patterns.map { |pattern| "%#{pattern}%" } + end + end + + def reasoning_trace(value) + text = reasoning_event_text(value) + return nil if text.blank? + + { label: "Thinking", text: text } + end + + def reasoning_event_text(value) + method = (value["method"] || value["type"]).to_s.tr("/", ".") + return nil unless method == "item.completed" + + item = value.dig("params", "item") || value["item"] + return nil unless item.is_a?(Hash) && item["type"].to_s == "reasoning" + + reasoning_item_text(item) + end + + # Claude Code's stream-json output persists each assistant API message as + # {"type":"assistant","message":{"content":[...]}} where extended thinking + # arrives in content blocks of type "thinking" (text under the "thinking" + # key). Partial stream_event lines never carry type == "assistant", so each + # thinking block surfaces exactly once. + def claude_thinking_trace(value) + text = claude_thinking_text(value) + return nil if text.blank? + + { label: "Thinking", text: text } + end + + def claude_thinking_text(value) + return nil unless value["type"].to_s == "assistant" + + message = value["message"] + content = message.is_a?(Hash) ? message["content"] : value["content"] + return nil unless content.is_a?(Array) + + content.filter_map do |part| + next unless part.is_a?(Hash) && part["type"].to_s == "thinking" + + part["thinking"].presence || part["text"].presence + end.join("\n").strip.presence + end + + def tool_trace(value) + completed_item_trace(value) || claude_tool_use_trace(value) || claude_tool_result_trace(value) + end + + def completed_item_trace(value) + method = (value["method"] || value["type"]).to_s.tr("/", ".") + return nil unless method == "item.completed" + + item = value.dig("params", "item") || value["item"] + return nil unless item.is_a?(Hash) + + case item["type"].to_s + when "commandExecution", "command_execution" + command_execution_trace(item) + when *TOOL_TRACE_ITEM_TYPES + generic_tool_item_trace(item) + end + end + + def command_execution_trace(item) + command = first_present(item["command"], item["cmd"]) + output = first_present( + item["aggregatedOutput"], + item["aggregated_output"], + item["output"], + item["stdout"], + item["stderr"] + ) + exit_code = first_present(item["exitCode"], item["exit_code"]) + status = first_present(item["status"], exit_code.present? ? "completed" : nil) + + sections = [] + sections << "Status: #{status}" if status.present? + sections << "Exit code: #{exit_code}" if exit_code.present? + sections << markdown_code_block(command, language: shell_language_for_command(command)) if command.present? + sections << "Output:\n\n#{markdown_code_block(output, language: "text")}" if output.present? + + text = sections.compact.join("\n\n").strip + return nil if text.blank? + + { + kind: "command", + label: "Ran 1 command", + text: text, + commands: [ + { + command: command.to_s, + output: output.to_s, + exit_code: exit_code, + status: status, + failed: command_failed?(status, exit_code) + } + ] + } + end + + def command_failed?(status, exit_code) + status.to_s.match?(/\A(?:failed|error|cancelled|timed_out)\z/i) || + (exit_code.present? && exit_code.to_i != 0) + end + + def generic_tool_item_trace(item) + label = trace_label_for_item(item) + name = first_present(item["name"], item["tool"], item["toolName"], item["tool_name"]) + input = item["input"] || item["arguments"] || item["args"] + output = item["output"] || item["result"] + + sections = [] + sections << "Status: #{item["status"]}" if item["status"].present? + sections << "Name: #{name}" if name.present? + sections << "Input:\n\n#{markdown_code_block(pretty_json(input))}" if input.present? + sections << "Output:\n\n#{markdown_code_block(pretty_json(output))}" if output.present? + + text = sections.compact.join("\n\n").strip + return nil if text.blank? + + { label: label, text: text } + end + + def claude_tool_use_trace(value) + return nil unless value["type"].to_s == "assistant" + + content = message_content(value) + return nil unless content.is_a?(Array) + + traces = content.filter_map do |part| + next unless part.is_a?(Hash) && part["type"].to_s == "tool_use" + + name = first_present(part["name"], part["tool"], "tool") + input = part["input"] || part["arguments"] + [ + "Use #{name}", + ("Input:\n\n#{markdown_code_block(pretty_json(input))}" if input.present?) + ].compact.join("\n\n") + end + + text = traces.join("\n\n").strip + return nil if text.blank? + + { label: traces.size == 1 ? "Tool call" : "Tool calls", text: text } + end + + def claude_tool_result_trace(value) + return nil unless %w[user tool].include?(value["type"].to_s) + + content = message_content(value) + return nil unless content.is_a?(Array) + + traces = content.filter_map do |part| + next unless part.is_a?(Hash) + next unless part["type"].to_s == "tool_result" || part["tool_use_id"].present? + + body = first_present(part["content"], part["text"], part["result"]) + next if body.blank? + + [ + ("Tool use: #{part["tool_use_id"]}" if part["tool_use_id"].present?), + markdown_code_block(pretty_json(body), language: "text") + ].compact.join("\n\n") + end + + text = traces.join("\n\n").strip + return nil if text.blank? + + { label: traces.size == 1 ? "Tool result" : "Tool results", text: text } + end + + def message_content(value) + message = value["message"] + message.is_a?(Hash) ? message["content"] : value["content"] + end + + def trace_label_for_item(item) + case item["type"].to_s + when "fileChange", "file_change" then "File change" + when "mcpToolCall", "mcp_tool_call" then "Tool call" + else "Tool call" + end + end + + def markdown_code_block(value, language: nil) + body = value.to_s.rstrip + return nil if body.blank? + + fence = "```" + fence += "`" while body.include?(fence) + "#{fence}#{language}\n#{body}\n#{fence}" + end + + def pretty_json(value) + case value + when String + value + else + JSON.pretty_generate(value) + end + rescue JSON::GeneratorError + value.to_s + end + + def shell_language_for_command(command) + command.to_s.match?(/\A(?:SELECT|WITH|INSERT|UPDATE|DELETE)\b/i) ? "sql" : "sh" + end + + # Claude/Amp reasoning lands in content (full text); Codex-native reasoning + # may only carry a summary. Prefer the fullest field available. + def reasoning_item_text(item) + [ + item["text"], + reasoning_part_text(item["content"]), + reasoning_part_text(item["summary"]) + ].find(&:present?) + end + + def reasoning_part_text(value) + entries = value.is_a?(Array) ? value : [ value ] + entries.filter_map do |part| + case part + when String then part + when Hash then part["text"].to_s + end + end.join("\n").strip.presence + end + + def latest_messages_for(keys) + return {} if keys.empty? + + CentaurSessionMessage + .where(thread_key: keys) + .select("distinct on (thread_key) session_messages.*") + .order(Arel.sql("thread_key, created_at desc, message_id desc")) + .index_by(&:thread_key) + end + + def latest_executions_for(keys) + return {} if keys.empty? + + CentaurSessionExecution + .where(thread_key: keys) + .select("distinct on (thread_key) session_executions.*") + .order(Arel.sql("thread_key, created_at desc, execution_id desc")) + .index_by(&:thread_key) + end + + def transcript_item_for_message(message) + metadata = message_metadata_hash(message) + + { + role: message.role, + label: transcript_message_label(message.role, metadata), + align: transcript_message_align(message.role, metadata), + text: resolve_slack_mentions(thread_message_text(message)), + created_at: message.created_at, + source: :message + } + end + + def count_records(model, keys) + return {} if keys.empty? + + model.where(thread_key: keys).group(:thread_key).count + end + + def thread_title(session) + stored = stored_session_title(session) + return clip_one_line(stored, 80) if stored + + metadata = session.metadata_hash + summary = metadata["summary"] + title = metadata["title"].presence || + metadata["generated_title"].presence || + metadata["summary_title"].presence || + metadata["thread_title"].presence || + (metadata["thread"].is_a?(Hash) ? metadata["thread"]["title"] : nil).presence || + (metadata["summary"].is_a?(Hash) ? metadata["summary"]["title"] : nil).presence || + (summary if summary.is_a?(String)).presence || + metadata["subject"].presence || + metadata["issue_title"].presence + return generated_thread_title(title) if title + + preview = thread_text_preview(@latest_messages[session.thread_key]) + generated = generated_thread_title(preview) + return generated if generated.present? + + human_thread_key(session.thread_key) + end + + # The title api-rs generates and writes onto sessions.title after a message + # append. Guarded because sessions mirrored from a snapshot taken before the + # title migration have no such column. + def stored_session_title(session) + session.title.presence if session.respond_to?(:title) + end + + def thread_source_icon(session) + thread_source_key(session) == "slack" ? "slack" : "computer" + end + + def thread_source_label(session) + source_label(thread_source_key(session)) + end + + def thread_harness_label(session) + case session.harness_type.to_s + when "codex" then "Codex" + when "claudecode" then "Claude Code" + when "amp" then "Amp" + else source_label(session.harness_type) + end + end + + # Model the thread most recently ran on. slackbotv2 records the effective + # model in execution metadata; for older rows without it, fall back to the + # deployment's default the way the sandbox resolves it: CLAUDE_MODEL / + # CODEX_MODEL env override first, then the model pinned in the harness + # config files when they are present. Nil (segment omitted) when none of + # those sources know the model. + def thread_model_label(session) + model = recorded_model(@latest_executions&.[](session.thread_key)&.metadata) || + recorded_model(session.metadata_hash) || + default_model_for_harness(session.harness_type.to_s) + # Uppercased for display, matching the Slack Console-link context line. + model&.upcase + end + + def recorded_model(metadata) + return unless metadata.is_a?(Hash) + + metadata["model"].presence + end + + def default_model_for_harness(harness_type) + env_name = HARNESS_DEFAULT_MODEL_ENVS[harness_type] + return unless env_name + + ENV[env_name].presence || self.class.baked_harness_default_model(harness_type) + end + + # Cached per (config dir, harness): the files are immutable within a deploy, + # and the dir key keeps tests with CENTAUR_HARNESS_CONFIG_DIR overrides + # isolated. + def self.baked_harness_default_model(harness_type) + relative = HARNESS_CONFIG_FILES[harness_type] + return unless relative + + dir = ENV["CENTAUR_HARNESS_CONFIG_DIR"].presence || + Rails.root.join("..", "..", "harness").to_s + cache = (@baked_harness_default_models ||= {}) + key = [ dir, harness_type ] + return cache[key] if cache.key?(key) + + cache[key] = parse_harness_default_model(File.join(dir, relative)) + end + + def self.parse_harness_default_model(path) + return unless File.file?(path) + + contents = File.read(path) + model = + if path.end_with?(".json") + parsed = JSON.parse(contents) + parsed["model"] if parsed.is_a?(Hash) + else + # Minimal TOML: the top-level `model = "..."` line in codex/config.toml. + contents[/^model\s*=\s*"([^"]+)"/, 1] + end + model.presence + rescue JSON::ParserError, SystemCallError + nil + end + + def thread_source_key(session) + metadata = session.metadata_hash + ( + metadata["repository"].presence || + metadata["repo"].presence || + metadata["platform"].presence || + metadata["source"].presence || + session.thread_key.to_s.split(":").first.presence || + "unknown" + ).to_s.downcase + end + + def source_label(value) + normalized = value.to_s.tr("_-", " ").squish + return "Slack" if normalized.casecmp("slack").zero? + return "Console" if normalized.casecmp("console").zero? + return "Unknown" if normalized.blank? + + normalized.split.map(&:capitalize).join(" ") + end + + def thread_user_label(session) + metadata = session.metadata_hash + metadata["user_name"].presence || + metadata["user_email"].presence || + metadata["actor_email"].presence || + metadata["slack_user_name"].presence || + metadata["actor_user_id"].presence || + metadata["user_id"].presence || + "unknown" + end + + def thread_message_text(message) + return "" unless message + + message.parts_array.filter_map do |part| + next unless part.is_a?(Hash) + + case part["type"] + when "text" then part["text"].to_s + when "image" then "[image]" + when "document" then "[document]" + end + end.join("\n").squish + end + + def thread_text_preview(message) + thread_message_text(message).truncate(120) + end + + def generated_thread_title(text) + title = text.to_s + .gsub(/<@[A-Z0-9]+(?:\|[^>]+)?>/, "") + .sub(/\A\s*@?centaur\b[:,]?\s*/i, "") + .sub(/\A\s*@?U[A-Z0-9]+\b[:,]?\s*/i, "") + .sub(/\A\s*@\S+\s+/, "") + .strip + title = title.sub(/\A[*_]{1,2}(.+?)[*_]{1,2}\s*/, "\\1 ").squish + clip_one_line(title, 80) + end + + def clip_one_line(value, max) + one_line = value.to_s.gsub(/\s+/, " ").strip + return one_line if one_line.length <= max + + "#{one_line.slice(0, [ max - 3, 0 ].max).rstrip}..." + end + + def transcript_item_for_event(event) + case event.event_type + when "session.execution_completed" + text = resolve_slack_mentions( + terminal_payload_text(event.payload_hash["result_text"] || event.payload_hash) + ) + role = "assistant" + label = assistant_author_label + when "session.execution_failed" + text = terminal_payload_text(event.payload_hash["error"] || event.payload_hash) + role = "system" + label = role + when "session.execution_cancelled" + text = "Execution cancelled." + role = "system" + label = role + end + + return nil if text.blank? + + { + role: role, + label: label, + align: :start, + text: text, + created_at: event.created_at, + source: :event + } + end + + def transcript_message_align(role, metadata) + return :end if slack_message_from_current_user?(metadata) + return :start if slack_message?(metadata) + + role == "user" ? :end : :start + end + + def transcript_message_label(role, metadata) + return slack_message_author_label(metadata) if slack_message?(metadata) + return assistant_author_label if role == "assistant" + + role + end + + def slack_message?(metadata) + metadata["platform"] == "slack" || metadata["source"] == "slackbotv2" + end + + def slack_message_from_current_user?(metadata) + slack_user_id = normalize_key(metadata["slack_user_id"] || metadata["user_id"]) + + slack_user_id.present? && current_slack_user_ids.include?(slack_user_id) + end + + def current_slack_user_ids + @current_slack_user_ids ||= slack_thread_owners_for_current_user + .filter_map { |owner| normalize_key(owner.user_id) } + .uniq + end + + def slack_message_author_label(metadata) + return assistant_author_label if slack_bot_user_id?(metadata["slack_user_id"]) + + current_user_metadata = + slack_message_from_current_user?(metadata) ? @selected_session&.metadata_hash : nil + + label_from_metadata(current_user_metadata) || + slack_resolved_user_label(metadata) || + label_from_metadata(metadata) || + "slack" + end + + def slack_resolved_user_label(metadata) + slack_user_id = normalize_key(metadata["slack_user_id"] || metadata["user_id"]) + return if slack_user_id.blank? + + slack_mention_labels_by_id[slack_user_id] + end + + def label_from_metadata(metadata) + return nil unless metadata + + [ + metadata["slack_display_name"], + metadata["slack_user_name"], + metadata["user_name"], + metadata["actor_user_id"], + metadata["user_id"], + metadata["slack_user_id"] + ].find(&:present?) + end + + def resolve_slack_mentions(text) + text.to_s.gsub(SLACK_MENTION_PATTERN) do + user_id = Regexp.last_match(1).presence || Regexp.last_match(3) + explicit_label = Regexp.last_match(2) + mention_label = slack_mention_labels_by_id[normalize_key(user_id)] || + format_slack_mention_label(explicit_label) || + "@#{user_id}" + + mention_label + end + end + + def slack_mention_labels_by_id + @slack_mention_labels_by_id ||= begin + user_ids = slack_user_ids_from_selected_thread + database_labels = slack_user_display_labels_from_database(user_ids) + session_metadata_labels = slack_user_display_labels_from_session_messages(user_ids) + metadata_labels = slack_user_display_labels_from_metadata + bot_labels = slack_bot_user_ids.index_with { assistant_author_label } + + metadata_labels.merge(session_metadata_labels).merge(database_labels).merge(bot_labels) + end + end + + def slack_user_ids_from_selected_thread + ids = [] + ids.concat(slack_user_ids_from_metadata(@selected_session&.metadata_hash)) + + Array(@selected_messages).each do |message| + ids.concat(slack_user_ids_from_metadata(message_metadata_hash(message))) + ids.concat(slack_mention_user_ids(thread_message_text(message))) + end + + Array(@selected_events).each do |event| + ids.concat(slack_mention_user_ids(terminal_payload_text(event.payload_hash))) + end + + ids.filter_map { |value| normalize_key(value) }.uniq + end + + def slack_user_display_labels_from_metadata + labels = {} + metadata_sources = [ @selected_session&.metadata_hash ] + metadata_sources.concat(Array(@selected_messages).map { |message| message_metadata_hash(message) }) + + metadata_sources.each do |metadata| + user_id = normalize_key(metadata&.[]("slack_user_id") || metadata&.[]("user_id")) + next if user_id.blank? + + label = slack_mention_label_from_metadata(metadata) + labels[user_id] = label if label.present? + end + + labels + end + + def slack_user_display_labels_from_database(user_ids) + user_ids = user_ids.filter_map { |value| normalize_key(value) }.uniq + return {} if user_ids.empty? + + connection = CentaurSessionRecord.connection + return {} unless connection.data_source_exists?("slack_sync_users") + + SlackSyncUser + .where("lower(user_id) IN (?)", user_ids) + .pluck(:user_id, :user_name, :display_name, :real_name) + .each_with_object({}) do |(user_id, user_name, display_name, real_name), labels| + user_id = normalize_key(user_id) + label = slack_mention_label_from_values(user_name, display_name, real_name) + labels[user_id] = label if user_id.present? && label.present? + end + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.debug("console_threads_slack_user_lookup_failed error=#{e.class}: #{e.message}") + {} + end + + def slack_user_display_labels_from_session_messages(user_ids) + user_ids = user_ids.filter_map { |value| normalize_key(value) }.uniq + return {} if user_ids.empty? + + rows = CentaurSessionMessage + .where(<<~SQL.squish, user_ids) + lower(coalesce( + nullif(metadata ->> 'slack_user_id', ''), + nullif(metadata ->> 'user_id', ''), + nullif(metadata ->> 'actor_user_id', '') + )) IN (?) + SQL + .order(created_at: :desc, message_id: :desc) + .pluck( + Arel.sql("metadata ->> 'slack_user_id'"), + Arel.sql("metadata ->> 'user_id'"), + Arel.sql("metadata ->> 'actor_user_id'"), + Arel.sql("metadata ->> 'slack_user_name'"), + Arel.sql("metadata ->> 'user_name'"), + Arel.sql("metadata ->> 'slack_display_name'"), + Arel.sql("metadata ->> 'display_name'") + ) + + rows.each_with_object({}) do |row, labels| + slack_user_id, user_id_value, actor_user_id, slack_user_name, user_name, slack_display_name, display_name = row + user_id = normalize_key(slack_user_id || user_id_value || actor_user_id) + next if user_id.blank? || labels.key?(user_id) + + label = slack_mention_label_from_values( + slack_user_name, + user_name, + slack_display_name, + display_name + ) + labels[user_id] = label if label.present? + end + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.debug("console_threads_slack_message_metadata_lookup_failed error=#{e.class}: #{e.message}") + {} + end + + def slack_mention_label_from_metadata(metadata) + return nil unless metadata + + slack_mention_label_from_values( + metadata["slack_user_name"], + metadata["user_name"], + metadata["slack_display_name"], + metadata["display_name"] + ) + end + + def slack_mention_label_from_values(*values) + values + .map { |value| value.to_s.strip } + .reject(&:blank?) + .reject { |value| slack_user_id?(value) } + .map { |value| format_slack_mention_label(value) } + .find(&:present?) + end + + def format_slack_mention_label(value) + value = value.to_s.strip + return nil if value.blank? + + "@#{value.delete_prefix("@")}" + end + + def slack_mention_user_ids(text) + text.to_s.scan(SLACK_MENTION_PATTERN).filter_map do |native_id, _label, plain_id| + native_id.presence || plain_id + end + end + + def slack_user_ids_from_metadata(metadata) + return [] unless metadata + + %w[slack_user_id user_id actor_user_id].filter_map { |key| metadata[key].presence } + end + + def slack_bot_user_id?(user_id) + slack_bot_user_ids.include?(normalize_key(user_id)) + end + + def slack_bot_user_ids + @slack_bot_user_ids ||= begin + ids = [ + ConsoleEnv["SLACK_BOT_USER_ID"], + ENV["SLACK_BOT_USER_ID"] + ] + + ids.concat(inferred_slack_bot_user_ids) + ids.filter_map { |value| normalize_key(value) }.uniq + end + end + + def inferred_slack_bot_user_ids + ids = [] + + Array(@selected_messages).each do |message| + metadata = message_metadata_hash(message) + if ActiveModel::Type::Boolean.new.cast(metadata["is_mention"]) + ids << slack_mention_user_ids(thread_message_text(message)).first + end + end + + terminal_texts = Array(@selected_events).filter_map do |event| + next unless event.event_type == "session.execution_completed" + + terminal_payload_text(event.payload_hash["result_text"] || event.payload_hash).presence + end + + if terminal_texts.any? + Array(@selected_messages).each do |message| + text = thread_message_text(message) + next unless terminal_texts.include?(text) + + ids.concat(slack_user_ids_from_metadata(message_metadata_hash(message))) + end + end + + ids.compact + end + + def slack_user_id?(value) + value.to_s.strip.match?(SLACK_USER_ID_PATTERN) + end + + def assistant_author_label + format_slack_mention_label( + ConsoleEnv["SLACKBOTV2_USER_NAME"].presence || + ENV["SLACKBOTV2_USER_NAME"].presence || + "ai" + ) + end + + def message_metadata_hash(message) + return message.metadata_hash if message.respond_to?(:metadata_hash) + + metadata = message.respond_to?(:metadata) ? message.metadata : nil + metadata.is_a?(Hash) ? metadata : {} + end + + def terminal_payload_text(value) + case value + when String + value.strip + when Array + value.lazy.map { |entry| terminal_payload_text(entry) }.find(&:present?).to_s + when Hash + %w[result result_text text final_text message delta content params].each do |key| + text = terminal_payload_text(value[key]) + return text if text.present? + end + "" + else + "" + end + end + + def thread_status_classes(status) + case status.to_s + when "active", "running", "queued" + "bg-centaur-500/10 text-centaur-300 ring-centaur-500/25" + when "failed", "error" + "bg-red-500/10 text-red-300 ring-red-500/25" + when "completed" + "bg-zinc-500/10 text-zinc-300 ring-zinc-500/25" + else + "bg-amber-500/10 text-amber-300 ring-amber-500/25" + end + end + + def human_thread_key(thread_key) + source, *parts = thread_key.to_s.split(":") + return thread_key if parts.empty? + + "#{source.titleize}: #{parts.last}" + end +end diff --git a/services/console/app/controllers/console/workflows_controller.rb b/services/console/app/controllers/console/workflows_controller.rb new file mode 100644 index 000000000..0f9819d3f --- /dev/null +++ b/services/console/app/controllers/console/workflows_controller.rb @@ -0,0 +1,151 @@ +class Console::WorkflowsController < ApplicationController + layout "console" + before_action :require_admin + + class_attribute :client_factory, default: -> { CentaurApiClient.new } + + PER_PAGE = 50 + + def index + @workflow_db_unavailable = false + @workflow_runs = [] + @queue_breakdown = {} + @page = page_param + @total_pages = 1 + + unless CentaurWorkflowRun.available? + @workflow_db_unavailable = true + return + end + + @total_workflows = CentaurWorkflowRun.workflow_count + @total_pages = [ (@total_workflows.to_f / PER_PAGE).ceil, 1 ].max + @page = [ @page, @total_pages ].min + + @workflow_runs = CentaurWorkflowRun.latest_per_workflow( + limit: PER_PAGE, + offset: (@page - 1) * PER_PAGE + ) + @queue_breakdown = CentaurWorkflowRun.latest_per_queue(@workflow_runs.map(&:workflow_key)) + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.warn("console_workflows_load_failed error=#{e.class}: #{e.message}") + @workflow_db_unavailable = true + @workflow_runs = [] + @queue_breakdown = {} + end + + def show + @workflow_db_unavailable = false + @workflow_name = params[:id].to_s + @workflow_runs = [] + @status_counts = {} + @queue_names = [] + @status = params[:status].presence + @queue = params[:queue].presence + @page = page_param + @total_pages = 1 + + unless CentaurWorkflowRun.available? + @workflow_db_unavailable = true + return + end + + @latest_run = CentaurWorkflowRun.for_workflow(@workflow_name, limit: 1).first + if @latest_run.blank? + response.status = :not_found + return + end + + @status_counts = CentaurWorkflowRun.status_counts(@workflow_name) + @total_runs = @status_counts.values.sum + @queue_names = CentaurWorkflowRun.queue_names(@workflow_name) + + @filtered_count = CentaurWorkflowRun.run_count(@workflow_name, status: @status, queue: @queue) + @total_pages = [ (@filtered_count.to_f / PER_PAGE).ceil, 1 ].max + @page = [ @page, @total_pages ].min + + @workflow_runs = CentaurWorkflowRun.for_workflow( + @workflow_name, + limit: PER_PAGE, + offset: (@page - 1) * PER_PAGE, + status: @status, + queue: @queue + ) + + load_workflow_api_details + rescue ActiveRecord::ActiveRecordError, PG::Error => e + Rails.logger.warn("console_workflow_load_failed workflow=#{@workflow_name} error=#{e.class}: #{e.message}") + @workflow_db_unavailable = true + @workflow_runs = [] + @latest_run = nil + end + + # Enqueue a run through the workflows API. Scheduled workflows are started + # with their registered schedule input so a forced run matches a normal tick. + def force_start + workflow_name = params[:id].to_s + schedule = workflow_schedules_for(workflow_name).first + result = api_client.create_workflow_run( + workflow_name: workflow_name, + input: schedule&.dig("input") + ) + notice = + if result["created"] == false + "A run with this idempotency key is already queued (#{result["run_id"]})." + else + "Run queued (#{result["run_id"]})." + end + redirect_to console_workflow_path(workflow_name), notice: notice + rescue StandardError => e + Rails.logger.warn("console_workflow_force_start_failed workflow=#{workflow_name} error=#{e.class}: #{e.message}") + redirect_to console_workflow_path(workflow_name), alert: "Could not start workflow: #{e.message}" + end + + private + + # Best-effort enrichment from the workflows API: the registered schedule + # (cron/interval, source path for the GitHub link) and the latest run's + # input/result/failure for debugging. The page renders without any of it + # when the API is unreachable. + def load_workflow_api_details + @workflow_schedules = workflow_schedules_for(@workflow_name) + @latest_run_detail = fetch_run_detail(@latest_run&.run_id) + + return if @latest_run_detail.blank? && @workflow_schedules.blank? + return if @latest_run&.display_status == "failed" + return unless @status_counts["failed"].to_i.positive? + + failed_run = CentaurWorkflowRun.for_workflow(@workflow_name, limit: 1, status: "failed").first + @latest_failure_detail = fetch_run_detail(failed_run&.run_id) + end + + def workflow_schedules_for(workflow_name) + response = api_client.list_workflow_schedules + Array(response["schedules"]).select do |schedule| + schedule.is_a?(Hash) && schedule["workflow_name"] == workflow_name + end + rescue StandardError => e + Rails.logger.warn("console_workflow_schedules_failed error=#{e.class}: #{e.message}") + [] + end + + def fetch_run_detail(run_id) + return nil if run_id.blank? + + response = api_client.get_workflow_run(run_id) + detail = response["run"] + detail.is_a?(Hash) ? detail : nil + rescue StandardError => e + Rails.logger.warn("console_workflow_run_detail_failed run=#{run_id} error=#{e.class}: #{e.message}") + nil + end + + def api_client + @api_client ||= self.class.client_factory.call + end + + def page_param + page = Integer(params[:page].to_s, 10, exception: false) || 1 + page < 1 ? 1 : page + end +end diff --git a/services/console/app/controllers/console_controller.rb b/services/console/app/controllers/console_controller.rb index 5a6fca196..8e62ba251 100644 --- a/services/console/app/controllers/console_controller.rb +++ b/services/console/app/controllers/console_controller.rb @@ -1,11 +1,14 @@ # Operator console: a lightweight, server-rendered HTML view over principals, # their effective grants, and secrets. Read-only; gated behind a console session -# via ApplicationController#require_login. Distinct from the JSON API. +# (ApplicationController#require_login) and restricted to admins (require_admin), +# like every Control/Data Sync page. Distinct from the JSON API. class ConsoleController < ApplicationController include SecretKinds layout "console" + before_action :require_admin + # Friendly labels for the source backend (and the gcp_auth credentials_provider # type). The secrets table shows only this -- the full reference lives on the # secret detail page. @@ -22,6 +25,12 @@ def principals def principal @principal = Principal.find_by_oid!(params[:id]) + @slack_channel_catalog = SlackChannelCatalog.fetch + @slack_channel_permissions = @principal.slack_channel_permissions.ordered + @slack_channel_options = @slack_channel_catalog.channels.map do |channel| + label = "#{channel.private ? "Private" : "Public"} ##{channel.name} (#{channel.id})" + [ label, channel.id ] + end @roles = @principal.roles.order(:id) @granted = { "static" => @principal.granted_static_secrets, diff --git a/services/console/app/controllers/launch_controller.rb b/services/console/app/controllers/launch_controller.rb new file mode 100644 index 000000000..977320a8c --- /dev/null +++ b/services/console/app/controllers/launch_controller.rb @@ -0,0 +1,30 @@ +# Entry point for the web+centaur:// protocol handler the PWA manifest +# registers. When the OS opens such a link, the installed app navigates here +# with the full custom-scheme URL in ?target=; we map it onto an in-app path +# and redirect. web+centaur://console/threads lands on /console/threads. +# +# Only strictly path-shaped targets survive the mapping (no dots, queries, or +# protocol-relative tricks), so a crafted link can never bounce the operator +# off-origin. Anything that doesn't parse falls back to the console root. +class LaunchController < ApplicationController + SCHEME_PREFIX = "web+centaur://".freeze + SAFE_PATH = %r{\A[A-Za-z0-9_/-]+\z} + + def show + redirect_to launch_path_for(params[:target].to_s) + end + + private + + def launch_path_for(target) + rest = target.delete_prefix(SCHEME_PREFIX) + return root_path if rest == target || rest.blank? + + # Collapse and trim slashes before re-rooting the path: "/#{path}" must + # never come out protocol-relative ("//host") or dot-traversable. + path = rest.squeeze("/").delete_prefix("/").delete_suffix("/") + return root_path unless path.match?(SAFE_PATH) + + "/#{path}" + end +end diff --git a/services/console/app/controllers/mcp/oauth_controller.rb b/services/console/app/controllers/mcp/oauth_controller.rb new file mode 100644 index 000000000..16adbe4bd --- /dev/null +++ b/services/console/app/controllers/mcp/oauth_controller.rb @@ -0,0 +1,553 @@ +require "base64" +require "digest" +require "uri" + +module Mcp + class OauthController < ApplicationController + layout "auth" + + skip_before_action :require_login, only: %i[metadata register authorize token] + skip_before_action :require_active_account, only: %i[metadata register authorize token] + # OAuth dynamic registration and token exchange are machine-to-machine + # endpoints with no browser session authority. Authorization approval keeps + # Rails CSRF protection; these two endpoints authenticate the protocol + # request itself (PKCE for code exchange). + skip_forgery_protection only: %i[register token] + + ACCESS_TOKEN_TTL_SECONDS = 1.hour.to_i + + # The shared role seeded onto new console-user principals. Admins attach + # tool secrets to this role to define what every MCP user gets by default. + USER_MCP_ROLE_FOREIGN_ID = "user-mcp" + + # GET /.well-known/oauth-authorization-server + def metadata + render json: { + issuer: public_base_url, + authorization_endpoint: URI.join(public_base_url, "/mcp/oauth/authorize").to_s, + token_endpoint: URI.join(public_base_url, "/mcp/oauth/token").to_s, + registration_endpoint: URI.join(public_base_url, "/mcp/oauth/register").to_s, + response_types_supported: [ "code" ], + grant_types_supported: McpOauthClient::DEFAULT_GRANT_TYPES, + code_challenge_methods_supported: [ "S256" ], + token_endpoint_auth_methods_supported: [ "none" ], + scopes_supported: McpOauthClient::DEFAULT_SCOPES, + resource_parameter_supported: true + } + end + + # POST /mcp/oauth/register + def register + requested_redirect_uris = + Array(params[:redirect_uris]).map(&:to_s).map(&:strip).reject(&:blank?) + client = McpOauthClient.create!( + name: params[:client_name].presence || "MCP client", + redirect_uris: requested_redirect_uris, + grant_types: normalize_list_param( + params[:grant_types], + McpOauthClient::DEFAULT_GRANT_TYPES + ), + response_types: normalize_list_param( + params[:response_types], + McpOauthClient::DEFAULT_RESPONSE_TYPES + ), + scopes: normalize_scope_param(params[:scope], McpOauthClient::DEFAULT_SCOPES), + metadata: registration_metadata + ) + + render json: { + client_id: client.public_client_id, + client_name: client.name, + redirect_uris: client.redirect_uris, + grant_types: client.grant_types, + response_types: client.response_types, + scope: client.scopes.join(" "), + token_endpoint_auth_method: "none" + }, status: :created + rescue ActiveRecord::RecordInvalid => e + oauth_error( + :invalid_client_metadata, + e.record.errors.full_messages.to_sentence, + status: :bad_request + ) + end + + # GET /mcp/oauth/authorize + def authorize + return redirect_to_login unless current_user + return redirect_to pending_path if current_user.pending? + return redirect_to login_path, alert: "Your account is disabled." if current_user.disabled? + + authorization = validated_authorization_request + return unless authorization + + assign_authorization_view(authorization) + render :authorize + end + + # POST /mcp/oauth/authorize + def approve + authorization = validated_authorization_request + return unless authorization + + unless params[:decision] == "approve" + return authorization_error( + authorization[:client], + :access_denied, + "The user denied the authorization request." + ) + end + + issue_authorization_code(authorization) + rescue ActiveRecord::RecordInvalid => e + authorization_error(nil, :server_error, e.record.errors.full_messages.to_sentence) + end + + # POST /mcp/oauth/token + def token + case params[:grant_type] + when "authorization_code" + exchange_authorization_code + when "refresh_token" + exchange_refresh_token + else + oauth_error(:unsupported_grant_type, "Unsupported grant_type.", status: :bad_request) + end + end + + private + + def validated_authorization_request + client = resolve_client(params[:client_id]) + return authorization_request_error(nil, :invalid_request, "Unknown client.") unless client + + unless params[:response_type] == "code" + return authorization_request_error( + client, + :unsupported_response_type, + "Only response_type=code is supported." + ) + end + unless client.redirect_uri_allowed?(params[:redirect_uri]) + return authorization_request_error( + client, + :invalid_request, + "redirect_uri is not registered for this client." + ) + end + unless params[:code_challenge_method] == "S256" + return authorization_request_error( + client, + :invalid_request, + "code_challenge_method must be S256." + ) + end + if params[:code_challenge].blank? + return authorization_request_error(client, :invalid_request, "code_challenge is required.") + end + + scopes = normalize_scope_param(params[:scope], McpOauthClient::DEFAULT_SCOPES) + unsupported = scopes - McpOauthClient::DEFAULT_SCOPES + if unsupported.any? + return authorization_request_error( + client, + :invalid_scope, + "Unsupported scope: #{unsupported.join(' ')}." + ) + end + + resource = resolve_requested_resource + return authorization_request_error(client, :invalid_target, "resource is required.") if resource.blank? + + { client: client, scopes: scopes, resource: resource } + end + + def authorization_request_error(client, error, description) + authorization_error(client, error, description) + nil + end + + def assign_authorization_view(authorization) + @client = authorization[:client] + @scopes = authorization[:scopes] + @resource = authorization[:resource] + @redirect_uri = params[:redirect_uri].to_s + @redirect_host = redirect_uri_host(@redirect_uri) + @authorization_params = authorization_form_params(authorization) + end + + def issue_authorization_code(authorization) + client = authorization[:client] + principal = principal_for_current_user + code = McpOauthAuthorizationCode.create!( + mcp_oauth_client: client, + user: current_user, + principal: principal, + redirect_uri: params[:redirect_uri].to_s, + code_challenge: params[:code_challenge].to_s, + resource: authorization[:resource], + scopes: authorization[:scopes] + ) + client.touch(:last_used_at) + + uri = URI.parse(params[:redirect_uri]) + query = Rack::Utils.parse_nested_query(uri.query) + query["code"] = code.plaintext_code + query["state"] = params[:state] if params[:state].present? + uri.query = query.to_query + redirect_to uri.to_s, allow_other_host: true + end + + def authorization_form_params(authorization) + { + response_type: params[:response_type].to_s, + client_id: authorization[:client].public_client_id, + redirect_uri: params[:redirect_uri].to_s, + scope: authorization[:scopes].join(" "), + state: params[:state].to_s, + resource: authorization[:resource], + code_challenge: params[:code_challenge].to_s, + code_challenge_method: params[:code_challenge_method].to_s + } + end + + def redirect_uri_host(value) + URI.parse(value).host + rescue URI::InvalidURIError + value + end + + def exchange_authorization_code + client = resolve_client(params[:client_id]) + return oauth_error(:invalid_client, "Unknown client.", status: :unauthorized) unless client + code = McpOauthAuthorizationCode.find_usable(params[:code]) + unless code + return oauth_error( + :invalid_grant, + "Authorization code is invalid or expired.", + status: :bad_request + ) + end + unless code.mcp_oauth_client == client + return oauth_error( + :invalid_grant, + "Authorization code was not issued to this client.", + status: :bad_request + ) + end + unless code.redirect_uri == params[:redirect_uri].to_s + return oauth_error( + :invalid_grant, + "redirect_uri does not match the authorization request.", + status: :bad_request + ) + end + unless pkce_valid?(code.code_challenge, params[:code_verifier].to_s) + return oauth_error(:invalid_grant, "PKCE verification failed.", status: :bad_request) + end + + refresh = nil + invalid_grant = false + inactive_user = false + McpOauthAuthorizationCode.transaction do + code.lock! + if code.consumed_at.present? || code.expires_at <= Time.current + invalid_grant = true + elsif !code.user.active? + inactive_user = true + code.consume! + code.user.revoke_mcp_oauth_refresh_tokens! + else + code.consume! + refresh = McpOauthRefreshToken.create!( + mcp_oauth_client: client, + user: code.user, + principal: code.principal, + resource: code.resource, + scopes: code.scopes + ) + end + end + if invalid_grant + return oauth_error( + :invalid_grant, + "Authorization code is invalid or expired.", + status: :bad_request + ) + end + if inactive_user + return oauth_error( + :invalid_grant, + "User account is not active.", + status: :bad_request + ) + end + + client.touch(:last_used_at) + render_token_response( + client: client, + user: code.user, + principal: code.principal, + resource: code.resource, + scopes: code.scopes, + refresh_token: refresh.plaintext_token + ) + end + + def exchange_refresh_token + client = resolve_client(params[:client_id]) + return oauth_error(:invalid_client, "Unknown client.", status: :unauthorized) unless client + refresh = McpOauthRefreshToken.find_usable(params[:refresh_token]) + unless refresh + return oauth_error( + :invalid_grant, + "Refresh token is invalid or expired.", + status: :bad_request + ) + end + unless refresh.mcp_oauth_client == client + return oauth_error( + :invalid_grant, + "Refresh token was not issued to this client.", + status: :bad_request + ) + end + + rotated = nil + invalid_grant = false + inactive_user = false + McpOauthRefreshToken.transaction do + refresh.lock! + if refresh.revoked_at.present? || refresh.expires_at <= Time.current + invalid_grant = true + elsif !refresh.user.active? + inactive_user = true + refresh.user.revoke_mcp_oauth_refresh_tokens! + else + refresh.update!(revoked_at: Time.current, last_used_at: Time.current) + rotated = McpOauthRefreshToken.create!( + mcp_oauth_client: client, + user: refresh.user, + principal: refresh.principal, + resource: refresh.resource, + scopes: refresh.scopes + ) + end + end + if invalid_grant + return oauth_error( + :invalid_grant, + "Refresh token is invalid or expired.", + status: :bad_request + ) + end + if inactive_user + return oauth_error( + :invalid_grant, + "User account is not active.", + status: :bad_request + ) + end + + client.touch(:last_used_at) + render_token_response( + client: client, + user: refresh.user, + principal: refresh.principal, + resource: refresh.resource, + scopes: refresh.scopes, + refresh_token: rotated.plaintext_token + ) + end + + def render_token_response(client:, user:, principal:, resource:, scopes:, refresh_token:) + now = Time.current.to_i + ttl = access_token_ttl_seconds + payload = { + iss: public_base_url, + sub: user.oid, + aud: resource, + exp: now + ttl, + nbf: now - 5, + iat: now, + jti: "mcpjwt_#{SecureRandom.hex(16)}", + scope: scopes.join(" "), + client_id: client.public_client_id, + principal_id: principal.oid, + principal_foreign_id: principal.foreign_id, + email: user.email, + name: user.name.presence || user.email + } + render json: { + access_token: Mcp::Jwt.encode(payload), + token_type: "Bearer", + expires_in: ttl, + scope: scopes.join(" "), + refresh_token: refresh_token + } + rescue KeyError => e + oauth_error(:server_error, e.message, status: :service_unavailable) + end + + def redirect_to_login + session[:return_to] = request.fullpath if request.request_method == "GET" + redirect_to login_path + end + + def authorization_error(client, error, description) + if client&.redirect_uri_allowed?(params[:redirect_uri]) + uri = URI.parse(params[:redirect_uri]) + query = Rack::Utils.parse_nested_query(uri.query) + query["error"] = error.to_s + query["error_description"] = description + query["state"] = params[:state] if params[:state].present? + uri.query = query.to_query + redirect_to uri.to_s, allow_other_host: true + else + render plain: description, status: :bad_request + end + end + + def oauth_error(error, description, status:) + render json: { error: error.to_s, error_description: description }, status: status + end + + def resolve_client(client_id) + McpOauthClient.find_by_oid(client_id) + end + + def resolve_requested_resource + # Fail closed: without a configured canonical resource URL we would + # otherwise mint tokens bound to any caller-supplied audience. + configured = normalize_mcp_resource_url(configured_mcp_resource_url) + return nil if configured.blank? + requested = params[:resource].presence + return nil if requested.present? && normalize_mcp_resource_url(requested) != configured + configured + end + + def configured_mcp_resource_url + ENV["CENTAUR_MCP_PUBLIC_URL"].presence || ConsoleEnv["MCP_PUBLIC_URL"].presence + end + + def normalize_mcp_resource_url(value) + uri = URI.parse(value.to_s.strip) + return nil unless %w[http https].include?(uri.scheme) && uri.host.present? + uri.fragment = nil + path = uri.path.to_s.sub(%r{/+\z}, "") + uri.path = path.end_with?("/mcp") ? path : "#{path}/mcp" + uri.to_s.sub(/\?\z/, "") + rescue URI::InvalidURIError + nil + end + + # Principal creation and role seeding share a transaction: seeding is + # create-only, so a principal that committed without its role would stay + # unseeded forever. The RecordNotUnique retry sits outside the transaction + # because a unique violation aborts the enclosing Postgres transaction; + # every violation source (principal, role, or assignment race) converges + # to the find path on the next pass. + def principal_for_current_user + Principal.transaction do + foreign_id = principal_foreign_id(current_user.email) + principal = Principal.find_or_initialize_by( + namespace: mcp_principal_namespace, foreign_id: foreign_id + ) + newly_created = principal.new_record? + principal.created_by ||= current_user + principal.name = current_user.name.presence || current_user.email + principal.labels = principal.labels.merge( + "managed-by" => "centaur", + "kind" => "console_user", + "console-user-id" => current_user.oid, + "email" => current_user.email + ).merge(slack_identity_labels_for(current_user)) + principal.save! + assign_user_mcp_role(principal) if newly_created + principal + end + rescue ActiveRecord::RecordNotUnique + retry + end + + # New console-user principals start with the shared user-mcp role. Seeded + # only at creation so an operator removing the role from a principal + # sticks, mirroring SessionRegistrar's seeding of the infra role for + # session principals. + def assign_user_mcp_role(principal) + role = Role + .create_with( + name: "User MCP", + labels: { "managed-by" => "centaur" }, + created_by: current_user + ) + .find_or_create_by!( + namespace: principal.namespace, + foreign_id: USER_MCP_ROLE_FOREIGN_ID + ) + principal.principal_roles.find_or_create_by!(role: role) + end + + # Slack's OIDC id_token is the authenticated source of the user's native + # Slack identity. Refuse an ambiguous account rather than guessing which + # workspace should determine company-context RLS. + def slack_identity_labels_for(user) + identities = user.user_identities.where(provider: UserIdentity::SLACK_PROVIDER).order(:id) + identities = identities.filter_map do |identity| + next if identity.subject.blank? || identity.team_id.blank? + + [ identity.subject, identity.team_id ] + end.uniq + return {} unless identities.one? + + slack_user_id, slack_team_id = identities.first + { "slack_user_id" => slack_user_id, "slack_team_id" => slack_team_id } + end + + def principal_foreign_id(email) + normalized = email.to_s.downcase.strip + safe = normalized.gsub(/[^A-Za-z0-9\-._~]/, "-").gsub(/-+/, "-").first(48) + digest = Digest::SHA256.hexdigest(normalized).first(12) + "console-user-#{safe}-#{digest}" + end + + def mcp_principal_namespace + ENV["CENTAUR_MCP_PRINCIPAL_NAMESPACE"].presence || + ConsoleEnv["MCP_PRINCIPAL_NAMESPACE"].presence || + "default" + end + + def access_token_ttl_seconds + raw = + ENV["CENTAUR_MCP_ACCESS_TOKEN_TTL_SECONDS"].presence || + ConsoleEnv["MCP_ACCESS_TOKEN_TTL_SECONDS"].presence + seconds = raw.to_i + seconds.positive? ? seconds : ACCESS_TOKEN_TTL_SECONDS + end + + def registration_metadata + params + .to_unsafe_h + .slice("client_uri", "logo_uri", "contacts", "software_id", "software_version") + .compact + end + + def normalize_list_param(value, default) + list = value.presence || default + Array(list).map(&:to_s).map(&:strip).reject(&:blank?).presence || default + end + + def normalize_scope_param(value, default) + return default if value.blank? + value.to_s.split(/[,\s]+/).map(&:strip).reject(&:blank?) + end + + def pkce_valid?(challenge, verifier) + return false if verifier.blank? + actual = Base64.urlsafe_encode64(Digest::SHA256.digest(verifier), padding: false) + ActiveSupport::SecurityUtils.secure_compare(actual, challenge.to_s) + rescue ArgumentError + false + end + end +end diff --git a/services/console/app/controllers/oauth/flows_controller.rb b/services/console/app/controllers/oauth/flows_controller.rb index 64773b969..fc45497a8 100644 --- a/services/console/app/controllers/oauth/flows_controller.rb +++ b/services/console/app/controllers/oauth/flows_controller.rb @@ -6,8 +6,8 @@ module Oauth # The OAuth consent flow, keyed by an app's well-known slug: # /oauth/:slug/start sends a team member to the IdP's consent screen, and # /oauth/:slug/callback turns the returned authorization code into a managed - # BrokerCredential linked to the OauthApp, then renders an centaur-console result - # page. + # BrokerCredential linked to the OauthApp, then sends the user back to the + # console Integrations page (or renders a result page on failure). # # Deliberately unauthenticated -- a team member connects an integration by # clicking a well-known link; there is no external app to integrate with, so @@ -91,7 +91,10 @@ def callback @credential = upsert_credential(state, result, identity) enqueue_identity_enrichment(@credential) - render_result(:success, identity: identity) + # Back to the Integrations page the user started from; failures below + # still render the standalone result page, which offers a retry link. + connected_as = " as #{identity[:email]}" if identity[:email].present? + redirect_to console_integrations_path, notice: "#{@app.slug} connected#{connected_as}." rescue Broker::ExchangeError => e render_result(:error, message: "Connecting the integration failed (#{e.reason}).") rescue ActiveRecord::RecordInvalid => e @@ -160,6 +163,12 @@ def exchange_code(code, code_verifier) def upsert_credential(state, result, identity) BrokerCredential.transaction do credential = BrokerCredential.find_or_initialize_by(oauth_app: @app, provider_subject: identity[:subject]) + # When the consenting browser carries a signed-in console session, + # remember which user connected this account. The Integrations page + # matches on it, so the card flips to "Connected" even when the + # provider account's email differs from the console login email. + # Never overwritten: the first linked user keeps the credential. + credential.created_by ||= current_user if credential.new_record? credential.namespace = @app.credential_namespace credential.foreign_id = "#{@app.provider}-#{@app.slug}-#{identity[:subject].downcase}" @@ -174,6 +183,7 @@ def upsert_credential(state, result, identity) provider_email: identity[:email], # Store exactly what the IdP granted, so the refresh POST re-requests it. scopes: granted_scopes(result, state), + labels: credential_labels(credential, identity), refresh_token: result.refresh_token, access_token: result.access_token, expires_at: now + expires_in, @@ -192,16 +202,28 @@ def granted_scopes(result, state) @provider.parse_granted_scopes(result.scope) end + def credential_labels(credential, identity) + labels = credential.labels || {} + return labels unless @app.provider == Oauth::Providers::Slack::KEY + return labels if identity[:team_id].blank? + + labels.merge("slack_team_id" => identity[:team_id]) + end + def identity_display_name(identity) identity[:name].presence || identity[:email].presence || identity[:subject] end def enqueue_identity_enrichment(credential) case @app.provider + when Oauth::Providers::Attio::KEY + Oauth::EnrichAttioCredentialIdentityJob.perform_later(credential.id) when Oauth::Providers::Slack::KEY Oauth::EnrichCredentialIdentityJob.perform_later(credential.id) when Oauth::Providers::Github::KEY Oauth::EnrichGithubCredentialIdentityJob.perform_later(credential.id) + when Oauth::Providers::Linear::KEY + Oauth::EnrichLinearCredentialIdentityJob.perform_later(credential.id) end end @@ -214,8 +236,8 @@ def enqueue_identity_enrichment(credential) # # Created once per credential (keyed on the broker_credential association, which # a unique index enforces) and left untouched on re-consent, so any operator - # edits -- a different header, extra rules -- survive. Has no created_by: the - # unauthenticated flow has no current user, like the credential it wraps. Left + # edits -- a different header, extra rules -- survive. Has no created_by: + # unlike the credential, no console feature keys off the secret's owner. Left # without a foreign_id: it is found by association, and copying the credential's # would risk colliding with an operator-created secret. def ensure_wrapping_secret(credential) @@ -242,14 +264,13 @@ def read_and_clear_flow_cookie nil end - # Renders the team-facing result page. +kind+ is :success, :denied, or - # :error; the matching HTTP status defaults sensibly but callers override it - # for the 4xx pre-consent rejections. - def render_result(kind, status: nil, message: nil, identity: nil, **) + # Renders the team-facing failure page. +kind+ is :denied or :error; the + # status defaults to 422 but callers override it for the 4xx pre-consent + # rejections. Success does not come through here -- the happy path + # redirects back to the console Integrations page. + def render_result(kind, status: :unprocessable_entity, message: nil) @kind = kind @message = message - @identity = identity - status ||= (kind == :success ? :ok : :unprocessable_entity) render :result, status: status end end diff --git a/services/console/app/controllers/sessions_controller.rb b/services/console/app/controllers/sessions_controller.rb index 1db9fb8b6..3e56e73ab 100644 --- a/services/console/app/controllers/sessions_controller.rb +++ b/services/console/app/controllers/sessions_controller.rb @@ -14,13 +14,13 @@ class SessionsController < ApplicationController skip_before_action :require_active_account def new - redirect_to console_principals_path if current_user&.active? + redirect_to safe_console_return_path if current_user&.active? end # Holding page for a signed-in but not-yet-approved user. Active users have no # reason to be here, so send them to the console. def pending - redirect_to console_principals_path if current_user&.active? + redirect_to default_console_landing_path if current_user&.active? end def create diff --git a/services/console/app/helpers/application_helper.rb b/services/console/app/helpers/application_helper.rb index 467e01ae7..8aa7bddbc 100644 --- a/services/console/app/helpers/application_helper.rb +++ b/services/console/app/helpers/application_helper.rb @@ -1,4 +1,12 @@ +require "cgi" + module ApplicationHelper + MARKDOWN_ALLOWED_TAGS = %w[ + a blockquote br code del div em h1 h2 h3 h4 li ol p pre strong + table tbody td th thead tr ul + ].freeze + MARKDOWN_ALLOWED_ATTRIBUTES = %w[class href rel target].freeze + # Truncates a string in the middle with an ellipsis (e.g. "salesforce…rest-api"), # keeping the head and tail visible -- useful for opaque ids where both ends # carry meaning. Returns the value unchanged when it already fits within +max+. @@ -23,6 +31,315 @@ def credential_status_classes(status) end end + def workflow_status_classes(status) + case status.to_s + when "completed" then "border-centaur-500/30 bg-centaur-500/10 text-centaur-300" + when "running" then "border-sky-500/40 bg-sky-500/10 text-sky-300" + when "failed" then "border-red-500/40 bg-red-500/10 text-red-300" + when "cancelled" then "border-ink-600 bg-ink-800/80 text-zinc-400" + when "pending", "sleeping" then "border-amber-500/40 bg-amber-500/10 text-amber-300" + else "border-ink-600 bg-ink-800/80 text-zinc-400" + end + end + + # Engine names rendered the way the Chats page renders harness types + # (Console::ThreadsController#thread_harness_label): known harnesses get + # their product names, anything else is capitalized word-wise. + def workflow_engine_label(harness_type) + case harness_type.to_s + when "codex" then "Codex" + when "claudecode" then "Claude Code" + when "amp" then "Amp" + when "" then nil + else harness_type.to_s.tr("_-", " ").squish.split.map(&:capitalize).join(" ") + end + end + + # GitHub URL for a workflow source path reported by the workflow host. + # Paths are repo-relative; an overlay-repo prefix ("centaur-tempo/...") maps + # to the tempo overlay repo, everything else to the main centaur repo. + def workflow_source_url(source_path) + path = source_path.to_s + return nil if path.blank? + + if path.start_with?("centaur-tempo/") + "https://github.com/tempoxyz/centaur-tempo/blob/main/#{path.delete_prefix("centaur-tempo/")}" + else + "https://github.com/paradigmxyz/centaur/blob/main/#{path}" + end + end + + # Human label for a workflow schedule from the workflows API, e.g. + # "cron */5 * * * *" or "every 5m". The kind is the serde-tagged enum + # {"type":"cron","cron":...} | {"type":"interval","interval_seconds":...}. + def workflow_schedule_label(schedule) + kind = schedule.is_a?(Hash) ? schedule["kind"] : nil + return nil unless kind.is_a?(Hash) + + case kind["type"] + when "cron" + "cron #{kind["cron"]}" + when "interval" + seconds = kind["interval_seconds"].to_i + "every #{seconds % 60 == 0 && seconds >= 60 ? "#{seconds / 60}m" : "#{seconds}s"}" + end + end + + # Pretty-printed JSON for workflow run payloads (input/result/failure). + # Falls back to to_s for values the generator refuses. + def workflow_debug_json(value) + JSON.pretty_generate(value) + rescue JSON::GeneratorError + value.to_s + end + + def workflow_duration_label(run) + started_at = run.started_or_created_at + finished_at = run.terminal_at + return "running" if started_at.present? && finished_at.blank? && run.display_status == "running" + return "—" if started_at.blank? || finished_at.blank? + + distance_of_time_in_words(started_at, finished_at) + end + + def console_icon(name, classes: "size-4") + case name + when "arrow-up" + outline_icon(classes, "M4.5 10.5 12 3m0 0 7.5 7.5M12 3v18") + when "database" + outline_icon( + classes, + "M4.5 6.75c0 1.243 3.358 2.25 7.5 2.25s7.5-1.007 7.5-2.25S16.142 4.5 12 4.5 4.5 5.507 4.5 6.75Zm0 0v10.5c0 1.243 3.358 2.25 7.5 2.25s7.5-1.007 7.5-2.25V6.75M4.5 12c0 1.243 3.358 2.25 7.5 2.25s7.5-1.007 7.5-2.25" + ) + when "computer" + outline_icon( + classes, + "M9 17.25v1.007a3 3 0 0 1-.879 2.122L7.5 21h9l-.621-.621A3 3 0 0 1 15 18.257V17.25m6-12V15A2.25 2.25 0 0 1 18.75 17.25H5.25A2.25 2.25 0 0 1 3 15V5.25A2.25 2.25 0 0 1 5.25 3h13.5A2.25 2.25 0 0 1 21 5.25Z" + ) + when "id-badge" + outline_icon( + classes, + "M6.75 3.75h10.5A2.25 2.25 0 0 1 19.5 6v12a2.25 2.25 0 0 1-2.25 2.25H6.75A2.25 2.25 0 0 1 4.5 18V6a2.25 2.25 0 0 1 2.25-2.25ZM9 8.25h6M9 15.75h6M9 12h6" + ) + when "ellipsis-horizontal" + tag.svg( + safe_join([ + tag.circle(cx: "6.75", cy: "12", r: "1"), + tag.circle(cx: "12", cy: "12", r: "1"), + tag.circle(cx: "17.25", cy: "12", r: "1") + ]), + xmlns: "http://www.w3.org/2000/svg", + viewBox: "0 0 24 24", + fill: "currentColor", + class: classes, + aria: { hidden: true }, + focusable: "false" + ) + when "key" + outline_icon( + classes, + "M15.75 7.5a4.5 4.5 0 1 1-1.118 2.966L21 16.834V19.5h-2.666l-1.5-1.5h-2.121l-1.5-1.5v-2.121l-1.179-1.179A4.5 4.5 0 0 1 15.75 7.5Z" + ) + when "link" + outline_icon( + classes, + "M13.5 6.75h2.25a4.5 4.5 0 0 1 0 9H13.5m-3-9H8.25a4.5 4.5 0 0 0 0 9h2.25M8.25 12h7.5" + ) + when "log-out" + outline_icon( + classes, + "M15.75 9V5.25A2.25 2.25 0 0 0 13.5 3h-6A2.25 2.25 0 0 0 5.25 5.25v13.5A2.25 2.25 0 0 0 7.5 21h6a2.25 2.25 0 0 0 2.25-2.25V15M12 12h9m0 0-3-3m3 3-3 3" + ) + when "moon" + outline_icon( + classes, + "M21.752 15.002A9.718 9.718 0 0 1 18 15.75c-5.385 0-9.75-4.365-9.75-9.75 0-1.33.266-2.598.748-3.752A9.753 9.753 0 0 0 3 11.25C3 16.635 7.365 21 12.75 21a9.753 9.753 0 0 0 9.002-5.998Z" + ) + when "magnifying-glass" + outline_icon( + classes, + "m21 21-5.197-5.197m0 0A7.5 7.5 0 1 0 5.196 5.196a7.5 7.5 0 0 0 10.607 10.607Z" + ) + when "message-square" + outline_icon( + classes, + "M6.75 5.25h10.5A2.25 2.25 0 0 1 19.5 7.5v6A2.25 2.25 0 0 1 17.25 15.75H10.5L6 19.5v-3.75A2.25 2.25 0 0 1 3.75 13.5v-6A2.25 2.25 0 0 1 6.75 5.25Z" + ) + when "panel-left" + outline_icon( + classes, + "M4.5 5.25A1.5 1.5 0 0 1 6 3.75h12a1.5 1.5 0 0 1 1.5 1.5v13.5a1.5 1.5 0 0 1-1.5 1.5H6a1.5 1.5 0 0 1-1.5-1.5V5.25ZM9 3.75v16.5" + ) + when "panel-right" + outline_icon( + classes, + "M4.5 5.25A1.5 1.5 0 0 1 6 3.75h12a1.5 1.5 0 0 1 1.5 1.5v13.5a1.5 1.5 0 0 1-1.5 1.5H6a1.5 1.5 0 0 1-1.5-1.5V5.25ZM15 3.75v16.5" + ) + when "plus" + outline_icon(classes, "M12 4.5v15m7.5-7.5h-15") + when "chevron-right" + outline_icon(classes, "m8.25 4.5 7.5 7.5-7.5 7.5") + when "check" + outline_icon(classes, "m4.5 12.75 6 6 9-13.5") + when "x-mark" + outline_icon(classes, "M6 18 18 6M6 6l12 12") + when "shield-check" + outline_icon( + classes, + "M12 3.75 19.5 6v5.25c0 4.207-2.765 8.04-7.5 9-4.735-.96-7.5-4.793-7.5-9V6L12 3.75Zm3.75 6-4.5 4.5-2.25-2.25" + ) + when "slack" + tag.svg( + tag.path( + d: "M5.042 15.165a2.528 2.528 0 0 1-2.52 2.523A2.528 2.528 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52v2.52ZM6.313 15.165a2.527 2.527 0 0 1 2.521-2.52 2.527 2.527 0 0 1 2.521 2.52v6.313A2.528 2.528 0 0 1 8.834 24a2.528 2.528 0 0 1-2.52-2.522v-6.313ZM8.834 5.042a2.528 2.528 0 0 1-2.52-2.52A2.528 2.528 0 0 1 8.834 0a2.528 2.528 0 0 1 2.521 2.522v2.52H8.834ZM8.834 6.313a2.528 2.528 0 0 1 2.521 2.521 2.528 2.528 0 0 1-2.521 2.521H2.522A2.528 2.528 0 0 1 0 8.834a2.528 2.528 0 0 1 2.522-2.521h6.312ZM18.956 8.834a2.528 2.528 0 0 1 2.522-2.521A2.528 2.528 0 0 1 24 8.834a2.528 2.528 0 0 1-2.522 2.521h-2.522V8.834ZM17.686 8.834a2.528 2.528 0 0 1-2.522 2.521 2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.164 0a2.528 2.528 0 0 1 2.522 2.522v6.312ZM15.164 18.956a2.528 2.528 0 0 1 2.522 2.522A2.528 2.528 0 0 1 15.164 24a2.527 2.527 0 0 1-2.52-2.522v-2.522h2.52ZM15.164 17.686a2.527 2.527 0 0 1-2.52-2.521 2.527 2.527 0 0 1 2.52-2.52h6.314A2.528 2.528 0 0 1 24 15.165a2.528 2.528 0 0 1-2.522 2.521h-6.314Z" + ), + xmlns: "http://www.w3.org/2000/svg", + viewBox: "0 0 24 24", + fill: "currentColor", + class: classes, + aria: { hidden: true }, + focusable: "false" + ) + when "sun" + outline_icon( + classes, + "M12 3v2.25M12 18.75V21M4.5 4.5l1.591 1.591M17.909 17.909 19.5 19.5M3 12h2.25M18.75 12H21M4.5 19.5l1.591-1.591M17.909 6.091 19.5 4.5M15.75 12a3.75 3.75 0 1 1-7.5 0 3.75 3.75 0 0 1 7.5 0Z" + ) + when "user-circle" + outline_icon( + classes, + "M15.75 9.75a3.75 3.75 0 1 1-7.5 0 3.75 3.75 0 0 1 7.5 0ZM4.5 19.5a8.25 8.25 0 1 1 15 0 9.72 9.72 0 0 0-15 0Z" + ) + when "workflow" + outline_icon( + classes, + "M6 6h3.75v3.75H6V6Zm8.25 8.25H18V18h-3.75v-3.75ZM6 14.25h3.75V18H6v-3.75Zm3.75-6.375H12a3 3 0 0 1 3 3v3.375M9.75 16.125H12a3 3 0 0 0 3-3V9.75" + ) + when "users" + outline_icon( + classes, + "M9.75 10.5a3.75 3.75 0 1 1 7.5 0 3.75 3.75 0 0 1-7.5 0ZM4.5 18.75a6.75 6.75 0 0 1 13.5 0M18 8.25a3 3 0 0 1 0 6M19.5 18.75a5.25 5.25 0 0 0-2.25-4.307" + ) + end + end + + # The brand logo for an OAuth provider as an inline SVG, or nil when we have + # no logo for it -- callers fall back to showing the provider name as text. + # Official brand marks keep their own colors (Google's G, Slack's pinwheel); + # GitHub's mark uses currentColor so it follows the theme. + def oauth_provider_logo(provider, classes: "size-6") + paths = + case provider.to_s + when "google" + [ + [ "#4285F4", "M23.52 12.273c0-.851-.076-1.67-.218-2.455H12v4.642h6.458a5.52 5.52 0 0 1-2.394 3.622v3.011h3.878c2.269-2.089 3.578-5.165 3.578-8.82Z" ], + [ "#34A853", "M12 24c3.24 0 5.956-1.075 7.942-2.907l-3.878-3.011c-1.075.72-2.45 1.145-4.064 1.145-3.125 0-5.771-2.111-6.715-4.948H1.276v3.109A11.995 11.995 0 0 0 12 24Z" ], + [ "#FBBC05", "M5.285 14.279A7.213 7.213 0 0 1 4.909 12c0-.79.136-1.56.376-2.279V6.612H1.276A11.995 11.995 0 0 0 0 12c0 1.936.464 3.769 1.276 5.388l4.009-3.109Z" ], + [ "#EA4335", "M12 4.773c1.762 0 3.344.605 4.587 1.794l3.442-3.442C17.951 1.19 15.235 0 12 0 7.31 0 3.253 2.69 1.276 6.612l4.009 3.109C6.229 6.884 8.875 4.773 12 4.773Z" ] + ] + when "slack" + [ + [ "#E01E5A", "M5.042 15.165a2.528 2.528 0 0 1-2.52 2.523A2.528 2.528 0 0 1 0 15.165a2.527 2.527 0 0 1 2.522-2.52h2.52v2.52ZM6.313 15.165a2.527 2.527 0 0 1 2.521-2.52 2.527 2.527 0 0 1 2.521 2.52v6.313A2.528 2.528 0 0 1 8.834 24a2.528 2.528 0 0 1-2.521-2.522v-6.313Z" ], + [ "#36C5F0", "M8.834 5.042a2.528 2.528 0 0 1-2.521-2.52A2.528 2.528 0 0 1 8.834 0a2.528 2.528 0 0 1 2.521 2.522v2.52H8.834ZM8.834 6.313a2.528 2.528 0 0 1 2.521 2.521 2.528 2.528 0 0 1-2.521 2.521H2.522A2.528 2.528 0 0 1 0 8.834a2.528 2.528 0 0 1 2.522-2.521h6.312Z" ], + [ "#2EB67D", "M18.956 8.834a2.528 2.528 0 0 1 2.522-2.521A2.528 2.528 0 0 1 24 8.834a2.528 2.528 0 0 1-2.522 2.521h-2.522V8.834ZM17.688 8.834a2.528 2.528 0 0 1-2.523 2.521 2.527 2.527 0 0 1-2.52-2.521V2.522A2.527 2.527 0 0 1 15.165 0a2.528 2.528 0 0 1 2.523 2.522v6.312Z" ], + [ "#ECB22E", "M15.165 18.956a2.528 2.528 0 0 1 2.523 2.522A2.528 2.528 0 0 1 15.165 24a2.527 2.527 0 0 1-2.52-2.522v-2.522h2.52ZM15.165 17.688a2.527 2.527 0 0 1-2.52-2.523 2.526 2.526 0 0 1 2.52-2.52h6.313A2.527 2.527 0 0 1 24 15.165a2.528 2.528 0 0 1-2.522 2.523h-6.313Z" ] + ] + when "github" + [ + [ "currentColor", "M12 .297c-6.63 0-12 5.373-12 12 0 5.303 3.438 9.8 8.205 11.385.6.113.82-.258.82-.577 0-.285-.01-1.04-.015-2.04-3.338.724-4.042-1.61-4.042-1.61C4.422 18.07 3.633 17.7 3.633 17.7c-1.087-.744.084-.729.084-.729 1.205.084 1.838 1.236 1.838 1.236 1.07 1.835 2.809 1.305 3.495.998.108-.776.417-1.305.76-1.605-2.665-.3-5.466-1.332-5.466-5.93 0-1.31.465-2.38 1.235-3.22-.135-.303-.54-1.523.105-3.176 0 0 1.005-.322 3.3 1.23.96-.267 1.98-.399 3-.405 1.02.006 2.04.138 3 .405 2.28-1.552 3.285-1.23 3.285-1.23.645 1.653.24 2.873.12 3.176.765.84 1.23 1.91 1.23 3.22 0 4.61-2.805 5.625-5.475 5.92.42.36.81 1.096.81 2.22 0 1.606-.015 2.896-.015 3.286 0 .315.21.69.825.57C20.565 22.092 24 17.592 24 12.297c0-6.627-5.373-12-12-12" ] + ] + when "granola" + [ + [ "currentColor", "M15.83 31.91C19.13 31.91 22.59 31.18 23.98 30.17C24.87 29.53 25.31 29.6 26.1 28.84C26.32 28.62 26.42 28.55 26.48 28.49C29.5 26.01 31.24 22.81 31.24 18.65C31.24 11.89 26.45 7.29 19.6 7.29C13.57 7.29 8.97 11.13 8.97 16.05C8.97 20.52 12.46 23.73 17.44 23.73C17.73 23.73 17.85 23.57 18.17 23.57C19.38 23.57 20.36 23.22 21 22.55C21.31 22.2 21.92 21.5 21.98 21.47C22.27 21.22 22.33 20.84 22.39 20.68C22.45 20.52 22.58 20.42 22.65 20.2C22.71 19.95 22.62 19.63 22.62 19.35C22.62 18.84 22.84 18.33 22.84 17.86C22.84 16.53 21.25 15.19 19.82 15.19C19.66 15.19 19.66 15.06 19.57 15.06C19.47 15.06 19.38 15.16 19.28 15.16C19.18 15.16 19.06 15 18.93 15C18.8 15 18.74 15.13 18.55 15.13C18.14 15.13 18.11 15.19 17.79 15.19C17.68 15.2 17.58 15.22 17.47 15.25C17.35 15.32 17.35 15.41 17.22 15.41C17.05 15.41 16.96 15.45 16.96 15.51C16.96 15.86 17 15.8 16.84 15.8C16.67 15.8 16.56 15.81 16.52 15.83C16.39 15.89 16.49 16.05 16.36 16.14C16.23 16.21 16.2 16.3 16.17 16.49C16.14 16.68 15.95 16.75 15.95 16.94C15.95 17.03 15.98 17.1 15.98 17.16C15.98 17.35 15.6 17.29 15.6 17.48C15.6 17.6 15.7 17.7 15.7 17.82C15.7 17.92 15.64 17.98 15.64 18.08C15.64 18.18 15.7 18.24 15.7 18.3C15.7 18.46 15.48 18.49 15.48 18.62C15.48 18.74 15.61 18.84 15.61 18.93C15.61 19 15.51 19.03 15.51 19.12C15.51 19.22 15.48 19.06 15.67 19.34C15.79 19.53 15.79 19.63 15.67 19.79C15.54 19.95 15.29 20.05 14.97 20.05C13.64 20.05 13.48 18.68 12.75 18.46C12.53 18.4 12.49 18.36 12.49 18.27C12.49 18.18 12.49 18.21 12.62 18.08C12.75 17.95 12.78 17.86 12.78 17.76C12.78 17.67 12.78 17.63 12.72 17.57C12.46 17.16 12.34 16.65 12.34 16.08C12.34 13.16 15.89 10.72 19.26 10.72C20.27 10.72 20.11 10.97 20.72 10.97C20.88 10.97 20.81 10.97 21.03 10.94C21.61 10.85 22.65 11.07 23.48 11.48C25.76 12.62 27.25 15.32 27.25 18.46C27.25 23.83 22.49 27.73 16.15 27.73C12.69 27.73 10.31 26.65 7.96 24.01C7.7 23.73 8.09 24.08 7.67 23.35C7.17 22.46 7.23 23.03 7.23 23.03C7.07 22.84 6.78 22.3 6.62 22.11C6.43 21.89 6.18 21.92 6.09 21.79C5.96 21.64 6.15 21.35 6.09 21.19C6.02 20.94 5.48 20.56 5.42 20.37C5.36 20.18 5.13 18.97 5.13 18.75C5.13 18.49 5.32 18.46 5.32 18.3C5.32 18.08 5.04 18.02 4.88 17.7C4.72 17.38 4.62 16.65 4.62 15.86C4.62 15.44 4.62 15.29 4.75 14.33C4.78 14.02 5.26 14.02 5.26 13.67C5.26 13.54 5.2 13.38 5.2 13.29C5.2 13.16 5.2 13.13 5.23 13.03C6.56 7.45 12.53 3.29 19.16 3.29C21.44 3.29 23.19 3.71 25.88 4.85C26.77 5.23 28.07 4.56 28.07 3.87C28.14 3.65 28.01 3.58 27.98 3.45C27.95 3.32 27.82 3.17 27.69 3.13C27.63 3.1 27.59 3.01 27.53 2.91C27.43 2.76 27.34 2.69 27.09 2.63C26.99 2.61 26.9 2.57 26.83 2.5C26.76 2.39 26.66 2.3 26.55 2.25C26.42 2.18 26.32 2.25 26.26 2.21C26.2 2.18 26.16 2.12 26.1 2.09C26.07 2.06 26 2.06 25.91 2.06C22.71 0.34 20.62 0.09 17.83 0.09C11.8 0.09 6.4 2.69 3.07 7.23C2.79 7.61 2.94 8.24 2.53 8.62C1.74 9.35 0.76 13.35 0.76 15.89C0.76 18.01 1.26 20.84 1.93 22.39C3.23 25.44 2.66 24.17 2.85 24.46C3.23 25.06 3.55 25.12 3.71 25.31C3.71 25.31 3.8 25.5 3.8 25.69C3.8 25.82 3.8 25.85 3.83 25.91C3.93 26.1 4.37 26.42 4.5 26.55C4.79 26.83 5.01 27.44 5.48 27.91C6.21 28.64 7.26 29.21 10.59 30.77C11.76 31.31 11.1 31.02 11.22 31.05C11.51 31.15 11.89 31.15 12.12 31.31C12.24 31.41 12.08 31.37 12.43 31.37C12.49 31.37 12.49 31.4 12.56 31.43C12.62 31.46 12.69 31.56 12.78 31.56C12.84 31.56 12.88 31.5 12.97 31.53C13.06 31.56 13.29 31.72 13.48 31.82C13.64 31.91 13.67 31.91 13.73 31.85C13.92 31.72 14.05 31.88 14.24 31.88C14.3 31.88 14.4 31.85 14.59 31.85C14.84 31.85 14.84 31.91 15.83 31.91" ] + ] + when "attio" + [ + [ "currentColor", "M30.65 17.78L28.06 13.64C28.06 13.64 28.05 13.62 28.04 13.62L27.84 13.29C27.46 12.67 26.79 12.3 26.06 12.3L21.89 12.29L21.6 12.75L16.62 20.72L16.35 21.16L18.44 24.5C18.82 25.12 19.49 25.49 20.22 25.49H26.06C26.78 25.49 27.46 25.11 27.84 24.5L28.05 24.17C28.05 24.17 28.06 24.16 28.06 24.16L30.65 20.01C31.07 19.33 31.07 18.46 30.65 17.78H30.65ZM29.86 19.52L27.27 23.66C27.26 23.68 27.24 23.7 27.23 23.71C27.14 23.81 27.02 23.83 26.97 23.83C26.91 23.83 26.76 23.81 26.67 23.66L24.08 19.51C24.05 19.47 24.03 19.42 24 19.37C23.98 19.32 23.96 19.27 23.95 19.22C23.89 19.01 23.89 18.79 23.95 18.58C23.98 18.48 24.02 18.37 24.08 18.28L26.66 14.14C26.66 14.14 26.67 14.13 26.67 14.13C26.73 14.04 26.81 13.99 26.88 13.98C26.9 13.97 26.93 13.97 26.95 13.97C26.96 13.97 26.96 13.97 26.97 13.97C27.03 13.97 27.18 13.99 27.27 14.14L29.86 18.28C30.1 18.65 30.1 19.14 29.86 19.52H29.86Z" ], + [ "currentColor", "M22.99 7.76C23.41 7.08 23.41 6.21 22.99 5.54L20.4 1.4L20.19 1.05C19.8 0.43 19.14 0.06 18.4 0.06H12.56C11.84 0.06 11.17 0.43 10.78 1.05L0.32 17.78C0.11 18.12 0 18.51 0 18.9C0 19.29 0.11 19.68 0.32 20.01L3.13 24.5C3.51 25.12 4.18 25.49 4.91 25.49H10.75C11.48 25.49 12.15 25.12 12.53 24.5L12.75 24.16C12.75 24.16 12.75 24.16 12.75 24.16C12.75 24.16 12.75 24.15 12.75 24.15L14.83 20.82L21.01 10.93L22.99 7.77L22.99 7.76ZM22.38 6.65C22.38 6.86 22.32 7.08 22.2 7.27L11.96 23.66C11.86 23.81 11.72 23.83 11.66 23.83C11.6 23.83 11.45 23.81 11.36 23.66L8.77 19.52C8.53 19.14 8.53 18.66 8.77 18.28L19.01 1.89C19.1 1.74 19.25 1.72 19.31 1.72C19.37 1.72 19.52 1.74 19.61 1.89L22.2 6.03C22.32 6.22 22.38 6.44 22.38 6.65V6.65Z" ] + ] + when "linear" + [ + [ "#5E6AD2", "M2.886 4.18A11.982 11.982 0 0 1 11.99 0C18.624 0 24 5.376 24 12.009c0 3.64-1.62 6.903-4.18 9.105L2.887 4.18ZM1.817 5.626l16.556 16.556c-.524.33-1.075.62-1.65.866L.951 7.277c.247-.575.537-1.126.866-1.65ZM.322 9.163l14.515 14.515c-.71.172-1.443.282-2.195.322L0 11.358a12 12 0 0 1 .322-2.195Zm-.17 4.862 9.823 9.824a12.02 12.02 0 0 1-9.824-9.824Z" ] + ] + end + return nil unless paths + + viewbox = + case provider.to_s + when "attio" then "0 0 31.08 25.55" + when "granola" then "0 0 32 32" + else "0 0 24 24" + end + + tag.svg( + safe_join(paths.map { |fill, d| tag.path(fill: fill, d: d) }), + xmlns: "http://www.w3.org/2000/svg", + viewBox: viewbox, + class: classes, + aria: { hidden: true }, + focusable: "false" + ) + end + + def console_markdown(text) + sanitize( + markdown_blocks(text.to_s).join, + tags: MARKDOWN_ALLOWED_TAGS, + attributes: MARKDOWN_ALLOWED_ATTRIBUTES + ) + end + + def console_sidebar_thread_title(session, latest_message = nil) + # sessions.title is the title api-rs generates on message append; prefer it + # over metadata heuristics. Guarded because snapshots mirrored before the + # title migration have no such column. + stored = session.title.presence if session.respond_to?(:title) + return console_sidebar_clip_one_line(stored, 48) if stored + + metadata = session.metadata_hash + summary = metadata["summary"] + title = metadata["title"].presence || + metadata["generated_title"].presence || + metadata["summary_title"].presence || + metadata["thread_title"].presence || + (metadata["thread"].is_a?(Hash) ? metadata["thread"]["title"] : nil).presence || + (metadata["summary"].is_a?(Hash) ? metadata["summary"]["title"] : nil).presence || + (summary if summary.is_a?(String)).presence || + metadata["subject"].presence || + metadata["issue_title"].presence + return console_sidebar_generated_thread_title(title) if title + + generated = console_sidebar_generated_thread_title(console_sidebar_thread_message_text(latest_message)) + return generated if generated.present? + + truncate_middle(session.thread_key, max: 42) + end + + def console_sidebar_thread_message_text(message) + return "" unless message + + message.parts_array.filter_map do |part| + next unless part.is_a?(Hash) + + case part["type"] + when "text" then part["text"].to_s + when "image" then "[image]" + when "document" then "[document]" + end + end.join("\n").squish + end + + def console_sidebar_generated_thread_title(text) + title = text.to_s + .gsub(/<@[A-Z0-9]+(?:\|[^>]+)?>/, "") + .sub(/\A\s*@?centaur\b[:,]?\s*/i, "") + .sub(/\A\s*@?U[A-Z0-9]+\b[:,]?\s*/i, "") + .sub(/\A\s*@\S+\s+/, "") + .strip + title = title.sub(/\A[*_]{1,2}(.+?)[*_]{1,2}\s*/, "\\1 ").squish + console_sidebar_clip_one_line(title, 48) + end + # The broker credential a record wraps when it is an OAuth-flow-managed static # secret; nil for ordinary secrets and for non-static kinds. Drives the "managed" # badge and the credential <-> secret cross-links. Lives in a helper (not a @@ -49,20 +366,230 @@ def id_meta_line(namespace, oid: nil) # Renders a UTC timestamp that the `localtime` Stimulus controller rewrites in # the viewer's local time zone. With relative: true it shows a "5 minutes ago" - # style string (absolute local time on hover). The ISO-8601 text is the + # style string (absolute local time on hover). Pass format: :compact with + # relative: true for short labels like "4d" or "1mo". The ISO-8601 text is the # pre-JS / no-JS fallback. Returns an em-dash placeholder for nil. - def local_time(time, relative: false) + def local_time(time, relative: false, format: nil) return tag.span("—", class: "text-zinc-600") if time.nil? iso = time.utc.iso8601 + data = { + controller: "localtime", + localtime_datetime_value: iso, + localtime_relative_value: relative + } + data[:localtime_format_value] = format.to_s if format.present? + tag.time( iso, datetime: iso, - data: { - controller: "localtime", - localtime_datetime_value: iso, - localtime_relative_value: relative - } + data: data + ) + end + + def outline_icon(classes, path) + tag.svg( + tag.path(d: path, "stroke-linecap": "round", "stroke-linejoin": "round"), + xmlns: "http://www.w3.org/2000/svg", + fill: "none", + viewBox: "0 0 24 24", + "stroke-width": "1.8", + stroke: "currentColor", + class: classes, + aria: { hidden: true }, + focusable: "false" ) end + + def markdown_blocks(raw_text) + lines = raw_text.to_s.gsub("\r\n", "\n").split("\n", -1) + blocks = [] + index = 0 + + while index < lines.length + line = lines[index] + start_index = index + + if line.blank? + index += 1 + elsif line.start_with?("```") + code_lines = [] + index += 1 + while index < lines.length && !lines[index].start_with?("```") + code_lines << lines[index] + index += 1 + end + index += 1 if index < lines.length + blocks << %(
#{ERB::Util.html_escape(code_lines.join("\n"))}
) + elsif (heading = line.match(/\A(\#{1,4})\s+(.+)\z/)) + level = heading[1].length + classes = "mb-2 mt-4 text-sm font-semibold text-zinc-100 first:mt-0" + blocks << %(#{markdown_inline(heading[2])}) + index += 1 + elsif line.match?(/\A\s*[-*+]\s+/) + items = [] + while index < lines.length && (item = lines[index].match(/\A\s*[-*+]\s+(.+)\z/)) + items << item[1] + index += 1 + end + blocks << %(
    #{items.map { |item| %(
  • #{markdown_inline(item)}
    • ) }.join}
) + elsif line.match?(/\A\s*\d+\.\s+/) + items = [] + while index < lines.length && (item = lines[index].match(/\A\s*\d+\.\s+(.+)\z/)) + items << item[1] + index += 1 + end + blocks << %(
    #{items.map { |item| %(
  1. #{markdown_inline(item)}
    2. ) }.join}
) + elsif line.match?(/\A\s*>\s?/) + quoted = [] + while index < lines.length && (quote = lines[index].match(/\A\s*>\s?(.*)\z/)) + quoted << quote[1] + index += 1 + end + blocks << %(
#{markdown_inline(quoted.join(" "))}
) + elsif markdown_table_row?(line) && markdown_table_separator?(lines[index + 1]) + header = markdown_table_cells(line) + alignments = markdown_table_alignments(lines[index + 1]) + index += 2 + rows = [] + while index < lines.length && markdown_table_row?(lines[index]) + rows << markdown_table_cells(lines[index]) + index += 1 + end + blocks << markdown_table(header, alignments, rows) + else + paragraph = [] + while index < lines.length && lines[index].present? && !markdown_block_start?(lines[index]) + paragraph << lines[index] + index += 1 + end + # Empty when the line is a table row without a separator: the progress + # guard below emits it as its own paragraph. + blocks << %(

#{markdown_inline(paragraph.join(" "))}

) unless paragraph.empty? + end + + # Guarantee forward progress: a block-start marker with no content (e.g. + # a bare "- ", "1. ", or "# ") matches a branch guard but not its inner + # consuming regex, which would otherwise spin this loop forever. Emit such + # a line as an escaped paragraph and advance. + if index == start_index + blocks << %(

#{markdown_inline(line)}

) + index += 1 + end + end + + blocks + end + + def markdown_block_start?(line) + line.start_with?("```") || + line.match?(/\A\#{1,4}\s+/) || + line.match?(/\A\s*[-*+]\s+/) || + line.match?(/\A\s*\d+\.\s+/) || + line.match?(/\A\s*>\s?/) || + markdown_table_row?(line) + end + + def markdown_table_row?(line) + stripped = line.to_s.strip + stripped.start_with?("|") && stripped.length > 1 + end + + def markdown_table_separator?(line) + return false unless line && markdown_table_row?(line) + + cells = markdown_table_cells(line) + cells.any? && cells.all? { |cell| cell.match?(/\A:?-+:?\z/) } + end + + def markdown_table_cells(line) + inner = line.strip.delete_prefix("|").delete_suffix("|") + inner.split(/(?#{markdown_inline(cell)}) + end + body_rows = rows.map do |row| + cells = Array.new(header.length) do |column| + %(#{markdown_inline(row[column].to_s)}) + end + "#{cells.join}" + end + %(
#{head_cells.join}#{body_rows.join}
) + end + + def markdown_inline(raw_text) + text = ERB::Util.html_escape(raw_text.to_s) + placeholders = [] + + text = text.gsub(/`([^`\n]+)`/) do + markdown_placeholder(placeholders, %(#{Regexp.last_match(1)})) + end + text = text.gsub(/\[([^\]]+)\]\((https?:\/\/[^\s)]+)\)/i) do + markdown_placeholder( + placeholders, + markdown_link(Regexp.last_match(1), CGI.unescapeHTML(Regexp.last_match(2))) + ) + end + text = text.gsub(%r{(?\1') + text = text.gsub(/__([^_\n]+)__/, '\1') + text = text.gsub(/~~([^~\n]+)~~/, '\1') + text = text.gsub(/(?\1') + text = text.gsub(/(?\1') + + placeholders.each_with_index do |html, offset| + text = text.gsub(markdown_token(offset), html) + end + + text + end + + def markdown_link(label, url) + unless url.to_s.match?(/\Ahttps?:\/\/[^\s<>"']+\z/i) + return label + end + + href = ERB::Util.html_escape(url) + %(#{label}) + end + + def markdown_placeholder(placeholders, html) + placeholders << html + markdown_token(placeholders.length - 1) + end + + def markdown_token(offset) + "%%MDPH#{offset}%%" + end + + def console_sidebar_clip_one_line(value, max) + one_line = value.to_s.gsub(/\s+/, " ").strip + return one_line if one_line.length <= max + + "#{one_line.slice(0, [ max - 3, 0 ].max).rstrip}..." + end end diff --git a/services/console/app/helpers/console/principals_helper.rb b/services/console/app/helpers/console/principals_helper.rb new file mode 100644 index 000000000..0c7e4a24d --- /dev/null +++ b/services/console/app/helpers/console/principals_helper.rb @@ -0,0 +1,19 @@ +module Console + # View helpers for the principal detail screen. + module PrincipalsHelper + def slack_channel_options_for_permission(permission, channel_options) + current_id = permission.channel_id.to_s + current_label = if permission.channel_name.present? + "##{permission.channel_name} (#{current_id})" + else + current_id + end + + options = channel_options.dup + if current_id.present? && !options.any? { |_label, value| value == current_id } + options.unshift([ current_label, current_id ]) + end + options + end + end +end diff --git a/services/console/app/javascript/application.js b/services/console/app/javascript/application.js index 0d7b49404..2d3bf78c7 100644 --- a/services/console/app/javascript/application.js +++ b/services/console/app/javascript/application.js @@ -1,3 +1,32 @@ // Configure your import map in config/importmap.rb. Read more: https://github.com/rails/importmap-rails import "@hotwired/turbo-rails" import "controllers" + +// PWA service worker: offline fallback page + static asset cache. Requires a +// secure context (https or localhost), so registration silently no-ops in +// plain-http dev setups. +if ("serviceWorker" in navigator) { + window.addEventListener("load", () => { + navigator.serviceWorker.register("/service-worker.js", { scope: "/" }).catch(() => {}) + }) +} + +// Ask the browser to exempt this origin's storage (IndexedDB, caches) from +// eviction under disk pressure. Granted silently for installed PWAs; a plain +// tab may ignore it. Best-effort either way. +if (navigator.storage?.persist) { + navigator.storage.persist().catch(() => {}) +} + +// Dock-icon badge for the installed app (running agents, pending approvals, +// ...). No-op in browsers without the Badging API or in a plain tab. +window.ConsoleBadge = { + set(count) { + if (!("setAppBadge" in navigator)) return + const update = count > 0 ? navigator.setAppBadge(count) : navigator.clearAppBadge() + update.catch(() => {}) + }, + clear() { + if ("clearAppBadge" in navigator) navigator.clearAppBadge().catch(() => {}) + } +} diff --git a/services/console/app/javascript/controllers/localtime_controller.js b/services/console/app/javascript/controllers/localtime_controller.js index 3bfa556d8..0f401c7f6 100644 --- a/services/console/app/javascript/controllers/localtime_controller.js +++ b/services/console/app/javascript/controllers/localtime_controller.js @@ -6,20 +6,40 @@ import { Controller } from "@hotwired/stimulus" // // data-localtime-relative-value="true" -> "5 minutes ago", with the absolute // local time as a hover tooltip. +// data-localtime-format-value="compact" -> "4d", with the absolute local time +// as a hover tooltip. +// +// Relative displays re-render every 30s so "now" ages into "1m" without a +// page visit, and truncate toward zero so 90s reads as 1m, not 2m. export default class extends Controller { - static values = { datetime: String, relative: Boolean } + static values = { datetime: String, format: String, relative: Boolean } connect() { - const date = new Date(this.datetimeValue) - if (isNaN(date.getTime())) return + this.date = new Date(this.datetimeValue) + if (isNaN(this.date.getTime())) return - const absolute = this.formatAbsolute(date) + this.render() + if (this.formatValue === "compact" || this.relativeValue) { + this.timer = setInterval(() => this.render(), 30000) + } + } - if (this.relativeValue) { - this.element.textContent = this.relativeFrom(date) + disconnect() { + if (this.timer) clearInterval(this.timer) + } + + render() { + const absolute = this.formatAbsolute(this.date) + + if (this.formatValue === "compact") { + this.element.textContent = this.compactRelativeFrom(this.date) + this.element.title = absolute + } else if (this.relativeValue) { + this.element.textContent = this.relativeFrom(this.date) this.element.title = absolute } else { this.element.textContent = absolute + this.element.title = absolute } } @@ -32,15 +52,27 @@ export default class extends Controller { } relativeFrom(date) { - const seconds = Math.round((date.getTime() - Date.now()) / 1000) + const seconds = Math.trunc((date.getTime() - Date.now()) / 1000) const rtf = new Intl.RelativeTimeFormat(undefined, { numeric: "auto" }) const units = [ ["year", 31536000], ["month", 2592000], ["day", 86400], ["hour", 3600], ["minute", 60] ] for (const [unit, secs] of units) { - if (Math.abs(seconds) >= secs) return rtf.format(Math.round(seconds / secs), unit) + if (Math.abs(seconds) >= secs) return rtf.format(Math.trunc(seconds / secs), unit) } return rtf.format(seconds, "second") } + + compactRelativeFrom(date) { + const seconds = Math.abs(Math.trunc((Date.now() - date.getTime()) / 1000)) + const units = [ + ["y", 31536000], ["mo", 2592000], ["w", 604800], + ["d", 86400], ["h", 3600], ["m", 60] + ] + for (const [unit, secs] of units) { + if (seconds >= secs) return `${Math.floor(seconds / secs)}${unit}` + } + return "now" + } } diff --git a/services/console/app/javascript/controllers/pwa_install_controller.js b/services/console/app/javascript/controllers/pwa_install_controller.js new file mode 100644 index 000000000..f308789e9 --- /dev/null +++ b/services/console/app/javascript/controllers/pwa_install_controller.js @@ -0,0 +1,44 @@ +import { Controller } from "@hotwired/stimulus" + +// Shows an "Install app" banner when the browser reports the console is +// installable, and drives the native install prompt. beforeinstallprompt fires +// once per page load — usually before any Stimulus controller connects, and +// never again across Turbo visits — so the deferred event is captured at +// module scope and controllers sync with it on connect. + +let deferredPrompt = null + +window.addEventListener("beforeinstallprompt", (event) => { + event.preventDefault() + deferredPrompt = event + window.dispatchEvent(new CustomEvent("pwa:installable")) +}) + +window.addEventListener("appinstalled", () => { + deferredPrompt = null + window.dispatchEvent(new CustomEvent("pwa:installed")) +}) + +export default class extends Controller { + static targets = ["banner"] + + connect() { + this.sync = () => { this.bannerTarget.hidden = !deferredPrompt } + window.addEventListener("pwa:installable", this.sync) + window.addEventListener("pwa:installed", this.sync) + this.sync() + } + + disconnect() { + window.removeEventListener("pwa:installable", this.sync) + window.removeEventListener("pwa:installed", this.sync) + } + + async install() { + if (!deferredPrompt) return + deferredPrompt.prompt() + await deferredPrompt.userChoice + deferredPrompt = null + this.sync() + } +} diff --git a/services/console/app/jobs/oauth/enrich_attio_credential_identity_job.rb b/services/console/app/jobs/oauth/enrich_attio_credential_identity_job.rb new file mode 100644 index 000000000..7ba634419 --- /dev/null +++ b/services/console/app/jobs/oauth/enrich_attio_credential_identity_job.rb @@ -0,0 +1,124 @@ +require "json" +require "net/http" +require "uri" + +module Oauth + class EnrichAttioCredentialIdentityJob < ApplicationJob + queue_as :default + + SELF_ENDPOINT = "https://api.attio.com/v2/self" + class AttioSelfRetryableError < StandardError; end + + retry_on AttioSelfRetryableError, wait: :polynomially_longer, attempts: 5 do |job, error| + credential_id = job.arguments.first + Rails.logger.warn do + "attio oauth credential identity enrichment failed after retries: " \ + "credential_id=#{credential_id.inspect} error=#{error.class}" + end + end + + class << self + attr_accessor :attio_api_http + end + + def perform(credential_id) + credential = BrokerCredential.includes(:oauth_app, :static_secret).find_by(id: credential_id) + return unless credential&.oauth_app&.provider == Oauth::Providers::Attio::KEY + return if credential.access_token.blank? + + workspace = attio_workspace(credential.access_token) + subject = workspace[:subject].presence + display_name = workspace[:name].presence || subject + if subject.blank? || display_name.blank? + Rails.logger.warn do + "attio oauth credential identity enrichment returned no identity: " \ + "credential=#{credential.oid}" + end + return + end + + old_name = credential.name + credential.update!( + name: "Attio – #{display_name}", + provider_subject: subject, + # Attio's token introspection identity is workspace-level, not user-level. + provider_email: nil, + foreign_id: "attio-#{credential.oauth_app.slug}-#{subject.downcase}" + ) + + secret = credential.static_secret + return unless secret + return if old_name.present? && secret.name != "#{old_name} token" + + secret.update!(name: "#{credential.name} token") + rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique => e + Rails.logger.warn do + "attio oauth credential identity enrichment failed to persist: " \ + "credential=#{credential&.oid || credential_id.inspect} error=#{e.class}" + end + end + + private + + def attio_workspace(access_token) + response = attio_api(access_token) + return {} unless response.is_a?(Hash) + + workspace_id = response["workspace_id"].presence + workspace_name = response["workspace_name"].presence + return {} if workspace_id.blank? + + { + subject: workspace_id.to_s, + name: workspace_name || response["workspace_slug"].presence || workspace_id.to_s + } + rescue AttioSelfRetryableError + raise + rescue StandardError => e + Rails.logger.debug { "attio oauth self lookup failed: #{e.class}" } + {} + end + + def attio_api(access_token) + return nil if access_token.blank? + + if self.class.attio_api_http + return self.class.attio_api_http.call( + url: SELF_ENDPOINT, + access_token: access_token + ) + end + + uri = URI.parse(SELF_ENDPOINT) + req = Net::HTTP::Get.new(uri) + req["Accept"] = "application/json" + req["Authorization"] = "Bearer #{access_token}" + req["User-Agent"] = "centaur-console" + + http = Net::HTTP.new(uri.host, uri.port) + http.use_ssl = uri.scheme == "https" + http.open_timeout = 5 + http.read_timeout = 5 + + response = http.request(req) + status = response.code.to_i + if status == 429 || status >= 500 + raise AttioSelfRetryableError, "attio self lookup http #{status}" + end + unless status / 100 == 2 + Rails.logger.warn { "attio oauth self lookup failed: status=#{status}" } + return nil + end + + parsed = JSON.parse(response.body.to_s) + parsed.is_a?(Hash) ? parsed : nil + rescue AttioSelfRetryableError + raise + rescue JSON::ParserError => e + Rails.logger.warn { "attio oauth self lookup returned invalid JSON: #{e.class}" } + nil + rescue StandardError => e + raise AttioSelfRetryableError, e.class.name + end + end +end diff --git a/services/console/app/jobs/oauth/enrich_github_credential_identity_job.rb b/services/console/app/jobs/oauth/enrich_github_credential_identity_job.rb index aba188904..276da8ae5 100644 --- a/services/console/app/jobs/oauth/enrich_github_credential_identity_job.rb +++ b/services/console/app/jobs/oauth/enrich_github_credential_identity_job.rb @@ -37,87 +37,27 @@ def perform(credential_id) return end - BrokerCredential.transaction do - credential.lock! - existing = BrokerCredential - .where(oauth_app: credential.oauth_app, provider_subject: subject) - .where.not(id: credential.id) - .first - - if existing - merge_pending_credential!(pending: credential, existing:, profile:, display_name:) - else - old_name = credential.name - credential.update!( - name: "GitHub – #{display_name}", - provider_subject: subject, - provider_email: profile[:email].presence || credential.provider_email, - foreign_id: "github-#{credential.oauth_app.slug}-#{subject.downcase}" - ) - rename_default_wrapper_secret!(credential, old_name) - end - end - rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique => e - Rails.logger.warn do - "github oauth credential identity enrichment failed to persist: " \ - "credential=#{credential&.oid || credential_id.inspect} error=#{e.class}" - end - end - - private - - def merge_pending_credential!(pending:, existing:, profile:, display_name:) - existing.lock! - old_name = existing.name - existing.update!( + old_name = credential.name + credential.update!( name: "GitHub – #{display_name}", - provider_email: profile[:email].presence || existing.provider_email || pending.provider_email, - access_token: pending.access_token, - refresh_token: pending.refresh_token, - scopes: pending.scopes, - expires_at: pending.expires_at, - last_refresh: pending.last_refresh, - next_attempt_at: pending.next_attempt_at, - failure_count: pending.failure_count, - dead: false, - dead_reason: nil + provider_subject: subject, + provider_email: profile[:email].presence || credential.provider_email, + foreign_id: "github-#{credential.oauth_app.slug}-#{subject.downcase}" ) - rename_default_wrapper_secret!(existing, old_name) - repoint_pending_sources!(pending, existing) - remove_pending_wrapper!(pending) - pending.destroy! - end - def rename_default_wrapper_secret!(credential, old_name) secret = credential.static_secret return unless secret return if old_name.present? && secret.name != "#{old_name} token" secret.update!(name: "#{credential.name} token") - end - - def repoint_pending_sources!(pending, existing) - SecretSource.referencing_broker_credential(pending).find_each do |source| - config = if source.config.is_a?(Hash) - source.config.merge("credential_id" => existing.oid) - else - { "credential_id" => existing.oid } - end - config.delete("credential_namespace") - source.update!(config: config) + rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique => e + Rails.logger.warn do + "github oauth credential identity enrichment failed to persist: " \ + "credential=#{credential&.oid || credential_id.inspect} error=#{e.class}" end end - def remove_pending_wrapper!(pending) - secret = pending.static_secret - return unless secret - - if secret.grants.exists? - secret.update!(broker_credential: nil) - else - secret.destroy! - end - end + private def github_profile(access_token) response = github_api(access_token) diff --git a/services/console/app/jobs/oauth/enrich_linear_credential_identity_job.rb b/services/console/app/jobs/oauth/enrich_linear_credential_identity_job.rb new file mode 100644 index 000000000..8c6db2b78 --- /dev/null +++ b/services/console/app/jobs/oauth/enrich_linear_credential_identity_job.rb @@ -0,0 +1,128 @@ +require "json" +require "net/http" +require "uri" + +module Oauth + class EnrichLinearCredentialIdentityJob < ApplicationJob + queue_as :default + + GRAPHQL_ENDPOINT = Oauth::Providers::Linear::GRAPHQL_ENDPOINT + VIEWER_QUERY = "{ viewer { id name email } }".freeze + class LinearProfileRetryableError < StandardError; end + + retry_on LinearProfileRetryableError, wait: :polynomially_longer, attempts: 5 do |job, error| + credential_id = job.arguments.first + Rails.logger.warn do + "linear oauth credential identity enrichment failed after retries: " \ + "credential_id=#{credential_id.inspect} error=#{error.class}" + end + end + + class << self + attr_accessor :linear_api_http + end + + def perform(credential_id) + credential = BrokerCredential.includes(:oauth_app, :static_secret).find_by(id: credential_id) + return unless credential&.oauth_app&.provider == Oauth::Providers::Linear::KEY + return if credential.access_token.blank? + + profile = linear_profile(credential.access_token) + subject = profile[:subject].presence + display_name = profile[:name].presence || profile[:email].presence || subject + if subject.blank? || display_name.blank? + Rails.logger.warn do + "linear oauth credential identity enrichment returned no identity: " \ + "credential=#{credential.oid}" + end + return + end + + old_name = credential.name + credential.update!( + name: "Linear – #{display_name}", + provider_subject: subject, + provider_email: profile[:email].presence || credential.provider_email, + foreign_id: "linear-#{credential.oauth_app.slug}-#{subject.downcase}" + ) + + secret = credential.static_secret + return unless secret + return if old_name.present? && secret.name != "#{old_name} token" + + secret.update!(name: "#{credential.name} token") + rescue ActiveRecord::RecordInvalid, ActiveRecord::RecordNotUnique => e + Rails.logger.warn do + "linear oauth credential identity enrichment failed to persist: " \ + "credential=#{credential&.oid || credential_id.inspect} error=#{e.class}" + end + end + + private + + def linear_profile(access_token) + response = linear_api(access_token) + viewer = response.is_a?(Hash) ? response.dig("data", "viewer") : nil + return {} unless viewer.is_a?(Hash) + + id = viewer["id"].presence + return {} if id.blank? + + { + subject: id.to_s, + email: viewer["email"].presence, + name: viewer["name"].presence + } + rescue LinearProfileRetryableError + raise + rescue StandardError => e + Rails.logger.debug { "linear oauth profile lookup failed: #{e.class}" } + {} + end + + def linear_api(access_token) + return nil if access_token.blank? + + if self.class.linear_api_http + return self.class.linear_api_http.call( + url: GRAPHQL_ENDPOINT, + access_token: access_token, + body: { query: VIEWER_QUERY } + ) + end + + uri = URI.parse(GRAPHQL_ENDPOINT) + req = Net::HTTP::Post.new(uri) + req["Accept"] = "application/json" + req["Authorization"] = "Bearer #{access_token}" + req["Content-Type"] = "application/json" + req["User-Agent"] = "centaur-console" + req.body = { query: VIEWER_QUERY }.to_json + + http = Net::HTTP.new(uri.host, uri.port) + http.use_ssl = uri.scheme == "https" + http.open_timeout = 5 + http.read_timeout = 5 + + response = http.request(req) + status = response.code.to_i + if status == 429 || status >= 500 + raise LinearProfileRetryableError, "linear viewer lookup http #{status}" + end + unless status / 100 == 2 + Rails.logger.warn { "linear oauth profile lookup failed: status=#{status}" } + return nil + end + + parsed = JSON.parse(response.body.to_s) + parsed.is_a?(Hash) ? parsed : nil + rescue LinearProfileRetryableError + raise + rescue JSON::ParserError => e + Rails.logger.warn { "linear oauth profile lookup returned invalid JSON: #{e.class}" } + nil + rescue StandardError => e + raise LinearProfileRetryableError, e.class.name + end + end +end diff --git a/services/console/app/models/broker_credential.rb b/services/console/app/models/broker_credential.rb index 214784f15..46132fe9a 100644 --- a/services/console/app/models/broker_credential.rb +++ b/services/console/app/models/broker_credential.rb @@ -18,15 +18,13 @@ class BrokerCredential < ApplicationRecord include ForeignIdCollisionGuard - GITHUB_APP_INSTALLATION = "github_app_installation" + GITHUB_APP_INSTALLATION = Broker::CredentialGrants::GITHUB_APP_INSTALLATION URL_SAFE_FORMAT = /\A[A-Za-z0-9\-._~]+\z/ URL_SAFE_MESSAGE = "must contain only URL-safe characters (A-Z, a-z, 0-9, -, ., _, ~)" PREQIN_TOKEN_ENDPOINT = Broker::CredentialGrants::PREQIN_TOKEN_ENDPOINT - GRANTS = (Broker::CredentialGrants::GRANTS + [ GITHUB_APP_INSTALLATION ]).uniq.freeze - REFRESHABLE_WITHOUT_TOKEN_GRANTS = - (Broker::CredentialGrants::REFRESHABLE_WITHOUT_TOKEN_GRANTS + [ GITHUB_APP_INSTALLATION ]).uniq.freeze + GRANTS = Broker::CredentialGrants::GRANTS # The access token must keep at least this much life past the scheduled # refresh, regardless of slack/fraction. Mirrors the 60s floor in @@ -76,7 +74,7 @@ class BrokerCredential < ApplicationRecord .where("(\"broker_credentials\".\"grant\" = ? AND " \ "(last_refresh IS NULL OR refresh_token IS NOT NULL)) OR " \ "\"broker_credentials\".\"grant\" IN (?)", - "refresh_token", REFRESHABLE_WITHOUT_TOKEN_GRANTS) + "refresh_token", Broker::CredentialGrants::REFRESHABLE_WITHOUT_TOKEN_GRANTS) .where("next_attempt_at IS NULL OR next_attempt_at <= ?", Time.current) } @@ -87,8 +85,7 @@ class BrokerCredential < ApplicationRecord validates :token_endpoint, presence: true # client_id is sourced from the linked OauthApp for flow-minted credentials, so # it is only required for standalone grants whose strategy uses it. - validates :client_id, presence: true, - if: -> { github_app_installation? || Broker::CredentialGrants.client_id_required?(self) } + validates :client_id, presence: true, if: -> { Broker::CredentialGrants.client_id_required?(self) } validates :external_user_key, format: { with: URL_SAFE_FORMAT, message: URL_SAFE_MESSAGE }, length: { maximum: 128 }, allow_nil: true validates :early_refresh_fraction, @@ -99,7 +96,6 @@ class BrokerCredential < ApplicationRecord validate :scopes_is_an_array validate :grant_credentials_present validate :token_endpoint_headers_valid - validate :github_app_private_key_present # OAuth client identity used for refresh. Flow-minted credentials delegate to # their OauthApp so a client-secret rotation on the app applies to every @@ -123,6 +119,10 @@ def refresh_client @refresh_client ||= Broker::RefreshClient.new end + def github_app_client + @github_app_client ||= Broker::GithubAppInstallationClient.new + end + def refresh_scopes_for_provider oauth_app&.provider_strategy&.refresh_scopes(scopes) || scopes end @@ -183,27 +183,10 @@ def auto_grant_matching_principals PrincipalCredentialReconciliation.new.apply_for_credential(self) end - def github_app_client - @github_app_client ||= Broker::GithubAppInstallationClient.new - end - def perform_refresh(now:) - if github_app_installation? - result = github_app_client.mint( - token_endpoint: token_endpoint, - app_id: effective_client_id, - private_key_pem: effective_client_secret, - timeout: refresh_timeout_seconds, - now: now - ) - return Broker::CredentialGrants::Outcome.new(result: result, clear_refresh_token: true, dead_reason: nil) - end - - Broker::CredentialGrants.refresh(self) + Broker::CredentialGrants.refresh(self, now: now) end - def github_app_installation? = grant == GITHUB_APP_INSTALLATION - def apply_success!(result, now:, clear_refresh_token:) expires_in = result.expires_in&.positive? ? result.expires_in : DEFAULT_EXPIRES_IN_SECONDS attrs = { @@ -275,8 +258,6 @@ def scopes_is_an_array end def grant_credentials_present - return if github_app_installation? - Broker::CredentialGrants.validate(self) end @@ -287,12 +268,6 @@ def token_endpoint_headers_valid errors.add(:token_endpoint_headers, "must be an object mapping header names to string values") unless valid end - def github_app_private_key_present - return unless github_app_installation? - - errors.add(:client_secret, "can't be blank for a GitHub App installation credential") if effective_client_secret.blank? - end - def default_preqin_token_endpoint return unless grant == "preqin" diff --git a/services/console/app/models/centaur_session.rb b/services/console/app/models/centaur_session.rb new file mode 100644 index 000000000..fdfd27e84 --- /dev/null +++ b/services/console/app/models/centaur_session.rb @@ -0,0 +1,28 @@ +class CentaurSession < CentaurSessionRecord + self.table_name = "sessions" + self.primary_key = "thread_key" + + has_many :messages, + class_name: "CentaurSessionMessage", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :session + has_many :executions, + class_name: "CentaurSessionExecution", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :session + has_many :events, + class_name: "CentaurSessionEvent", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :session + + scope :recent_first, -> { order(Arel.sql("coalesce(updated_at, created_at) desc"), :thread_key) } + + def readonly? = true + + def metadata_hash + metadata.is_a?(Hash) ? metadata : {} + end +end diff --git a/services/console/app/models/centaur_session_event.rb b/services/console/app/models/centaur_session_event.rb new file mode 100644 index 000000000..4f5ed2d8d --- /dev/null +++ b/services/console/app/models/centaur_session_event.rb @@ -0,0 +1,16 @@ +class CentaurSessionEvent < CentaurSessionRecord + self.table_name = "session_events" + self.primary_key = "event_id" + + belongs_to :session, + class_name: "CentaurSession", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :events + + def readonly? = true + + def payload_hash + payload.is_a?(Hash) ? payload : {} + end +end diff --git a/services/console/app/models/centaur_session_execution.rb b/services/console/app/models/centaur_session_execution.rb new file mode 100644 index 000000000..aec091050 --- /dev/null +++ b/services/console/app/models/centaur_session_execution.rb @@ -0,0 +1,12 @@ +class CentaurSessionExecution < CentaurSessionRecord + self.table_name = "session_executions" + self.primary_key = "execution_id" + + belongs_to :session, + class_name: "CentaurSession", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :executions + + def readonly? = true +end diff --git a/services/console/app/models/centaur_session_message.rb b/services/console/app/models/centaur_session_message.rb new file mode 100644 index 000000000..4713c775f --- /dev/null +++ b/services/console/app/models/centaur_session_message.rb @@ -0,0 +1,20 @@ +class CentaurSessionMessage < CentaurSessionRecord + self.table_name = "session_messages" + self.primary_key = "message_id" + + belongs_to :session, + class_name: "CentaurSession", + foreign_key: :thread_key, + primary_key: :thread_key, + inverse_of: :messages + + def readonly? = true + + def parts_array + parts.is_a?(Array) ? parts : [] + end + + def metadata_hash + metadata.is_a?(Hash) ? metadata : {} + end +end diff --git a/services/console/app/models/centaur_session_record.rb b/services/console/app/models/centaur_session_record.rb new file mode 100644 index 000000000..a04c51fd2 --- /dev/null +++ b/services/console/app/models/centaur_session_record.rb @@ -0,0 +1,57 @@ +class CentaurSessionRecord < ActiveRecord::Base + self.abstract_class = true + + DEFAULT_DATABASE_NAME = "ai_v2".freeze + + class << self + private + + def session_database_configuration + explicit_url = + ConsoleEnv["CENTAUR_DATABASE_URL"].presence || ENV["CENTAUR_DATABASE_URL"].presence + if explicit_url + return { + adapter: "postgresql", + encoding: "unicode", + pool: ENV.fetch("RAILS_MAX_THREADS", 5), + url: explicit_url + } + end + + config = primary_database_configuration.deep_symbolize_keys + + # When the primary config carries a :url (the common single-URL dev setup + # where database.yml's default block sets url: ), Rails' + # UrlConfig merges the URL-derived keys OVER sibling hash keys. That means + # a database path inside the primary URL would override any :database we + # set here, silently pointing the session models at the console's own DB. + # Resolve the URL into discrete connection params and drop :url so the + # ai_v2 database name below is authoritative. + if config[:url].present? + resolved = ActiveRecord::DatabaseConfigurations::ConnectionUrlResolver + .new(config.delete(:url)) + .to_hash + .symbolize_keys + config = config.merge(resolved) + else + config.delete(:url) + end + + config[:database] = session_database_name(config) + config + end + + def primary_database_configuration + env_config = Rails.application.config.database_configuration.fetch(Rails.env) + (env_config["primary"] || env_config).deep_dup + end + + def session_database_name(config) + ConsoleEnv["CENTAUR_DATABASE_NAME"].presence || + ENV["CENTAUR_DATABASE_NAME"].presence || + (Rails.env.test? ? config[:database] : DEFAULT_DATABASE_NAME) + end + end + + establish_connection session_database_configuration +end diff --git a/services/console/app/models/centaur_workflow_run.rb b/services/console/app/models/centaur_workflow_run.rb new file mode 100644 index 000000000..f12124f07 --- /dev/null +++ b/services/console/app/models/centaur_workflow_run.rb @@ -0,0 +1,158 @@ +class CentaurWorkflowRun < CentaurSessionRecord + self.table_name = "centaur_readonly_workflow_runs" + self.primary_key = "run_id" + + RECENCY_SQL = + "coalesce(completed_at, failed_at, cancelled_at, started_at, " \ + "first_started_at, available_at, created_at)".freeze + + RECENT_ORDER = Arel.sql("#{RECENCY_SQL} desc, task_id desc") + + # Mirrors #workflow_key: runs are grouped under their workflow name, falling + # back to the task name for runs enqueued before workflow names were recorded. + WORKFLOW_KEY_SQL = + "coalesce(nullif(workflow_name, ''), nullif(task_name, ''))".freeze + + # SQL twin of #display_status so status filters and tab counts agree with the + # badge rendered for each row. + DISPLAY_STATUS_SQL = <<~SQL.squish.freeze + CASE + WHEN cancelled_at IS NOT NULL THEN 'cancelled' + WHEN failed_at IS NOT NULL THEN 'failed' + WHEN completed_at IS NOT NULL THEN 'completed' + WHEN claimed OR state = 'running' THEN 'running' + ELSE coalesce(nullif(state, ''), 'unknown') + END + SQL + + scope :recent_first, -> { order(RECENT_ORDER) } + + class << self + def available? + connection.data_source_exists?(table_name) + end + + def recent(limit:) + recent_first.limit(limit).to_a + end + + # The most recent run of each distinct workflow, newest activity first. + def latest_per_workflow(limit:, offset: 0) + find_by_sql([ <<~SQL, { limit: limit, offset: offset } ]) + SELECT * FROM ( + SELECT DISTINCT ON (#{WORKFLOW_KEY_SQL}) * + FROM #{table_name} + ORDER BY #{WORKFLOW_KEY_SQL}, #{RECENCY_SQL} DESC, task_id DESC + ) latest_runs + ORDER BY #{RECENCY_SQL} DESC, task_id DESC + LIMIT :limit OFFSET :offset + SQL + end + + def workflow_count + count_by_sql( + "SELECT COUNT(*) FROM " \ + "(SELECT DISTINCT #{WORKFLOW_KEY_SQL} FROM #{table_name}) workflow_keys" + ) + end + + # The most recent run per (workflow, queue) for the given workflow keys, + # grouped by workflow key. Each run carries a queue_run_count attribute with + # that queue's total run count. Feeds the per-queue lines on the index. + def latest_per_queue(workflow_keys) + keys = workflow_keys.compact.uniq + return {} if keys.empty? + + runs = find_by_sql([ <<~SQL, { keys: keys } ]) + SELECT DISTINCT ON (#{WORKFLOW_KEY_SQL}, queue_name) *, + COUNT(*) OVER (PARTITION BY #{WORKFLOW_KEY_SQL}, queue_name) AS queue_run_count + FROM #{table_name} + WHERE #{WORKFLOW_KEY_SQL} IN (:keys) + ORDER BY #{WORKFLOW_KEY_SQL}, queue_name, #{RECENCY_SQL} DESC, task_id DESC + SQL + + runs + .group_by(&:workflow_key) + .transform_values { |queue_runs| queue_runs.sort_by { |run| run.recency_at&.to_time.to_i }.reverse } + end + + def for_workflow(workflow_name, limit:, offset: 0, status: nil, queue: nil) + workflow_scope(workflow_name, status: status, queue: queue) + .recent_first + .limit(limit) + .offset(offset) + .to_a + end + + def run_count(workflow_name, status: nil, queue: nil) + workflow_scope(workflow_name, status: status, queue: queue).count + end + + # { "completed" => 12, "running" => 1, ... } for one workflow's runs. + def status_counts(workflow_name) + workflow_scope(workflow_name) + .group(Arel.sql(DISPLAY_STATUS_SQL)) + .count + end + + def queue_names(workflow_name) + workflow_scope(workflow_name) + .distinct + .order(:queue_name) + .pluck(:queue_name) + end + + def queue_label_for(queue_name) + suffix = queue_name.to_s.delete_prefix("centaur_workflows").delete_prefix("_") + suffix.presence&.tr("_", " ") || "default" + end + + private + + def workflow_scope(workflow_name, status: nil, queue: nil) + scope = where( + "workflow_name = :workflow_name OR " \ + "((workflow_name IS NULL OR workflow_name = '') AND task_name = :workflow_name)", + workflow_name: workflow_name + ) + scope = scope.where("#{DISPLAY_STATUS_SQL} = ?", status) if status.present? + scope = scope.where(queue_name: queue) if queue.present? + scope + end + end + + def readonly? = true + + def workflow_name_label + workflow_name.presence || task_name.presence || "unknown workflow" + end + + def workflow_key + workflow_name.presence || task_name.presence + end + + def queue_label + self.class.queue_label_for(queue_name) + end + + def display_status + return "cancelled" if cancelled_at.present? + return "failed" if failed_at.present? + return "completed" if completed_at.present? + return "running" if claimed || state == "running" + + state.presence || "unknown" + end + + def started_or_created_at + started_at || first_started_at || created_at + end + + def terminal_at + completed_at || failed_at || cancelled_at + end + + def recency_at + terminal_at || started_at || first_started_at || available_at || created_at + end +end diff --git a/services/console/app/models/concerns/hashed_token_lookup.rb b/services/console/app/models/concerns/hashed_token_lookup.rb new file mode 100644 index 000000000..9bd7ce5b1 --- /dev/null +++ b/services/console/app/models/concerns/hashed_token_lookup.rb @@ -0,0 +1,27 @@ +require "digest" + +# Lookup of single-use secrets stored as SHA-256 digests. Including models +# declare the digest column and must define a `usable` scope. +module HashedTokenLookup + extend ActiveSupport::Concern + + class_methods do + # Acts as both setter (when called with a value) and getter. + # class McpOauthRefreshToken < ApplicationRecord + # token_hash_attribute :token_hash + # end + def token_hash_attribute(value = nil) + @token_hash_attribute = value.to_sym if value + @token_hash_attribute or + raise NotImplementedError, "#{name} must declare `token_hash_attribute :...`" + end + + def hash_token(value) + Digest::SHA256.hexdigest(value.to_s) + end + + def find_usable(value) + usable.find_by(token_hash_attribute => hash_token(value)) + end + end +end diff --git a/services/console/app/models/mcp_oauth_authorization_code.rb b/services/console/app/models/mcp_oauth_authorization_code.rb new file mode 100644 index 000000000..ed3e46856 --- /dev/null +++ b/services/console/app/models/mcp_oauth_authorization_code.rb @@ -0,0 +1,36 @@ +class McpOauthAuthorizationCode < ApplicationRecord + include HashedTokenLookup + + oid_prefix "moa" + token_hash_attribute :code_hash + + CODE_TTL = 10.minutes + TOKEN_PREFIX = "mcpauth_".freeze + + attr_accessor :plaintext_code + + belongs_to :mcp_oauth_client + belongs_to :user + belongs_to :principal + + before_validation :issue_code, on: :create + + validates :code_hash, presence: true, uniqueness: true + validates :redirect_uri, :code_challenge, :resource, :expires_at, presence: true + validates :scopes, presence: true + + scope :usable, -> { where(consumed_at: nil).where("expires_at > ?", Time.current) } + + def consume! + update!(consumed_at: Time.current) + end + + private + + def issue_code + self.expires_at ||= CODE_TTL.from_now + return if code_hash.present? + self.plaintext_code = "#{TOKEN_PREFIX}#{SecureRandom.urlsafe_base64(48)}" + self.code_hash = self.class.hash_token(plaintext_code) + end +end diff --git a/services/console/app/models/mcp_oauth_client.rb b/services/console/app/models/mcp_oauth_client.rb new file mode 100644 index 000000000..38649610f --- /dev/null +++ b/services/console/app/models/mcp_oauth_client.rb @@ -0,0 +1,93 @@ +require "ipaddr" +require "uri" + +class McpOauthClient < ApplicationRecord + oid_prefix "moc" + + DEFAULT_GRANT_TYPES = %w[authorization_code refresh_token].freeze + DEFAULT_RESPONSE_TYPES = %w[code].freeze + DEFAULT_SCOPES = %w[mcp:tools].freeze + + has_many :authorization_codes, class_name: "McpOauthAuthorizationCode", dependent: :destroy + has_many :refresh_tokens, class_name: "McpOauthRefreshToken", dependent: :destroy + + validates :redirect_uris, presence: true + validate :redirect_uris_valid + validate :grant_types_supported + validate :response_types_supported + validate :scopes_supported + + def public_client_id = oid + + def redirect_uri_allowed?(uri) + return false unless self.class.allowed_redirect_uri?(uri) + + requested = URI.parse(uri.to_s) + redirect_uris.any? do |registered| + next false unless self.class.allowed_redirect_uri?(registered) + next true if registered == uri.to_s + + registered_uri = URI.parse(registered.to_s) + loopback_redirect_uri_match?(registered_uri, requested) + rescue URI::InvalidURIError + false + end + rescue URI::InvalidURIError + false + end + + private + + def redirect_uris_valid + return errors.add(:redirect_uris, "must be an array") unless redirect_uris.is_a?(Array) + errors.add(:redirect_uris, "must not be empty") if redirect_uris.empty? + redirect_uris.each do |uri| + errors.add(:redirect_uris, "#{uri.inspect} is not an allowed public-client redirect URI") unless self.class.allowed_redirect_uri?(uri) + end + end + + def grant_types_supported + return errors.add(:grant_types, "must be an array") unless grant_types.is_a?(Array) + unsupported = grant_types - DEFAULT_GRANT_TYPES + errors.add(:grant_types, "contains unsupported values: #{unsupported.join(', ')}") if unsupported.any? + end + + def response_types_supported + return errors.add(:response_types, "must be an array") unless response_types.is_a?(Array) + unsupported = response_types - DEFAULT_RESPONSE_TYPES + errors.add(:response_types, "contains unsupported values: #{unsupported.join(', ')}") if unsupported.any? + end + + def scopes_supported + return errors.add(:scopes, "must be an array") unless scopes.is_a?(Array) + unsupported = scopes - DEFAULT_SCOPES + errors.add(:scopes, "contains unsupported values: #{unsupported.join(', ')}") if unsupported.any? + end + + def self.allowed_redirect_uri?(value) + uri = URI.parse(value.to_s) + uri.scheme == "http" && loopback_host?(uri.host) + rescue URI::InvalidURIError + false + end + + def loopback_redirect_uri_match?(registered_uri, requested_uri) + return false unless registered_uri.scheme == "http" && requested_uri.scheme == "http" + return false unless self.class.loopback_host?(registered_uri.host) + return false unless self.class.loopback_host?(requested_uri.host) + return false if registered_uri.port && registered_uri.port != registered_uri.default_port + return false unless registered_uri.path == requested_uri.path + return false unless registered_uri.query == requested_uri.query + + true + end + + def self.loopback_host?(host) + normalized = host.to_s.downcase + return true if normalized == "localhost" + + IPAddr.new(normalized).loopback? + rescue IPAddr::Error + false + end +end diff --git a/services/console/app/models/mcp_oauth_refresh_token.rb b/services/console/app/models/mcp_oauth_refresh_token.rb new file mode 100644 index 000000000..d6b3809d4 --- /dev/null +++ b/services/console/app/models/mcp_oauth_refresh_token.rb @@ -0,0 +1,36 @@ +class McpOauthRefreshToken < ApplicationRecord + include HashedTokenLookup + + oid_prefix "mor" + token_hash_attribute :token_hash + + DEFAULT_TTL = 90.days + TOKEN_PREFIX = "mcprt_".freeze + + attr_accessor :plaintext_token + + belongs_to :mcp_oauth_client + belongs_to :user + belongs_to :principal + + before_validation :issue_token, on: :create + + validates :token_hash, presence: true, uniqueness: true + validates :resource, :expires_at, presence: true + validates :scopes, presence: true + + scope :usable, -> { where(revoked_at: nil).where("expires_at > ?", Time.current) } + + def revoke! + update!(revoked_at: Time.current) + end + + private + + def issue_token + self.expires_at ||= DEFAULT_TTL.from_now + return if token_hash.present? + self.plaintext_token = "#{TOKEN_PREFIX}#{SecureRandom.urlsafe_base64(48)}" + self.token_hash = self.class.hash_token(plaintext_token) + end +end diff --git a/services/console/app/models/principal.rb b/services/console/app/models/principal.rb index bbd6aff61..0a458d784 100644 --- a/services/console/app/models/principal.rb +++ b/services/console/app/models/principal.rb @@ -1,4 +1,7 @@ +require "uri" + class Principal < ApplicationRecord + FEEDBACK_API_SECRET_NAME = "SLACK_FEEDBACK_API_KEY".freeze oid_prefix "prn" include ForeignIdCollisionGuard @@ -11,38 +14,39 @@ class Principal < ApplicationRecord has_many :proxies, dependent: :nullify has_many :principal_roles, dependent: :destroy has_many :roles, through: :principal_roles + has_many :slack_channel_permissions, dependent: :destroy has_many :sync_config_snapshots, class_name: "PrincipalSyncConfigSnapshot", dependent: :destroy + has_many :mcp_oauth_authorization_codes, dependent: :destroy + has_many :mcp_oauth_refresh_tokens, dependent: :destroy belongs_to :created_by, class_name: "User" + accepts_nested_attributes_for :slack_channel_permissions, + allow_destroy: true, + reject_if: :reject_slack_channel_permission_attributes? + after_commit :auto_grant_matching_oauth_credentials, on: %i[create update] + before_validation :apply_sandbox_repo_cache_label before_commit :bump_own_sync_config_cache_version, on: :update, if: :sync_config_fields_changed? URL_SAFE_FORMAT = /\A[A-Za-z0-9\-._~]+\z/ URL_SAFE_MESSAGE = "must contain only URL-safe characters (A-Z, a-z, 0-9, -, ., _, ~)" + SANDBOX_REPO_CACHE_LABEL = "centaur.sandbox_repo_cache".freeze + SANDBOX_REPO_CACHE_VALUES = %w[none public all].freeze validates :namespace, presence: true, format: { with: URL_SAFE_FORMAT, message: URL_SAFE_MESSAGE } validates :foreign_id, uniqueness: { scope: :namespace, allow_nil: true }, format: { with: URL_SAFE_FORMAT, message: URL_SAFE_MESSAGE }, allow_nil: true + validates :sandbox_repo_cache, inclusion: { in: SANDBOX_REPO_CACHE_VALUES } # Stand-in for an inline secret value in redacted config: effective_config # reports that a control_plane source carries a value without revealing it. REDACTED = "[redacted]".freeze - - # Managed proxies do not read the baked local proxy YAML; they receive their - # runtime config from iron-control sync. Keep this in sync with - # services/iron-proxy/iron-proxy.yaml and the infra fragment. - MANAGED_PROXY_CONFIG = { - "upstream_response_header_timeout" => "120s" - }.freeze + SLACK_CHANNEL_ID_LABEL = "slack_channel_id".freeze + SLACK_CHANNEL_ID_FORMAT = /\A[CDG][A-Z0-9]{8,}\z/ # The config of a principal with no effective grants; also what an unassigned # proxy resolves to. - EMPTY_CONFIG = { - "proxy" => MANAGED_PROXY_CONFIG, - "secrets" => [].freeze, - "transforms" => [].freeze, - "postgres" => [].freeze - }.freeze + EMPTY_CONFIG = { "secrets" => [], "transforms" => [], "postgres" => [] }.freeze # Every grant this principal resolves to: its own direct grants plus the # grants of every role it is assigned. Secrets reachable through more than one @@ -125,16 +129,56 @@ def sync_postgres # it passes through untouched. def effective_config(redact_secrets: true) served = served_credentials - config = self.class.with_managed_proxy_config( - "secrets" => proxy_secrets_for(served), + config = { + "secrets" => proxy_secrets_for(served) + generated_proxy_secrets, "transforms" => proxy_transforms_for(served), "postgres" => sync_postgres - ) + } redact_secrets ? self.class.redact_live_secrets(config) : config end - def self.with_managed_proxy_config(config) - config.merge("proxy" => MANAGED_PROXY_CONFIG) + def apply_default_sandbox_capabilities!(supplied = {}) + return unless new_record? + + defaults = SystemSetting.current.principal_defaults + unless supplied_key?(supplied, :sandbox_repo_cache) + self.sandbox_repo_cache = defaults[:sandbox_repo_cache] + end + unless supplied_key?(supplied, :sandbox_observability_enabled) + self.sandbox_observability_enabled = defaults[:sandbox_observability_enabled] + end + unless supplied_key?(supplied, :sandbox_api_server_enabled) + self.sandbox_api_server_enabled = defaults[:sandbox_api_server_enabled] + end + end + + def labels_with_sandbox_capabilities + labels.to_h.merge(SANDBOX_REPO_CACHE_LABEL => sandbox_repo_cache) + end + + def slack_channel_permissions_payload + permissions = if association(:slack_channel_permissions).loaded? + slack_channel_permissions.sort_by { |permission| [ permission.channel_id, permission.id ] } + else + slack_channel_permissions.ordered + end + permissions.map(&:as_permission_json) + end + + def slack_upload_channel_ids + slack_channel_ids_for(:upload_enabled) + end + + def slack_download_channel_ids + slack_channel_ids_for(:download_enabled) + end + + def slack_history_channel_ids + slack_channel_ids_for(:history_enabled) + end + + def slack_jwt_channel_ids + (slack_upload_channel_ids + slack_download_channel_ids + slack_history_channel_ids).uniq end def self.bump_sync_config_cache_versions(ids) @@ -174,6 +218,18 @@ def auto_grant_matching_oauth_credentials PrincipalCredentialReconciliation.new.apply_for_principal(self) end + def apply_sandbox_repo_cache_label + self[:labels] = labels.to_h.merge(SANDBOX_REPO_CACHE_LABEL => sandbox_repo_cache) + end + + def supplied_key?(attributes, key) + attributes.key?(key) || attributes.key?(key.to_s) + end + + def reject_slack_channel_permission_attributes?(attributes) + attributes["channel_id"].blank? + end + # The credentials actually delivered to the proxy, grouped by type, after # cross-type conflict resolution. Static secrets without a deliverable source # are dropped first (the proxy can't resolve a value for them) so a @@ -200,7 +256,51 @@ def served_credentials end def proxy_secrets_for(served) - served[:static].map(&:to_proxy_secret) + served[:static].map do |secret| + entry = secret.to_proxy_secret + next entry unless secret.name == FEEDBACK_API_SECRET_NAME + + # Tool manifests cannot know a Helm release-qualified Service hostname. + # The API accepts this credential only alongside the generated caller + # JWT and binds feedback-improvement:* sessions to that principal, so + # extend only its request rules to the JWT's canonical API hosts. + entry["rules"] = (entry.fetch("rules", []) + api_server_hosts.map { |host| { "host" => host } }).uniq + entry + end + end + + def generated_proxy_secrets + secret = api_server_jwt_secret + secret ? [ secret ] : [] + end + + def api_server_jwt_secret + return nil unless sandbox_api_server_enabled? + + token = ApiServer::Jwt.encode_for_principal(self) + return nil if token.blank? + + rules = api_server_hosts.map { |host| { "host" => host } } + return nil if rules.empty? + + { + "source" => { "type" => "control_plane", "value" => token }, + "inject" => { "header" => "Authorization", "formatter" => "Bearer {{ .Value }}" }, + "rules" => rules + } + end + + def slack_channel_ids_for(permission) + slack_channel_permissions.where(permission => true).ordered.pluck(:channel_id) + end + + def api_server_hosts + configured = ENV["CENTAUR_API_SERVER_PROXY_HOSTS"].to_s.split(",") + from_url = self.class.host_from_url(ENV["CENTAUR_API_URL"]) + (configured + [ from_url, "centaur-api-rs", "api" ]) + .map { |host| host.to_s.strip.downcase.delete_suffix(".") } + .reject(&:blank?) + .uniq end def proxy_transforms_for(served) @@ -215,6 +315,12 @@ def proxy_transforms_for(served) transforms end + def self.host_from_url(value) + URI.parse(value.to_s).host + rescue URI::InvalidURIError + nil + end + # Cross-type conflict resolution. The wire protocol applies the `secrets` array # (static secrets) before the `transforms` array (gcp_auth, aws_auth, hmac_sign, # oauth_token), so the proxy's last-transform-wins cannot let a direct static @@ -338,7 +444,9 @@ def granted_secrets_by_priority(model, foreign_key, includes:) end def sync_config_fields_changed? - previous_changes.key?("name") || previous_changes.key?("labels") + previous_changes.key?("name") || + previous_changes.key?("labels") || + previous_changes.key?("sandbox_api_server_enabled") end def bump_own_sync_config_cache_version diff --git a/services/console/app/models/principal_sync_config_snapshot.rb b/services/console/app/models/principal_sync_config_snapshot.rb index 1c09bb9f0..a1c6e23ad 100644 --- a/services/console/app/models/principal_sync_config_snapshot.rb +++ b/services/console/app/models/principal_sync_config_snapshot.rb @@ -10,12 +10,25 @@ class PrincipalSyncConfigSnapshot < ApplicationRecord validates :principal_cache_version, presence: true validates :principal_id, uniqueness: { scope: :principal_cache_version } + # Returns the freshest usable snapshot, stale-while-revalidate style. When + # the current-version snapshot is stale or missing, exactly one caller + # rebuilds it (non-blocking row lock on the principal); concurrent callers + # are served the stale snapshot immediately instead of queuing behind the + # rebuild. Config invalidations fan out to every proxy of a principal at + # once (cache-version bumps, TTL expiry), so blocking here previously + # stampeded all of them onto one row lock, each holding a request thread + # and DB connection for the full rebuild. + # + # Serving a stale snapshot is safe: iron-proxy treats the config hash as an + # ETag and re-applies on its next 5s poll once the rebuild lands. Only a + # cold start (no snapshot at any version) blocks until the build finishes, + # because there is nothing stale to serve. def self.fetch_for(principal) version = principal.sync_config_cache_version snapshot = find_by(principal: principal, principal_cache_version: version) - return snapshot if snapshot&.fresh? + return snapshot if snapshot&.fresh_for?(principal) - build_for(principal) + try_build_for(principal) || snapshot || latest_for(principal) || build_for(principal) end def self.prune_expired! @@ -26,19 +39,61 @@ def fresh? updated_at >= TTL.ago end + def fresh_for?(principal) + fresh? && !api_server_jwt_window_stale?(principal) + end + + # Most recent snapshot at any cache version; the stale fallback while + # another session rebuilds. Old versions survive until prune_expired! + # (RETENTION), which comfortably covers a rebuild. + def self.latest_for(principal) + where(principal: principal).order(updated_at: :desc).first + end + def self.build_for(principal) - principal.with_lock do - principal.reload - version = principal.sync_config_cache_version - snapshot = find_or_initialize_by(principal: principal, principal_cache_version: version) - return snapshot if snapshot.persisted? && snapshot.fresh? - - config = principal.effective_config(redact_secrets: false) - snapshot.payload = config - snapshot.save! - snapshot + principal.with_lock { build_within_lock(principal) } + rescue ActiveRecord::RecordNotUnique + retry + end + + # Non-blocking variant of build_for: acquires the principal row lock with + # SKIP LOCKED and returns nil when another session already holds it. + def self.try_build_for(principal) + transaction do + locked = Principal.lock("FOR UPDATE SKIP LOCKED").find_by(id: principal.id) + next nil unless locked + + build_within_lock(locked) end rescue ActiveRecord::RecordNotUnique retry end + + # Assumes the caller holds the principal's row lock and passes the freshly + # locked (reloaded) record, so sync_config_cache_version is current. + def self.build_within_lock(principal) + version = principal.sync_config_cache_version + snapshot = find_or_initialize_by(principal: principal, principal_cache_version: version) + return snapshot if snapshot.persisted? && snapshot.fresh_for?(principal) + + snapshot.payload = principal.effective_config(redact_secrets: false) + if snapshot.changed? + snapshot.save! + else + # A rebuild that yields an identical payload must still restart the TTL, + # or the snapshot stays permanently stale and every poll re-runs the + # expensive effective_config rebuild. + snapshot.touch + end + snapshot + end + + def api_server_jwt_window_stale?(principal) + return false unless principal.sandbox_api_server_enabled? + + return false if principal.slack_jwt_channel_ids.empty? + return false if ENV["CENTAUR_JWT_SIGNING_SECRET"].to_s.blank? + + updated_at.to_i < ApiServer::Jwt.window_start_for(principal, Time.current.to_i) + end end diff --git a/services/console/app/models/proxy.rb b/services/console/app/models/proxy.rb index f54f9929a..cf7645b7b 100644 --- a/services/console/app/models/proxy.rb +++ b/services/console/app/models/proxy.rb @@ -46,11 +46,7 @@ def sync_config end def sync_config_snapshot - config = if principal - Principal.with_managed_proxy_config(PrincipalSyncConfigSnapshot.fetch_for(principal).payload) - else - Principal::EMPTY_CONFIG - end + config = principal ? PrincipalSyncConfigSnapshot.fetch_for(principal).payload : Principal::EMPTY_CONFIG { config_hash: config_hash_for(config), config: config } end diff --git a/services/console/app/models/slack_channel_permission.rb b/services/console/app/models/slack_channel_permission.rb new file mode 100644 index 000000000..8849b6a9e --- /dev/null +++ b/services/console/app/models/slack_channel_permission.rb @@ -0,0 +1,51 @@ +class SlackChannelPermission < ApplicationRecord + belongs_to :principal + + before_validation :normalize_channel_fields + after_commit :bump_principal_sync_config_cache_version + + validates :channel_id, presence: true, + format: { with: Principal::SLACK_CHANNEL_ID_FORMAT, message: "is not a valid Slack channel ID" }, + uniqueness: { scope: :principal_id } + validates :upload_enabled, inclusion: { in: [ true, false ] } + validates :download_enabled, inclusion: { in: [ true, false ] } + validates :history_enabled, inclusion: { in: [ true, false ] } + validate :at_least_one_permission + + scope :ordered, -> { order(:channel_id, :id) } + + def self.replace_for_principal!(principal, permission_rows) + transaction do + principal.slack_channel_permissions.destroy_all + permission_rows.each do |attrs| + principal.slack_channel_permissions.create!(attrs) + end + end + end + + def as_permission_json + { + "channel_id" => channel_id, + "channel_name" => channel_name, + "upload_enabled" => upload_enabled, + "download_enabled" => download_enabled, + "history_enabled" => history_enabled + } + end + + private + + def normalize_channel_fields + self.channel_id = channel_id.to_s.strip.upcase + self.channel_name = channel_name.to_s.strip.presence + end + + def at_least_one_permission + return if upload_enabled || download_enabled || history_enabled + errors.add(:base, "Select at least one Slack permission") + end + + def bump_principal_sync_config_cache_version + Principal.bump_sync_config_cache_versions(principal_id) + end +end diff --git a/services/console/app/models/slack_sync_user.rb b/services/console/app/models/slack_sync_user.rb new file mode 100644 index 000000000..625803871 --- /dev/null +++ b/services/console/app/models/slack_sync_user.rb @@ -0,0 +1,7 @@ +class SlackSyncUser < CentaurSessionRecord + self.table_name = "slack_sync_users" + + def readonly? + true + end +end diff --git a/services/console/app/models/system_setting.rb b/services/console/app/models/system_setting.rb new file mode 100644 index 000000000..6334a8795 --- /dev/null +++ b/services/console/app/models/system_setting.rb @@ -0,0 +1,30 @@ +class SystemSetting < ApplicationRecord + attr_readonly :singleton + + before_validation :force_singleton, on: :create + + validates :singleton, inclusion: { in: [ true ] }, uniqueness: true + validates :default_sandbox_repo_cache, inclusion: { in: Principal::SANDBOX_REPO_CACHE_VALUES } + validates :default_sandbox_observability_enabled, inclusion: { in: [ true, false ] } + validates :default_sandbox_api_server_enabled, inclusion: { in: [ true, false ] } + + def self.current + first || create!(singleton: true) + rescue ActiveRecord::RecordNotUnique + first + end + + def principal_defaults + { + sandbox_repo_cache: default_sandbox_repo_cache, + sandbox_observability_enabled: default_sandbox_observability_enabled, + sandbox_api_server_enabled: default_sandbox_api_server_enabled + } + end + + private + + def force_singleton + self.singleton = true + end +end diff --git a/services/console/app/models/user.rb b/services/console/app/models/user.rb index 83e8c31be..82dd0aa57 100644 --- a/services/console/app/models/user.rb +++ b/services/console/app/models/user.rb @@ -7,11 +7,16 @@ class User < ApplicationRecord has_secure_password validations: false has_many :api_keys, dependent: :destroy + has_many :mcp_oauth_refresh_tokens, dependent: :destroy has_many :user_identities, dependent: :destroy belongs_to :approved_by, class_name: "User", optional: true - # pending: signed in via SSO but not yet approved -- cannot use the console. - # active: approved operator. disabled: access revoked. + after_update :revoke_mcp_oauth_refresh_tokens_when_disabled, + if: -> { saved_change_to_status? && disabled? } + + # active: normal operator (SSO users are provisioned active). pending: legacy + # state from the retired approval queue, flipped to active on next SSO login. + # disabled: access revoked. enum :status, { pending: "pending", active: "active", disabled: "disabled" }, default: :pending, validate: true @@ -28,27 +33,34 @@ def approve!(by:) update!(status: :active, approved_at: Time.current, approved_by: by) end + def revoke_mcp_oauth_refresh_tokens! + now = Time.current + mcp_oauth_refresh_tokens.usable.update_all(revoked_at: now, updated_at: now) + end + # Resolves the console user behind a verified SSO identity, creating or linking # as needed, and (re)caches the identity's email/name. A returning login matches # by the stable (provider, subject). A new identity links to an existing user # only when the IdP-verified email matches -- an unverified email must never - # adopt an account -- otherwise a new user is created: active + admin when the - # email is on the bootstrap allowlist, pending otherwise. +identity+ is the - # provider strategy's { subject:, email:, email_verified:, name: } hash. + # adopt an account -- otherwise a new active user is created (admin when the + # verified email is on the bootstrap allowlist). +identity+ is the provider + # strategy's { subject:, email:, email_verified:, name: } hash. def self.link_or_provision(provider:, identity:) transaction do - if (existing = UserIdentity.find_by(provider: provider, subject: identity[:subject])) - existing.update!(email: identity[:email], email_verified: identity[:email_verified]) - user = existing.user - user.update!(name: identity[:name]) if identity[:name].present? && user.name.blank? - next user - end - - user = linkable_user(identity) || create!(provisioned_attributes(identity)) - user.user_identities.create!( - provider: provider, subject: identity[:subject], - email: identity[:email], email_verified: identity[:email_verified] - ) + user = + if (existing = UserIdentity.find_by(provider: provider, subject: identity[:subject])) + existing.update!(identity_attributes(provider:, identity:)) + existing.user.tap do |u| + u.update!(name: identity[:name]) if identity[:name].present? && u.name.blank? + end + else + (linkable_user(identity) || create!(provisioned_attributes(identity))).tap do |u| + u.user_identities.create!( + identity_attributes(provider:, identity:).merge(provider:, subject: identity[:subject]) + ) + end + end + activate_on_login(user) user end end @@ -61,11 +73,37 @@ def self.linkable_user(identity) end private_class_method :linkable_user - # Attributes for a brand-new SSO user: active + admin when bootstrap-allowlisted - # by a verified IdP email, pending otherwise. + def self.identity_attributes(provider:, identity:) + attributes = { email: identity[:email], email_verified: identity[:email_verified] } + if provider == UserIdentity::SLACK_PROVIDER && identity[:team_id].present? + attributes[:team_id] = identity[:team_id] + end + attributes + end + private_class_method :identity_attributes + + # Attributes for a brand-new SSO user: everyone is provisioned active -- the + # console is only reachable on the internal network, so a completed SSO login + # is sufficient and there is no admin-approval queue. Admin additionally + # requires a bootstrap-allowlisted, IdP-verified email. def self.provisioned_attributes(identity) admin = identity[:email_verified] == true && ConsoleAuth.bootstrap_admin?(identity[:email]) - { email: identity[:email], name: identity[:name], status: admin ? :active : :pending, admin: admin } + { email: identity[:email], name: identity[:name], status: :active, admin: admin } end private_class_method :provisioned_attributes + + # Flips a pending user to active on login: covers accounts provisioned pending + # under the old approval-queue policy. Never touches disabled accounts and + # never grants admin. + def self.activate_on_login(user) + return unless user.pending? + user.update!(status: :active, approved_at: Time.current) + end + private_class_method :activate_on_login + + private + + def revoke_mcp_oauth_refresh_tokens_when_disabled + revoke_mcp_oauth_refresh_tokens! + end end diff --git a/services/console/app/models/user_identity.rb b/services/console/app/models/user_identity.rb index 624f139ed..2eff28faf 100644 --- a/services/console/app/models/user_identity.rb +++ b/services/console/app/models/user_identity.rb @@ -8,8 +8,10 @@ class UserIdentity < ApplicationRecord belongs_to :user PROVIDERS = %w[google slack].freeze + SLACK_PROVIDER = "slack".freeze normalizes :email, with: ->(e) { e.to_s.strip.downcase.presence } + normalizes :team_id, with: ->(id) { id.to_s.strip.presence } validates :provider, presence: true, inclusion: { in: PROVIDERS } validates :subject, presence: true, uniqueness: { scope: :provider } diff --git a/services/console/app/services/centaur_api_client.rb b/services/console/app/services/centaur_api_client.rb index ee280a4a8..aac4761a1 100644 --- a/services/console/app/services/centaur_api_client.rb +++ b/services/console/app/services/centaur_api_client.rb @@ -69,6 +69,47 @@ def ingest_google_docs_sync_batch(payload) post("/api/admin/google/docs-sync/batch", payload) end + def create_session(thread_key:, harness_type:, metadata: {}, persona_id: nil, + on_harness_conflict: "reject") + payload = { + harness_type: harness_type, + metadata: metadata, + on_harness_conflict: on_harness_conflict + } + payload[:persona_id] = persona_id if persona_id.present? + + post("/api/session/#{escape_path(thread_key)}", payload) + end + + def append_session_messages(thread_key:, messages:) + post("/api/session/#{escape_path(thread_key)}/messages", { messages: messages }) + end + + def execute_session(thread_key:, input_lines:, idempotency_key: nil, metadata: {}) + payload = { + input_lines: input_lines, + metadata: metadata + } + payload[:idempotency_key] = idempotency_key if idempotency_key.present? + + post("/api/session/#{escape_path(thread_key)}/execute", payload) + end + + def list_workflow_schedules + get("/api/workflows/schedules") + end + + def get_workflow_run(run_id) + get("/api/workflows/runs/#{escape_path(run_id)}") + end + + def create_workflow_run(workflow_name:, input: nil) + payload = { workflow_name: workflow_name } + payload[:input] = input unless input.nil? + + post("/api/workflows/runs", payload) + end + private def get(path, params = {}) diff --git a/services/console/app/services/google_docs/sync_credential.rb b/services/console/app/services/google_docs/sync_credential.rb index adff08df0..5f2ffc600 100644 --- a/services/console/app/services/google_docs/sync_credential.rb +++ b/services/console/app/services/google_docs/sync_credential.rb @@ -154,7 +154,7 @@ def files_list_params(modified_after:, page_token:) "q" => query.join(" and "), "pageSize" => self.class.page_size, "fields" => [ - "nextPageToken", + "nextPageToken,", "files(id,name,mimeType,webViewLink,driveId,owners,lastModifyingUser,", "capabilities,labelInfo,trashed,explicitlyTrashed,createdTime,modifiedTime,version)" ].join, diff --git a/services/console/app/services/principal_credential_reconciliation.rb b/services/console/app/services/principal_credential_reconciliation.rb index 9da05f7a6..1a9334f5d 100644 --- a/services/console/app/services/principal_credential_reconciliation.rb +++ b/services/console/app/services/principal_credential_reconciliation.rb @@ -1,17 +1,19 @@ -# Finds Slack/Google OAuth-flow credentials that appear to belong to the same -# human as an existing user principal, then automatically grants their wrapper -# static secrets to that principal. +# Finds OAuth-flow credentials (Slack/Google/GitHub/...) that appear to belong +# to the same human as an existing user principal, then automatically grants +# their wrapper static secrets to that principal. class PrincipalCredentialReconciliation Entry = Struct.new( :principal, - :slack_credentials, - :google_credentials, - :slack_grants, - :google_grants, + :credentials_by_provider, + :granted_by_credential_id, keyword_init: true ) do def credentials - slack_credentials + google_credentials + credentials_by_provider.values.flatten + end + + def credentials_for(provider) + credentials_by_provider[provider] || [] end def actionable_credentials @@ -19,38 +21,36 @@ def actionable_credentials end def granted?(credential) - slack_grants[credential.id] || google_grants[credential.id] || false + granted_by_credential_id[credential.id] || false end end USER_KIND = "user" + # Minted by the MCP OAuth flow (Mcp::OauthController#principal_for_current_user) + # for a console user connecting an MCP client. These principals match + # credentials only through their console User record (primary email plus + # verified identity emails) -- never through mutable principal labels or + # provider-subject labels, which would widen the trust boundary beyond the + # authenticated user. + CONSOLE_USER_KIND = "console_user" + CONSOLE_USER_ID_LABEL = "console-user-id" SLACK_PROVIDER = Oauth::Providers::Slack::KEY GOOGLE_PROVIDER = Oauth::Providers::Google::KEY EMAIL_LABELS = %w[email google_email slack_email].freeze - SLACK_USER_LABELS = %w[slack_user_id].freeze - GOOGLE_SUBJECT_LABELS = %w[google_subject].freeze + # Principal labels carrying a provider-native identity. When a principal has + # one for a provider, it takes precedence over email matching for that + # provider's credentials. Providers without an entry (for example github) + # match by email only. PROVIDER_SUBJECT_LABELS = { - SLACK_PROVIDER => SLACK_USER_LABELS, - GOOGLE_PROVIDER => GOOGLE_SUBJECT_LABELS + SLACK_PROVIDER => %w[slack_user_id], + GOOGLE_PROVIDER => %w[google_subject] }.freeze SLACK_TEAM_LABEL = "slack_team_id" def entries - slack = provider_credentials(SLACK_PROVIDER) - google = provider_credentials(GOOGLE_PROVIDER) - slack_by_subject = credentials_by_subject(slack) - google_by_subject = credentials_by_subject(google) - slack_by_email = credentials_by_email(slack) - google_by_email = credentials_by_email(google) - + indexes = credential_indexes user_principals.select { |principal| user_principal?(principal) }.filter_map do |principal| - entry_for( - principal, - slack_by_subject: slack_by_subject, - slack_by_email: slack_by_email, - google_by_subject: google_by_subject, - google_by_email: google_by_email - ) + entry_for(principal, indexes: indexes) end.sort_by do |entry| [ entry.principal.namespace, entry.principal.name.to_s, entry.principal.foreign_id.to_s ] end @@ -89,6 +89,13 @@ def apply_all private + # Every registered OAuth-flow provider participates: a provider without + # subject labels still reconciles by email, so new registry entries get + # matching for free. + def providers + Oauth::Providers.keys + end + def apply_entry(entry) return { requested: 0, created: 0 } unless entry @@ -96,11 +103,16 @@ def apply_entry(entry) created = entry.actionable_credentials.count do |credential| grant_credential(entry.principal, credential) end - sync_principal_provider_labels(entry.principal, entry.google_credentials) + sync_principal_provider_labels(entry.principal, entry.credentials) { requested: requested, created: created } end def sync_principal_provider_labels(principal, credentials) + if console_user_principal?(principal) + sync_console_user_slack_labels(principal, credentials) + return + end + google_credentials = credentials.select do |credential| credential.oauth_app&.provider == GOOGLE_PROVIDER end @@ -129,49 +141,48 @@ def grant_credential(principal, credential) false end - def entry_for( - principal, - slack_by_subject: nil, - slack_by_email: nil, - google_by_subject: nil, - google_by_email: nil - ) + def entry_for(principal, indexes: nil) return nil unless user_principal?(principal) - slack_by_subject ||= credentials_by_subject(provider_credentials(SLACK_PROVIDER)) - slack_by_email ||= credentials_by_email(provider_credentials(SLACK_PROVIDER)) - google_by_subject ||= credentials_by_subject(provider_credentials(GOOGLE_PROVIDER)) - google_by_email ||= credentials_by_email(provider_credentials(GOOGLE_PROVIDER)) - + indexes ||= credential_indexes emails = principal_emails(principal) - slack_credentials = provider_credentials_for( - principal, - subject_label_keys: SLACK_USER_LABELS, - credentials_by_subject: slack_by_subject, - credentials_by_email: slack_by_email, - emails: emails, - provider: SLACK_PROVIDER - ) - google_credentials = provider_credentials_for( - principal, - subject_label_keys: GOOGLE_SUBJECT_LABELS, - credentials_by_subject: google_by_subject, - credentials_by_email: google_by_email, - emails: emails, - provider: GOOGLE_PROVIDER - ) - - return nil if slack_credentials.empty? && google_credentials.empty? + credentials_by_provider = providers.each_with_object({}) do |provider, acc| + matched = provider_credentials_for( + principal, + provider: provider, + subject_index: indexes[provider][:subjects], + email_index: indexes[provider][:emails], + emails: emails + ) + acc[provider] = matched if matched.any? + end + return nil if credentials_by_provider.empty? Entry.new( principal: principal, - slack_credentials: slack_credentials, - google_credentials: google_credentials, - slack_grants: grant_status(principal, slack_credentials), - google_grants: grant_status(principal, google_credentials) + credentials_by_provider: credentials_by_provider, + granted_by_credential_id: grant_status(principal, credentials_by_provider.values.flatten) ) end + # TODO(perf): this loads every oauth-flow credential in the system -- O(C) + # rows per Principal create/update, since apply_for_principal runs in an + # after_commit. Negligible while C is in the hundreds. Add the optimization + # when oauth-flow credential count reaches the low thousands or principal + # writes show up in latency traces, whichever comes first: replace the + # single-principal path with a candidate query (namespace-scoped + # `LOWER(provider_email) IN (...) OR provider_subject IN (...)`, backed by + # indexes on (namespace, LOWER(provider_email)) and (namespace, + # provider_subject)), which is O(K) in the credentials of the one matched + # human. Keep the SQL normalization identical to normalize_email / + # normalize_key. entries/apply_all legitimately need the full load. + def credential_indexes + providers.index_with do |provider| + credentials = provider_credentials(provider) + { subjects: index_by_subject(credentials), emails: index_by_email(credentials) } + end + end + def provider_credentials(provider) BrokerCredential .joins(:oauth_app) @@ -187,57 +198,48 @@ def user_principals def user_principal?(principal) labels = principal.labels || {} - labels["kind"] == USER_KIND || - (EMAIL_LABELS + SLACK_USER_LABELS + GOOGLE_SUBJECT_LABELS).any? do |key| - labels[key].present? - end + return true if [ USER_KIND, CONSOLE_USER_KIND ].include?(labels["kind"]) + + (EMAIL_LABELS + PROVIDER_SUBJECT_LABELS.values.flatten).any? do |key| + labels[key].present? + end end - def credentials_by_subject(credentials) + def index_by_subject(credentials) credentials.each_with_object(Hash.new { |hash, key| hash[key] = [] }) do |credential, acc| subject = normalize_key(credential.provider_subject) acc[subject] << credential if subject end end - def credentials_by_email(credentials) + def index_by_email(credentials) credentials.each_with_object(Hash.new { |hash, key| hash[key] = [] }) do |credential, acc| email = normalize_email(credential.provider_email) acc[email] << credential if email end end - def provider_credentials_for( - principal, - subject_label_keys:, - credentials_by_subject:, - credentials_by_email:, - emails:, - provider: - ) - native = credentials_for_subject_labels( - principal, - subject_label_keys, - credentials_by_subject, - provider - ) + def provider_credentials_for(principal, provider:, subject_index:, email_index:, emails:) + native = credentials_for_subject_labels(principal, provider, subject_index) return native if native.any? - credentials_for_emails(principal, emails, credentials_by_email, provider) + credentials_for_emails(principal, emails, email_index, provider) end - def credentials_for_subject_labels(principal, label_keys, credentials_by_subject, provider) + def credentials_for_subject_labels(principal, provider, subject_index) + return [] if console_user_principal?(principal) + labels = principal.labels || {} - subjects = label_keys.filter_map { |key| normalize_key(labels[key]) }.uniq + subjects = subject_label_keys(provider).filter_map { |key| normalize_key(labels[key]) }.uniq subjects - .flat_map { |subject| credentials_by_subject[subject] || [] } + .flat_map { |subject| subject_index[subject] || [] } .select { |credential| credential_matches_principal?(principal, credential, provider) } .uniq end - def credentials_for_emails(principal, emails, credentials_by_email, provider) + def credentials_for_emails(principal, emails, email_index, provider) emails - .flat_map { |email| credentials_by_email[email] || [] } + .flat_map { |email| email_index[email] || [] } .select { |credential| credential_matches_principal?(principal, credential, provider) } .uniq end @@ -247,8 +249,11 @@ def credential_matches_principal?(principal, credential, provider = nil) return false unless supported_provider?(credential) return false unless credential.namespace == principal.namespace return false if provider == SLACK_PROVIDER && !slack_team_matches?(principal, credential) + if console_user_principal?(principal) + return principal_emails(principal).include?(normalize_email(credential.provider_email)) + end - subjects = PROVIDER_SUBJECT_LABELS.fetch(provider) + subjects = subject_label_keys(provider) .filter_map { |key| normalize_key(principal.labels&.[](key)) } .uniq if subjects.any? @@ -258,8 +263,12 @@ def credential_matches_principal?(principal, credential, provider = nil) end end + def subject_label_keys(provider) + PROVIDER_SUBJECT_LABELS.fetch(provider, []) + end + def supported_provider?(credential) - PROVIDER_SUBJECT_LABELS.key?(credential.oauth_app&.provider) + providers.include?(credential.oauth_app&.provider) end # Slack user ids are workspace-scoped. If either side carries a team label, @@ -269,18 +278,73 @@ def slack_team_matches?(principal, credential) principal_team = normalize_key(principal.labels&.[](SLACK_TEAM_LABEL)) credential_team = normalize_key(credential.labels&.[](SLACK_TEAM_LABEL)) || normalize_key(credential.oauth_app&.labels&.[](SLACK_TEAM_LABEL)) + return true if console_user_principal?(principal) && principal_team.blank? return true if principal_team.blank? && credential_team.blank? principal_team.present? && principal_team == credential_team end + def sync_console_user_slack_labels(principal, credentials) + slack_credentials = credentials.select do |credential| + credential.oauth_app&.provider == SLACK_PROVIDER + end + return if slack_credentials.empty? + + slack_user_id = unique_present_value(slack_credentials.map(&:provider_subject)) + slack_team_id = unique_present_value(slack_credentials.map { |credential| slack_team_for(credential) }) + return unless slack_user_id && slack_team_id + + labels = principal.labels || {} + updates = { + "slack_user_id" => slack_user_id, + SLACK_TEAM_LABEL => slack_team_id + } + return if updates.all? { |key, value| labels[key] == value } + + principal.update!(labels: labels.merge(updates)) + end + + def slack_team_for(credential) + credential.labels&.[](SLACK_TEAM_LABEL).presence || + credential.oauth_app&.labels&.[](SLACK_TEAM_LABEL).presence + end + + def console_user_principal?(principal) + (principal.labels || {})["kind"] == CONSOLE_USER_KIND + end + def principal_emails(principal) + if console_user_principal?(principal) + return console_user_emails(principal).filter_map { |email| normalize_email(email) }.uniq + end + labels = principal.labels || {} EMAIL_LABELS.map { |key| labels[key] } .filter_map { |email| normalize_email(email) } .uniq end + # Console-user principals carry the console user's oid, so every verified + # identity email of that user participates in matching -- a credential + # registered under a secondary verified email still reaches the principal. + # Unverified emails are excluded: an unverified address must not adopt + # someone else's credentials. + def console_user_emails(principal) + user_oid = principal.labels&.[](CONSOLE_USER_ID_LABEL) + return [] if user_oid.blank? + + @console_user_emails ||= {} + @console_user_emails.fetch(user_oid) do + user = User.find_by_oid(user_oid) + emails = if user + [ user.email ] + user.user_identities.where(email_verified: true).pluck(:email) + else + [] + end + @console_user_emails[user_oid] = emails + end + end + def grant_status(principal, credentials) secret_ids = credentials.filter_map { |credential| credential.static_secret&.id } granted_secret_ids = if secret_ids.empty? diff --git a/services/console/app/services/slack_channel_catalog.rb b/services/console/app/services/slack_channel_catalog.rb new file mode 100644 index 000000000..353836a68 --- /dev/null +++ b/services/console/app/services/slack_channel_catalog.rb @@ -0,0 +1,131 @@ +require "digest" +require "json" +require "net/http" +require "uri" + +class SlackChannelCatalog + Channel = Data.define(:id, :name, :private) + Result = Data.define(:channels, :error, :configured) do + def ok? + error.blank? + end + end + + DEFAULT_API_URL = "https://slack.com/api".freeze + DEFAULT_TYPES = "public_channel,private_channel".freeze + CACHE_TTL = 5.minutes + OPEN_TIMEOUT_SECONDS = 2 + READ_TIMEOUT_SECONDS = 5 + WRITE_TIMEOUT_SECONDS = 2 + + def self.fetch + token = ENV["CENTAUR_CONSOLE_SLACK_BOT_TOKEN"].presence || ENV["SLACK_BOT_TOKEN"].presence + return Result.new(channels: [], error: "SLACK_BOT_TOKEN is not configured.", configured: false) if token.blank? + + api_url = ENV["SLACK_API_URL"].presence || DEFAULT_API_URL + key = cache_key(token: token, api_url: api_url) + cached = Rails.cache.read(key) + return deserialize_result(cached) if cached + + result = new(token: token, api_url: api_url).fetch + Rails.cache.write(key, serialize_result(result), expires_in: CACHE_TTL) if result.ok? + result + end + + def self.cache_key(token:, api_url:) + token_digest = Digest::SHA256.hexdigest(token) + api_url_digest = Digest::SHA256.hexdigest(api_url) + "slack_channel_catalog/v1/#{api_url_digest}/#{token_digest}" + end + + def self.serialize_result(result) + { + "channels" => result.channels.map do |channel| + { "id" => channel.id, "name" => channel.name, "private" => channel.private } + end, + "error" => result.error, + "configured" => result.configured + } + end + + def self.deserialize_result(payload) + return payload if payload.is_a?(Result) + + channels = Array(payload["channels"]).map do |channel| + Channel.new( + id: channel.fetch("id"), + name: channel.fetch("name"), + private: channel.fetch("private") + ) + end + Result.new(channels: channels, error: payload["error"], configured: payload["configured"]) + end + + def initialize(token:, api_url:) + @token = token + @api_url = api_url.to_s.delete_suffix("/") + end + + def fetch + channels = [] + cursor = nil + loop do + body = request_page(cursor) + return Result.new(channels: [], error: body.fetch("error", "Slack API request failed."), configured: true) unless body["ok"] + + channels.concat(Array(body["channels"]).filter_map { |channel| parse_channel(channel) }) + cursor = body.dig("response_metadata", "next_cursor").to_s + break if cursor.blank? + end + + Result.new( + channels: channels.sort_by { |channel| [ channel.name.downcase, channel.id ] }, + error: nil, + configured: true + ) + rescue JSON::ParserError + Result.new(channels: [], error: "Slack API response was not JSON.", configured: true) + rescue StandardError => e + Result.new(channels: [], error: "Slack API request failed: #{e.message}", configured: true) + end + + private + + def request_page(cursor) + uri = URI("#{@api_url}/conversations.list") + params = { + types: DEFAULT_TYPES, + exclude_archived: "true", + limit: "1000" + } + params[:cursor] = cursor if cursor.present? + uri.query = URI.encode_www_form(params) + + request = Net::HTTP::Get.new(uri) + request["Authorization"] = "Bearer #{@token}" + request["Accept"] = "application/json" + + response = Net::HTTP.start( + uri.host, + uri.port, + use_ssl: uri.scheme == "https", + open_timeout: OPEN_TIMEOUT_SECONDS, + read_timeout: READ_TIMEOUT_SECONDS, + write_timeout: WRITE_TIMEOUT_SECONDS + ) do |http| + http.request(request) + end + return { "ok" => false, "error" => "HTTP #{response.code}" } unless response.code.to_i.between?(200, 299) + + JSON.parse(response.body) + end + + def parse_channel(channel) + return nil unless channel.is_a?(Hash) + id = channel["id"].to_s + name = channel["name"].to_s + return nil if id.blank? || name.blank? + + Channel.new(id: id, name: name, private: channel["is_private"] == true) + end +end diff --git a/services/console/app/views/console/_control_tabs.html.erb b/services/console/app/views/console/_control_tabs.html.erb new file mode 100644 index 000000000..867fb8c29 --- /dev/null +++ b/services/console/app/views/console/_control_tabs.html.erb @@ -0,0 +1,20 @@ +<% control_tabs = [ + { label: "Principals", path: console_principals_path, match: "/console/principals", root_active: true }, + { label: "Roles", path: console_roles_path, match: "/console/roles" }, + { label: "Secrets", path: console_secrets_path, match: "/console/secrets" }, + { label: "Credentials", path: console_credentials_path, match: "/console/credentials" }, + { label: "Apps", path: console_oauth_apps_path, match: "/console/oauth_apps" } +] %> +<% control_tabs << { label: "Users", path: console_users_path, match: "/console/users" } if acting_admin? %> +<% control_tabs << { label: "Settings", path: edit_console_system_settings_path, match: "/console/settings" } if acting_admin? %> + + diff --git a/services/console/app/views/console/_page_header.html.erb b/services/console/app/views/console/_page_header.html.erb new file mode 100644 index 000000000..33bb1e06a --- /dev/null +++ b/services/console/app/views/console/_page_header.html.erb @@ -0,0 +1,29 @@ +<% title = local_assigns.fetch(:title) %> +<% subtitle = local_assigns[:subtitle] %> +<% actions = local_assigns[:actions] %> +<% icon = local_assigns[:icon] %> +<% title_class = local_assigns[:title_class].presence || "page-title" %> +<% subtitle_class = local_assigns[:subtitle_class].presence || "page-subtitle" %> +<% header_class = [ "console-page-header", local_assigns[:class] ].compact.join(" ") %> + +
+
+ <% if icon.present? %> + + <%= console_icon(icon, classes: "size-4") %> + + <% end %> +
+

<%= title %>

+ <% if subtitle.present? %> +
<%= subtitle %>
+ <% end %> +
+
+ + <% if actions.present? %> +
+ <%= actions %> +
+ <% end %> +
diff --git a/services/console/app/views/console/base_secrets/edit.html.erb b/services/console/app/views/console/base_secrets/edit.html.erb index 451e0b5de..050808fd2 100644 --- a/services/console/app/views/console/base_secrets/edit.html.erb +++ b/services/console/app/views/console/base_secrets/edit.html.erb @@ -10,7 +10,7 @@ <% if (cred = managed_credential(@secret)) %> -
+
Managed secret. Maintained by the <% if cred.oauth_app %><%= cred.oauth_app.slug %> <% end %>OAuth integration (broker credential <%= cred.oid %>). diff --git a/services/console/app/views/console/credentials.html.erb b/services/console/app/views/console/credentials.html.erb index c86df06f9..91998686d 100644 --- a/services/console/app/views/console/credentials.html.erb +++ b/services/console/app/views/console/credentials.html.erb @@ -1,12 +1,11 @@ <% content_for :title, "Credentials · Centaur Console" %> -
-
-

Managed Credentials

-

<%= pluralize(@credentials.size, "broker credential") %>. Token material is never shown.

-
- + Add Credential -
+<%= render "control_tabs" %> + +<%= render "console/page_header", + title: "Managed Credentials", + subtitle: "#{pluralize(@credentials.size, "broker credential")}. Token material is never shown.", + actions: link_to("+ Add Credential", new_console_broker_credential_path, class: "btn-primary") %> <% if @credentials.empty? %>

No managed credentials.

diff --git a/services/console/app/views/console/etls/index.html.erb b/services/console/app/views/console/etls/index.html.erb index 5f27f9aff..89625ea0e 100644 --- a/services/console/app/views/console/etls/index.html.erb +++ b/services/console/app/views/console/etls/index.html.erb @@ -1,15 +1,16 @@ -<% content_for :title, "ETLs · Centaur Console" %> +<% content_for :title, "Data Sync · Centaur Console" %> -
-
-

ETLs

-

Upload Slack public-channel archive exports, start imports, and track ingestion status.

-
-
+<% sync_meta = capture do %> +
<%= pluralize(@archive_imports.size, "archive import") %> shown
-
+<% end %> + +<%= render "console/page_header", + title: "Data Sync", + subtitle: "Upload Slack public-channel archive exports, start imports, and track ingestion status.", + actions: sync_meta %>
+ +<%= render "console/page_header", + title: "Integrations", + subtitle: "Connect your accounts so Centaur can act on your behalf." %> + +<% if @oauth_apps.empty? %> +

No integrations available yet.

+<% else %> +
+ <% @oauth_apps.each do |app| %> + <% start_url = "#{public_base_url}/oauth/#{app.slug}/start" %> +
+
+ <% logo = oauth_provider_logo(app.provider, classes: "size-8 shrink-0") %> + <% if logo %> + <%= logo %> + <% else %> + <%= app.provider.first(2) %> + <% end %> +
+
<%= app.slug %>
+
<%= app.provider %>
+
+
+ <% if app.description.present? %> +

<%= app.description %>

+ <% end %> +
+ <% credential = @credentials_by_app_id[app.id] %> + <% if credential %> + Reconnect + "> + "> + <%= credential.dead? ? "Needs reconnecting" : "Connected" %> + + <% else %> + Connect + <% end %> +
+
+ <% end %> +
+

Connecting opens the provider's consent screen. Once you approve, Centaur stores and refreshes the credential automatically.

+<% end %> diff --git a/services/console/app/views/console/oauth_app.html.erb b/services/console/app/views/console/oauth_app.html.erb index 79d03016b..1357de653 100644 --- a/services/console/app/views/console/oauth_app.html.erb +++ b/services/console/app/views/console/oauth_app.html.erb @@ -1,7 +1,7 @@ <% content_for :title, "#{@oauth_app.slug} · Centaur Console" %>
- ← back to OAuth apps + ← back to Apps

<%= @oauth_app.slug %>

<% if @oauth_app.enabled? %> diff --git a/services/console/app/views/console/oauth_apps.html.erb b/services/console/app/views/console/oauth_apps.html.erb index 34cf3fda7..c30f757a9 100644 --- a/services/console/app/views/console/oauth_apps.html.erb +++ b/services/console/app/views/console/oauth_apps.html.erb @@ -1,15 +1,14 @@ -<% content_for :title, "OAuth Apps · Centaur Console" %> +<% content_for :title, "Apps · Centaur Console" %> -
-
-

OAuth Apps

-

<%= pluralize(@oauth_apps.size, "OAuth app") %> driving the public consent flow.

-
- + Add OAuth App -
+<%= render "console/control_tabs" %> + +<%= render "console/page_header", + title: "Apps", + subtitle: "#{pluralize(@oauth_apps.size, "OAuth app")} driving the public consent flow.", + actions: link_to("+ Add App", new_console_oauth_app_form_path, class: "btn-primary") %> <% if @oauth_apps.empty? %> -

No OAuth apps.

+

No apps.

<% else %>
diff --git a/services/console/app/views/console/oauth_apps/edit.html.erb b/services/console/app/views/console/oauth_apps/edit.html.erb index 7c835daef..a61872eb6 100644 --- a/services/console/app/views/console/oauth_apps/edit.html.erb +++ b/services/console/app/views/console/oauth_apps/edit.html.erb @@ -2,7 +2,7 @@ <% content_for :title, "Edit #{title} · Centaur Console" %>
- ← back to OAuth app + ← back to App

Edit <%= title %>

OAuth diff --git a/services/console/app/views/console/oauth_apps/new.html.erb b/services/console/app/views/console/oauth_apps/new.html.erb index 827fd9c03..a62d7666b 100644 --- a/services/console/app/views/console/oauth_apps/new.html.erb +++ b/services/console/app/views/console/oauth_apps/new.html.erb @@ -1,9 +1,9 @@ -<% content_for :title, "New OAuth app · Centaur Console" %> +<% content_for :title, "New App · Centaur Console" %>
- ← back to OAuth apps + ← back to Apps
-

New OAuth app

+

New App

OAuth
diff --git a/services/console/app/views/console/principal.html.erb b/services/console/app/views/console/principal.html.erb index f5eb311e1..52e902bfa 100644 --- a/services/console/app/views/console/principal.html.erb +++ b/services/console/app/views/console/principal.html.erb @@ -2,9 +2,16 @@
← back to principals -

- <%= @principal.name.presence || @principal.foreign_id.presence || "Principal" %> -

+
+

+ <%= @principal.name.presence || @principal.foreign_id.presence || "Principal" %> +

+
+ <%= button_to "Delete", console_delete_principal_path(@principal.oid), method: :delete, + class: "cursor-pointer rounded border border-red-500/40 px-3 py-1.5 text-sm text-red-300 transition-colors hover:border-red-500/60 hover:bg-red-500/10", + data: { turbo_confirm: "Delete this principal? Direct grants, role assignments, MCP tokens, and sync snapshots are removed. Proxies are unassigned. This cannot be undone." } %> +
+
<%= @principal.oid %> · @@ -23,6 +30,133 @@ <% end %>
+ +
+
+

Slack Channel Permissions

+
+
+ <% checkbox_class = "mt-1 h-4 w-4 rounded border-ink-500 bg-ink-800 text-centaur-500 focus:ring-centaur-500" %> + <% channel_options = @slack_channel_options || [] %> + <% permissions = @slack_channel_permissions || [] %> + <%= form_with model: @principal, + url: console_principal_slack_channel_permissions_path(@principal.oid), + method: :patch, + data: { turbo: false }, + class: "space-y-0" do |form| %> + <% if @slack_channel_catalog&.error.present? %> +
+ <%= @slack_channel_catalog.error %> +
+ <% end %> + + <% if permissions.any? %> +
+
+ + + + + + + + + + + <% permissions.each do |permission| %> + + <% if permission.channel_id.to_s.start_with?("D") %> + + + + + + <% else %> + <%= form.fields_for :slack_channel_permissions, permission do |permission_fields| %> + + + + + + <% end %> + <% end %> + + <% end %> + +
ChannelUploadDownloadHistoryRemove
+
DM <%= permission.channel_name.presence || permission.channel_id %>
+
<%= permission.channel_id %>
+ 		+ <%= check_box_tag nil, "1", permission.upload_enabled, disabled: true, class: checkbox_class, aria: { label: "Upload for #{permission.channel_id}" } %> + + <%= check_box_tag nil, "1", permission.download_enabled, disabled: true, class: checkbox_class, aria: { label: "Download for #{permission.channel_id}" } %> + + <%= check_box_tag nil, "1", permission.history_enabled, disabled: true, class: checkbox_class, aria: { label: "History for #{permission.channel_id}" } %> + + API-managed + + <%= permission_fields.hidden_field :id %> + <%= permission_fields.hidden_field :channel_name, value: nil %> + <% if channel_options.any? %> + <%= permission_fields.select :channel_id, + options_for_select(slack_channel_options_for_permission(permission, channel_options), permission.channel_id), + {}, + class: "form-input" %> + <% else %> + <%= permission_fields.text_field :channel_id, class: "form-input", placeholder: "C0123456789" %> + <% end %> + + <%= permission_fields.check_box :upload_enabled, class: checkbox_class, aria: { label: "Upload for #{permission.channel_id}" } %> + + <%= permission_fields.check_box :download_enabled, class: checkbox_class, aria: { label: "Download for #{permission.channel_id}" } %> + + <%= permission_fields.check_box :history_enabled, class: checkbox_class, aria: { label: "History for #{permission.channel_id}" } %> + + <%= permission_fields.check_box :_destroy, class: checkbox_class, aria: { label: "Remove #{permission.channel_id}" } %> +
+
+ <% else %> +
+
No Slack channels selected.
+
+ <% end %> + + <% new_permission = SlackChannelPermission.new(upload_enabled: true, download_enabled: true, history_enabled: true) %> + <%= form.fields_for :slack_channel_permissions, new_permission do |permission_fields| %> +
+ + + + +
+ <% end %> + +
+ <%= submit_tag "Save Slack channel permissions", class: "btn-primary" %> +
+ <% end %> +
+
+
@@ -34,16 +168,18 @@ method: :patch, data: { turbo: false }, class: "space-y-4" do %> -
diff --git a/services/console/app/views/console/workflows/show.html.erb b/services/console/app/views/console/workflows/show.html.erb new file mode 100644 index 000000000..b45335c6f --- /dev/null +++ b/services/console/app/views/console/workflows/show.html.erb @@ -0,0 +1,205 @@ +<% title = @latest_run&.workflow_name_label || @workflow_name %> +<% content_for :title, "#{title} · Centaur Console" %> + +<% if @workflow_db_unavailable %> +
+ ← back to workflows +
+
+ Workflow database is unavailable. Console needs the API workflow read-only views to show runs. +
+<% elsif @latest_run.blank? %> +
+ ← back to workflows +
+
+ No workflow runs found for <%= @workflow_name %>. +
+<% else %> + <% status = @latest_run.display_status %> + <% schedule = @workflow_schedules&.first %> + <% source_path = schedule&.dig("source_path").presence %> + <% source_url = workflow_source_url(source_path) %> +
+ ← back to workflows +
+

+ <%= truncate_middle(@latest_run.workflow_name_label, max: 72) %> +

+ + <%= status.tr("_", " ") %> + +
+ <%= button_to "Manually Trigger", run_console_workflow_path(@workflow_name), method: :post, + class: "btn-primary", + data: { turbo_confirm: "Manually trigger a #{@workflow_name} run?#{schedule ? " It starts with the registered schedule input." : ""}" } %> +
+
+
+ <%= @latest_run.task_name.presence || "workflow" %> + · + <%= pluralize(number_with_delimiter(@total_runs), "run") %> + <% if source_url %> + · + + <%= source_path %> ↗ + + <% end %> +
+
+ +
+

Overview

+ <% schedule_text = nil %> + <% if schedule %> + <% kind = schedule["kind"].is_a?(Hash) ? schedule["kind"] : {} %> + <% schedule_text = [ + workflow_schedule_label(schedule), + (schedule["timezone"].presence if kind["type"] == "cron"), + ("disabled" if schedule["enabled"] == false) + ].compact.join(" · ") %> + <% end %> +
+ <% [ + [ "Queue", @queue_names.map { |queue_name| CentaurWorkflowRun.queue_label_for(queue_name) }.join(", ").presence || @latest_run.queue_label ], + [ "Engine", workflow_engine_label(@latest_run.harness_type) ], + ([ "Schedule", schedule_text ] if schedule_text.present?), + [ "Started", local_time(@latest_run.started_or_created_at) ], + [ "Duration", workflow_duration_label(@latest_run) ], + [ "Attempts", safe_join([ (@latest_run.attempts || 0).to_s, tag.span(" / #{@latest_run.max_attempts.presence || "unlimited"}", class: "text-zinc-600") ]) ], + [ "Runs", pluralize(number_with_delimiter(@total_runs), "run") ] + ].compact.each do |label, value| %> +
+
<%= label %>
+
<%= value.presence || tag.span("—", class: "text-zinc-600") %>
+
+ <% end %> +
+
+ + <% debug_rows = [] %> + <% if @latest_run_detail.present? %> + <% input = @latest_run_detail["input"] %> + <% debug_rows << [ "Input", input, nil ] unless input.nil? || input == {} || input == "" %> + <% debug_rows << [ "Result", @latest_run_detail["result"], nil ] unless @latest_run_detail["result"].nil? %> + <% debug_rows << [ "Failure", @latest_run_detail["failure"], @latest_run_detail["run_id"] ] unless @latest_run_detail["failure"].nil? %> + <% end %> + <% if @latest_failure_detail.present? && @latest_failure_detail["failure"].present? %> + <% debug_rows << [ "Last failure", @latest_failure_detail["failure"], @latest_failure_detail["run_id"] ] %> + <% end %> + <% if debug_rows.any? %> +
+

Debugging

+
+ <% debug_rows.each do |label, value, run_id| %> +
+
"><%= label %>
+
+ <% if run_id.present? %> +
run <%= run_id %>
+ <% end %> + 
<%= workflow_debug_json(value) %>
+
+
+ <% end %> +
+
+ <% end %> + +

Historical Runs

+ + <%# Filter tabs. Changing a filter resets to page 1; the pager keeps filters. %> + <% filter_url = ->(status: @status, queue: @queue) { + query = { status: status, queue: queue }.compact + query.empty? ? request.path : "#{request.path}?#{query.to_query}" + } %> + <% status_order = %w[running pending sleeping completed failed cancelled] %> + <% status_tabs = (status_order & @status_counts.keys) + (@status_counts.keys - status_order).sort %> +
+ <%= link_to filter_url.call(status: nil), class: "chip #{'chip-on' if @status.blank?}" do %> + all <%= number_with_delimiter(@total_runs) %> + <% end %> + <% status_tabs.each do |status_tab| %> + <%= link_to filter_url.call(status: status_tab), class: "chip #{'chip-on' if @status == status_tab}" do %> + <%= status_tab.tr("_", " ") %> <%= number_with_delimiter(@status_counts[status_tab]) %> + <% end %> + <% end %> +
+ + <% if @queue_names.size > 1 %> +
+ Queue + <%= link_to "all", filter_url.call(queue: nil), class: "chip #{'chip-on' if @queue.blank?}" %> + <% @queue_names.each do |queue_name| %> + <%= link_to CentaurWorkflowRun.queue_label_for(queue_name), + filter_url.call(queue: queue_name), + class: "chip #{'chip-on' if @queue == queue_name}" %> + <% end %> +
+ <% end %> + +
+ + + + + + + + + + + + + <% if @workflow_runs.empty? %> + + + + <% end %> + + <% @workflow_runs.each do |run| %> + <% run_status = run.display_status %> + + + + + + + + + + + + + + <% end %> + +
StatusQueueAttemptsStartedFinishedRun
+ No runs match the selected filters. +
+ + <%= run_status.tr("_", " ") %> + + + <%= run.queue_label %> + + <%= run.attempts || 0 %> / <%= run.max_attempts.presence || "unlimited" %> + + <%= local_time(run.started_or_created_at, relative: true) %> + <% if run.created_at.present? && run.started_or_created_at != run.created_at %> +
queued <%= local_time(run.created_at, relative: true, format: :compact) %>
+ <% end %> + 		+ <%= local_time(run.terminal_at, relative: true) %> + +
<%= truncate_middle(run.run_id, max: 34) %>
+
<%= truncate_middle(run.task_id, max: 34) %>
+
+ + <%= render "pagination", + page: @page, + total_pages: @total_pages, + total_count: @filtered_count.to_i, + unit: "run" %> +
+<% end %> diff --git a/services/console/app/views/layouts/application.html.erb b/services/console/app/views/layouts/application.html.erb index 916b8f096..24e960ecc 100644 --- a/services/console/app/views/layouts/application.html.erb +++ b/services/console/app/views/layouts/application.html.erb @@ -11,10 +11,11 @@ <%= yield :head %> - <%# Enable PWA manifest for installable apps (make sure to enable in config/routes.rb too!) %> - <%#= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> + <%= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> + + <%# Includes all stylesheet files in app/assets/stylesheets %> <%= stylesheet_link_tag :app, "data-turbo-track": "reload" %> diff --git a/services/console/app/views/layouts/console.html.erb b/services/console/app/views/layouts/console.html.erb index 84a35f76b..b8b61cc01 100644 --- a/services/console/app/views/layouts/console.html.erb +++ b/services/console/app/views/layouts/console.html.erb @@ -5,7 +5,74 @@ <%= csrf_meta_tags %> - + <%= tag.link rel: "manifest", href: pwa_manifest_path(format: :json) %> + + + + <%# Tailwind is compiled by the standalone binary into app/assets/builds/tailwind.css. See config/tailwind.config.js for the centaur/ink palette and radii overrides. %> @@ -18,64 +85,1911 @@ - -
-
-
-
- - <%= image_tag "centaur-lockup-white.svg", alt: "Centaur", class: "h-7 w-auto glow", width: 497, height: 127 %> + <% threads_view = request.path.start_with?("/console/threads") %> + <% workflows_view = request.path.start_with?("/console/workflows") %> + <%# The Control and Data Sync sections are admin-only (each controller enforces + require_admin server-side); Integrations is for everyone -- it lists the + public consent start links any team member can use. %> + <% nav_items = [] %> + <% if acting_admin? %> + <% control_matches = [ "/console/principals", "/console/roles", "/console/secrets", "/console/credentials", "/console/oauth_apps", "/console/users" ] %> + <% nav_items = [ + { label: "Control", icon: "shield-check", path: console_principals_path, matches: control_matches, root_active: true }, + { label: "Data Sync", icon: "database", path: console_etls_path, matches: [ "/console/etls" ] } + ] %> + <% end %> + <% nav_items << { label: "Integrations", icon: "link", path: console_integrations_path, matches: [ "/console/integrations" ] } %> + + "> + <% if descoped? %> +
+ Admin permissions paused — viewing the console as an operator + <%= button_to "Restore admin", console_descope_path, method: :delete, + class: "console-descope-restore" %> +
+ <% end %> +
+
-
- <% if flash[:notice] %> -
<%= flash[:notice] %>
- <% end %> - <% if flash[:alert] %> -
<%= flash[:alert] %>
+ <% if current_user %> + <% end %> - <%= yield %> -
+ +
"> +
"> + <% if flash[:notice] %> +
<%= flash[:notice] %>
+ <% end %> + <% if flash[:alert] %> +
<%= flash[:alert] %>
+ <% end %> + <%= yield %> +
+
+ + diff --git a/services/console/app/views/mcp/oauth/authorize.html.erb b/services/console/app/views/mcp/oauth/authorize.html.erb new file mode 100644 index 000000000..0a1a4cef1 --- /dev/null +++ b/services/console/app/views/mcp/oauth/authorize.html.erb @@ -0,0 +1,49 @@ +<% content_for :title, "Authorize MCP Client · Centaur Console" %> + +
+ <%= image_tag "centaur-lockup-white.svg", alt: "Centaur", class: "mx-auto h-9 w-auto", width: 497, height: 127 %> +

Authorize MCP access.

+
+ +
+
+
+

<%= @client.name %>

+

<%= @redirect_host %>

+
+ +
+
+
Resource
+
<%= @resource %>
+
+
+
Scope
+
<%= @scopes.join(" ") %>
+
+
+
Signed in as
+
<%= current_user.email %>
+
+
+
+ + <%= form_with url: "/mcp/oauth/authorize", method: :post, data: { turbo: false }, class: "mt-6" do %> + <% @authorization_params.each do |key, value| %> + <%= hidden_field_tag key, value %> + <% end %> + +
+ <%= button_tag "Deny", + type: "submit", + name: "decision", + value: "deny", + class: "cursor-pointer rounded border border-ink-600 bg-ink-800/60 px-4 py-2 text-sm text-zinc-300 transition-colors hover:border-zinc-500 hover:text-zinc-100" %> + <%= button_tag "Allow", + type: "submit", + name: "decision", + value: "approve", + class: "cursor-pointer rounded border border-centaur-500/40 bg-centaur-500/10 px-4 py-2 text-sm text-centaur-300 transition-colors hover:bg-centaur-500/20 hover:text-centaur-200" %> +
+ <% end %> +
diff --git a/services/console/app/views/oauth/flows/result.html.erb b/services/console/app/views/oauth/flows/result.html.erb index 35fb78c0c..53d5b5369 100644 --- a/services/console/app/views/oauth/flows/result.html.erb +++ b/services/console/app/views/oauth/flows/result.html.erb @@ -1,11 +1,9 @@ <% heading, accent = case @kind - when :success then [ "Connected", "text-emerald-300" ] - when :denied then [ "Not connected", "text-amber-300" ] - else [ "Something went wrong", "text-red-300" ] + when :denied then [ "Not connected", "text-amber-300" ] + else [ "Something went wrong", "text-red-300" ] end - app_name = @app&.slug %> <% content_for :title, "#{heading} · Centaur Console" %> @@ -15,19 +13,8 @@

<%= heading %>

- <% if @kind == :success %> -

- <%= app_name %> is connected<%= " as #{@identity[:email]}" if @identity && @identity[:email].present? %>. -

- <% if @credential %> -

Credential

-

<%= @credential.oid %>

- <% end %> -

Your access is being managed automatically. You can close this tab.

- <% else %> -

<%= @message %>

- <% if @app&.enabled? %> - Try again - <% end %> +

<%= @message %>

+ <% if @app&.enabled? %> + Try again <% end %>
diff --git a/services/console/app/views/pwa/manifest.json.erb b/services/console/app/views/pwa/manifest.json.erb index 9959147f7..e107dee41 100644 --- a/services/console/app/views/pwa/manifest.json.erb +++ b/services/console/app/views/pwa/manifest.json.erb @@ -1,16 +1,73 @@ { + "id": "/", "name": "Centaur Console", + "short_name": "Centaur", + "description": "Operator console for Centaur.", + "start_url": "/", + "scope": "/", + "display": "standalone", + "theme_color": "#050506", + "background_color": "#050506", "icons": [ { - "src": "/icon.svg", + "src": "/pwa-icon.svg", "type": "image/svg+xml", "sizes": "any" + }, + { + "src": "/pwa-icon-192.png", + "type": "image/png", + "sizes": "192x192" + }, + { + "src": "/pwa-icon-512.png", + "type": "image/png", + "sizes": "512x512" + }, + { + "src": "/pwa-icon-512.png", + "type": "image/png", + "sizes": "512x512", + "purpose": "maskable" } ], - "start_url": "/", - "display": "standalone", - "scope": "/", - "description": "Centaur Console.", - "theme_color": "#28c26a", - "background_color": "#050506" + "launch_handler": { + "client_mode": "navigate-existing" + }, + "shortcuts": [ + { + "name": "Chats", + "url": "/console/threads", + "icons": [{ "src": "/pwa-icon-192.png", "sizes": "192x192", "type": "image/png" }] + }, + { + "name": "Workflows", + "url": "/console/workflows", + "icons": [{ "src": "/pwa-icon-192.png", "sizes": "192x192", "type": "image/png" }] + }, + { + "name": "Integrations", + "url": "/console/integrations", + "icons": [{ "src": "/pwa-icon-192.png", "sizes": "192x192", "type": "image/png" }] + } + ], + "protocol_handlers": [ + { + "protocol": "web+centaur", + "url": "/launch?target=%s" + } + ], + "file_handlers": [ + { + "action": "/", + "accept": { + "application/json": [".json"], + "application/jsonl": [".jsonl", ".ndjson"], + "text/plain": [".txt", ".log"], + "text/markdown": [".md"], + "text/csv": [".csv"], + "application/x-yaml": [".yml", ".yaml"] + } + } + ] } diff --git a/services/console/app/views/pwa/service-worker.js b/services/console/app/views/pwa/service-worker.js index b3a13fb7b..483c5fc09 100644 --- a/services/console/app/views/pwa/service-worker.js +++ b/services/console/app/views/pwa/service-worker.js @@ -1,26 +1,78 @@ -// Add a service worker for processing Web Push notifications: +// Service worker for the installed Centaur Console PWA. // -// self.addEventListener("push", async (event) => { -// const { title, options } = await event.data.json() -// event.waitUntil(self.registration.showNotification(title, options)) -// }) -// -// self.addEventListener("notificationclick", function(event) { -// event.notification.close() -// event.waitUntil( -// clients.matchAll({ type: "window" }).then((clientList) => { -// for (let i = 0; i < clientList.length; i++) { -// let client = clientList[i] -// let clientPath = (new URL(client.url)).pathname -// -// if (clientPath == event.notification.data.path && "focus" in client) { -// return client.focus() -// } -// } -// -// if (clients.openWindow) { -// return clients.openWindow(event.notification.data.path) -// } -// }) -// ) -// }) +// Deliberately conservative: console pages are session-authenticated and +// server-rendered, so HTML is never cached. Navigations go straight to the +// network and only fall back to the offline page when the network itself is +// unreachable. Digested assets under /assets/ (propshaft) and the static PWA +// icons are safe to cache forever. + +const CACHE_VERSION = "centaur-console-v1"; +const OFFLINE_URL = "/offline.html"; +const PRECACHE_URLS = [OFFLINE_URL, "/pwa-icon.svg", "/pwa-icon-192.png", "/pwa-icon-512.png"]; + +self.addEventListener("install", (event) => { + event.waitUntil( + caches.open(CACHE_VERSION).then((cache) => cache.addAll(PRECACHE_URLS)).then(() => self.skipWaiting()) + ); +}); + +self.addEventListener("activate", (event) => { + event.waitUntil( + caches.keys() + .then((keys) => Promise.all(keys.filter((key) => key !== CACHE_VERSION).map((key) => caches.delete(key)))) + .then(() => self.clients.claim()) + ); +}); + +// Web Push: show whatever the server sent ({ title, options }); options.data.path +// tells notificationclick where to focus. The backend send path (VAPID keys, +// subscription storage) ships separately -- until then these never fire. +self.addEventListener("push", (event) => { + if (!event.data) return + const { title, options } = event.data.json() + event.waitUntil(self.registration.showNotification(title || "Centaur Console", options)) +}) + +self.addEventListener("notificationclick", (event) => { + event.notification.close() + const path = event.notification.data?.path || "/" + event.waitUntil( + clients.matchAll({ type: "window", includeUncontrolled: true }).then((clientList) => { + const existing = clientList.find((client) => new URL(client.url).pathname === path && "focus" in client) + if (existing) return existing.focus() + return clients.openWindow ? clients.openWindow(path) : undefined + }) + ) +}) + +self.addEventListener("fetch", (event) => { + const request = event.request; + if (request.method !== "GET") return; + + const url = new URL(request.url); + if (url.origin !== self.location.origin) return; + + // Navigations: network first, offline page as the last resort. Never served + // from cache, so login state and Turbo behavior are unchanged when online. + if (request.mode === "navigate") { + event.respondWith( + fetch(request).catch(() => caches.match(OFFLINE_URL, { cacheName: CACHE_VERSION })) + ); + return; + } + + // Digested assets and static icons: cache first, populate on miss. + const cacheable = url.pathname.startsWith("/assets/") || PRECACHE_URLS.includes(url.pathname); + if (!cacheable) return; + + event.respondWith( + caches.open(CACHE_VERSION).then(async (cache) => { + const cached = await cache.match(request); + if (cached) return cached; + + const response = await fetch(request); + if (response.ok) cache.put(request, response.clone()); + return response; + }) + ); +}); diff --git a/services/console/config/initializers/security_credential_validation.rb b/services/console/config/initializers/security_credential_validation.rb new file mode 100644 index 000000000..5afdc6d40 --- /dev/null +++ b/services/console/config/initializers/security_credential_validation.rb @@ -0,0 +1,5 @@ +# frozen_string_literal: true + +require Rails.root.join("lib/security_credential_validation") + +SecurityCredentialValidation.validate! diff --git a/services/console/config/routes.rb b/services/console/config/routes.rb index d10db590a..b8d27235f 100644 --- a/services/console/config/routes.rb +++ b/services/console/config/routes.rb @@ -5,9 +5,15 @@ # Can be used by load balancers and uptime monitors to verify that the app is live. get "up" => "rails/health#show", as: :rails_health_check - # Render dynamic PWA files from app/views/pwa/* (remember to link manifest in application.html.erb) - # get "manifest" => "rails/pwa#manifest", as: :pwa_manifest - # get "service-worker" => "rails/pwa#service_worker", as: :pwa_service_worker + # PWA manifest + service worker, rendered from app/views/pwa/*. Served by + # Rails::PwaController (framework controller, no console session required) so + # the browser can fetch them outside an authenticated page load. Both layouts + # link the manifest and register the worker. + get "manifest" => "rails/pwa#manifest", as: :pwa_manifest + get "service-worker" => "rails/pwa#service_worker", as: :pwa_service_worker + # Target of the manifest's web+centaur:// protocol handler: maps the + # custom-scheme URL in ?target= onto an in-app path and redirects. + get "launch", to: "launch#show", as: :launch # Operator console session login (cookie-based, separate from the API key auth). get "login", to: "sessions#new", as: :login @@ -22,10 +28,35 @@ get "auth/:provider/start", to: "session_oauth#start", as: :auth_start get "auth/:provider/callback", to: "session_oauth#callback", as: :auth_callback + # MCP OAuth authorization server. MCP clients discover this from api-rs' + # OAuth protected-resource metadata and register public PKCE clients here. + get ".well-known/oauth-authorization-server", to: "mcp/oauth#metadata" + get ".well-known/openid-configuration", to: "mcp/oauth#metadata" + post "mcp/oauth/register", to: "mcp/oauth#register" + get "mcp/oauth/authorize", to: "mcp/oauth#authorize" + post "mcp/oauth/authorize", to: "mcp/oauth#approve" + post "mcp/oauth/token", to: "mcp/oauth#token" + # Operator console (server-rendered HTML UI). root "console#principals" get "console/principals", to: "console#principals", as: :console_principals + namespace :console do + get "principals/new", to: "principals#new", as: :new_principal + post "principals", to: "principals#create", as: :create_principal + end get "console/principals/:id", to: "console#principal", as: :console_principal + namespace :console do + resources :threads, only: %i[index create] + resources :workflows, only: %i[index show] do + member do + post :run, action: :force_start + end + end + # Lazily-loaded sidebar thread list (Turbo Frame src). Kept off the main + # page render so the unindexed cross-database sessions query does not block + # every console page. See ApplicationController#load_console_sidebar_threads. + get "sidebar_threads", to: "threads#sidebar", as: :sidebar_threads + end namespace :console do resources :roles, only: %i[index show new create edit update] do member do @@ -38,7 +69,9 @@ # extra /roles and /grants path segments keep these clear of the show route above # and avoid clobbering the console_principal_path helper. namespace :console do + delete "principals/:id", to: "principals#destroy", as: :delete_principal patch "principals/:id/sandbox_access", to: "principals#update_sandbox_access", as: :principal_sandbox_access + patch "principals/:id/slack_channel_permissions", to: "principals#update_slack_channel_permissions", as: :principal_slack_channel_permissions post "principals/:id/roles", to: "principals#assign_role", as: :principal_assign_role delete "principals/:id/roles/:role_id", to: "principals#unassign_role", as: :principal_unassign_role post "principals/:id/grants", to: "principals#grant_secret", as: :principal_grant_secret @@ -64,6 +97,9 @@ end get "console/credentials/:id", to: "console#credential", as: :console_credential get "console/oauth_apps", to: "console#oauth_apps", as: :console_oauth_apps + # User-facing list of enabled OAuth apps and their consent start links. Not + # admin-gated: any signed-in team member connects integrations from here. + get "console/integrations", to: "console/integrations#index", as: :console_integrations get "console/etls", to: "console/etls#index", as: :console_etls namespace :console do post "etls/slack_archive_imports", @@ -97,6 +133,10 @@ post :promote end end + resource :system_settings, only: %i[edit update], path: "settings" + # Admin self-descope ("view as operator"): pause (admin-only) and restore + # admin permissions. A singular resource because it's a per-session flag. + resource :descope, only: %i[create destroy] end namespace :api do @@ -139,6 +179,7 @@ end member do get "effective_config" + post "slack_channel_permissions", action: :upsert_slack_channel_permission end # Role assignments for a principal. :id is the role's oid. resources :roles, only: %i[index create destroy], controller: :principal_roles diff --git a/services/console/config/tailwind.config.js b/services/console/config/tailwind.config.js index 353c0ea8d..9bcaafe3f 100644 --- a/services/console/config/tailwind.config.js +++ b/services/console/config/tailwind.config.js @@ -22,7 +22,16 @@ module.exports = { } }, fontFamily: { - mono: ['JetBrains Mono', 'ui-monospace', 'SFMono-Regular', 'Menlo', 'monospace'] + mono: [ + 'Berkeley Mono', + 'Berkeley Mono Variable', + 'BerkeleyMono', + 'JetBrains Mono', + 'ui-monospace', + 'SFMono-Regular', + 'Menlo', + 'monospace' + ] } }, // Very small radii everywhere for the sharp, terminal-ish look. diff --git a/services/console/db/migrate/.checksums.sha256 b/services/console/db/migrate/.checksums.sha256 new file mode 100644 index 000000000..f2185c0a2 --- /dev/null +++ b/services/console/db/migrate/.checksums.sha256 @@ -0,0 +1,58 @@ +06a3ee5f3df9cc99325741dc0bdd5c4a2bfd9a1314a0e3fcf713553c941f3acd services/console/db/migrate/20260527035805_create_principals.rb +4ba27f02d3b7ff38d2460eb96c67533977d688df0fa008bc036cbab3055bb945 services/console/db/migrate/20260527044342_create_secret_sources.rb +a78205749a1d72ac96fc64daa3f4572a79d70ea06451ef369c936a8c77347861 services/console/db/migrate/20260527044343_create_request_rules.rb +8f9cac9a30549e8ed8c898b9c956924e756bc9c2abd041e5c68ce37e918726cb services/console/db/migrate/20260527171520_create_static_secrets.rb +75497b5e6f57c11d26b7a96b96494716246df2d9e71b94b6ebaa51b85e3e90c4 services/console/db/migrate/20260527171532_add_static_secret_to_secret_sources.rb +1a3766e64fd90c0dc8f07453499cac77563283216da2fdd568dc5aea16e15cdd services/console/db/migrate/20260527171533_add_static_secret_to_request_rules.rb +8d9f835e1e91d281f47cd6941cb2091741a69c9e8b74dceb4cb4f97039c22893 services/console/db/migrate/20260527182421_create_grants.rb +1a8c5898704d9cf4274eafd3785fa733b8ccbeaaae84bc0453a86384c843eac5 services/console/db/migrate/20260527194107_create_proxies.rb +a419966656d3ede76e22fca929cc5d1f647e1416d2d9e36826d5e78d2443b0e5 services/console/db/migrate/20260527195243_create_users.rb +0341df756cbf384037647b8f7db7343a5038caacb8280b199f6921527e381ba2 services/console/db/migrate/20260527195244_create_api_keys.rb +5a60f4bbc9a07720cfd1ef1438fbc485622fddfea42053904fed5bbbe77c6840 services/console/db/migrate/20260527210955_add_name_to_principals_and_foreign_id_to_static_secrets.rb +c8635827c3b0d381b4383928a631178cfadd6680cb3f6578aa200a976c83f530 services/console/db/migrate/20260527211535_require_namespace_with_default.rb +85fe086f1f810ba2e94175554f25952c1a4b6469b88cae4c7cb2741df845a653 services/console/db/migrate/20260527220000_add_deleted_at_to_api_keys.rb +92df7aad0c0c62adc5de1ac5006e77d36970b8c9c235981c9146f7dd4f550007 services/console/db/migrate/20260527220001_add_created_by_to_api_resources.rb +c2fe8c092cf5a44acbc2717fc33f478ccd9aa1cedb43689490559a8dae72bf7a services/console/db/migrate/20260528111433_add_secret_to_secret_sources.rb +36cf54cdc95ce968422c4cd33cd14dc7015e8fecfa7ea7164770e15d97c826c0 services/console/db/migrate/20260601174019_create_gcp_auth_secrets.rb +498228cda8c216e3a13c9ce30e4aaf4b5befcf03c7a98a485dc773ffd02137ba services/console/db/migrate/20260601174020_create_oauth_token_secrets.rb +22a7cc9db3fd08751db47c88b2b20bd48fa9ea55c309554b3d1ae1b879b2ef7b services/console/db/migrate/20260601174023_add_credential_owners_to_secret_sources.rb +639fe7879ac9f673f337aa4782a1c5d126c42dcc3194f2d0435974bfd3ce6335 services/console/db/migrate/20260601174024_add_credential_owners_to_request_rules.rb +30511ae3b025f475c8ab96ce6362253ac53462fbb92d2101aeeee2a7766180c8 services/console/db/migrate/20260601174025_add_grantables_to_grants.rb +4d64f47cc332469d12e3cef283cd2bea0b7aca90226fbcec4af579d958380785 services/console/db/migrate/20260602041000_create_roles.rb +da70d039fd0e7d51c76bbfd9d24857ebcfb1abd46a35c995af5a6212b50bcdb4 services/console/db/migrate/20260602041005_create_principal_roles.rb +907574afe8721e5a224b31675cc18afdb12ebcb7d9870a0bcdd0e8069221f3e3 services/console/db/migrate/20260602041006_add_role_to_grants.rb +c748db464e2efcb8ac077e57b5934a1853635d35d10f95da7e6d6dce05f14fc5 services/console/db/migrate/20260602050000_allow_unassigned_proxies.rb +8e4e664428666b4f5ea7bb36eb3d20a197cbd1eed8a0b84c93997eecb053cfb6 services/console/db/migrate/20260602060000_create_pg_dsn_secrets.rb +b9962c8a2b05d4238feee644e6c9379720a1cb7475b4e4b39e19e431ad98ac97 services/console/db/migrate/20260602060001_add_pg_dsn_secret_to_secret_sources.rb +2c8a001daa5cc0be3423fbc6e1359531127c2c420215766ff053cda82484e58a services/console/db/migrate/20260602060002_add_pg_dsn_secret_to_grants.rb +46ef9b9aa2b335f4ecc3b45cc02b1aff02bd7aa45a71a4a70f11da02c6212e69 services/console/db/migrate/20260603044111_add_database_to_pg_dsn_secrets.rb +d6adfe03263fe5c594f7523b39a372f950cb88ab5c427dac4f56d0f7a605631e services/console/db/migrate/20260603120000_create_hmac_secrets.rb +fe25c26fc6d0946e68f87366d17c43ca50056020d74b9d3df3b59240f5bc8a6f services/console/db/migrate/20260603120001_add_hmac_secret_to_owners.rb +4ab9c1d1cf2f6dd65ac33f3864c6c281e6b19f3220107b47c5a0d4ac6327a02a services/console/db/migrate/20260603130000_dedupe_grants.rb +cb12a43516c40e12667b9fea7cb181fee96da60025aacc93c75a21317b0c9cba services/console/db/migrate/20260604120000_create_broker_credentials.rb +2e1d9a94b475f6dd9fbce1848a006470d1486d78810a24fc2a77a6bce6a99ed9 services/console/db/migrate/20260605155736_require_database_on_pg_dsn_secrets.rb +7f16a2ae6c8f5893c0c957fdd28cbe73fa8779f11ab63bf9dbdf210dfbefc833 services/console/db/migrate/20260608210001_create_aws_auth_secrets.rb +65700b692b1a6fd02e422f7f302d20e606d5fe035495601d73ad15843ea54337 services/console/db/migrate/20260608210002_add_aws_auth_secret_to_owners.rb +bc9f4b244a1d50359404df195afa8e69e73b0451ec803479466b5e3c1d4ad81e services/console/db/migrate/20260611044016_add_priority_to_grants.rb +00e7ad61e8ef82b90974428f7031d74686684e0a997f6b234f7777b8e25b2867 services/console/db/migrate/20260611120000_create_oauth_apps.rb +5879414ba43e03cc592e4d85225251fb3946d8828facf410a461b41c23862a6e services/console/db/migrate/20260611120100_add_oauth_app_to_broker_credentials.rb +5bb25ba2b54e7396f2986db633badcb1df3b340d8639105e43fdb4cb7d9f04a3 services/console/db/migrate/20260611130000_add_settings_to_pg_dsn_secrets.rb +40e8bc9dffc10999be08978a2db95a0001455b07a89e20d80f492325b968946b services/console/db/migrate/20260612100000_add_sso_fields_to_users.rb +1dada378c4607ef1c30b3a20df1bffbe57baa6897fa0cb1bc58b74f7bd0c7e14 services/console/db/migrate/20260612100100_create_user_identities.rb +29a4b7bd7320b867b197823f28b344ff39fd92b66a066c7a94f9651e96ec9075 services/console/db/migrate/20260615224547_link_static_secrets_to_broker_credentials.rb +72035da9fe85193521f80f299ed589177c45ca021b3bd67946fc09b10c64f1b3 services/console/db/migrate/20260616170000_create_principal_sync_config_snapshots.rb +d5692acab1afd06b580bd2eff3d4a1c48cb4061754d74f02c834be5b0cccdded services/console/db/migrate/20260623231029_create_gcp_id_token_secrets.rb +a35c4a53ac5d33fbab5f92c1643c41c37cfc90769a081a88330877f7d607643f services/console/db/migrate/20260623231036_add_gcp_id_token_secret_to_owners.rb +d1cf6747505c85c857a886784302de7e5fe36ed0b36404b05c75441298fadf8b services/console/db/migrate/20260624000100_add_password_grant_to_broker_credentials.rb +c7820a644016fe4f9198b0c95f7cd413877e0bf2e2042d8b2636a74d3b374f5a services/console/db/migrate/20260625002334_add_api_key_to_broker_credentials.rb +a8206f082c52b95137fd198f24f9cccd66163578f18e3c19fe5ef028cf1ee40b services/console/db/migrate/20260625030000_add_sandbox_capabilities_to_principals.rb +958a8ed342d55e2114cd3460fe4b6bc7f197969a2001cead7f8562f97938b249 services/console/db/migrate/20260630090000_create_mcp_oauth_clients.rb +ea51537c7adbf0c5f3654c41d36414557611e28cf76950e548813ac962f04b0a services/console/db/migrate/20260630090001_create_mcp_oauth_authorization_codes.rb +c538b2c610843b38d0da84965c3140b9c06f45aaa963978296f8367bfc8615aa services/console/db/migrate/20260630090002_create_mcp_oauth_refresh_tokens.rb +13b0e05b089dea0794858e5d3dd24a4de0a8ff318915d827ca9653993abcfb01 services/console/db/migrate/20260702000000_add_sandbox_api_server_capability_to_principals.rb +cdf72afc5539c2329ea92fe1ea7b6d148cc556f26955ff1aa693a146f84945b6 services/console/db/migrate/20260707190000_migrate_sandbox_repo_cache_to_principal_label.rb +5b8f825df95199fb47a740f15a36263aeb5ddb58f15473aba0c476b4d08e71f1 services/console/db/migrate/20260709160000_add_team_id_to_user_identities.rb +aee4f8181e9ad68a949e3ad1e21be9ed9b8ef84bd0eb171a2c782e53bc9ef4e1 services/console/db/migrate/20260709174916_create_slack_channel_permissions.rb +f2e59517094a049919adfcc48e84bf9f1569f13ccbf43a14ccfe4da346f22a09 services/console/db/migrate/20260709223000_backfill_slack_channel_permissions_from_labels.rb +43387034d3afe8942416311cdad834a2f0e001722884aa4086d2db2e8dabdac3 services/console/db/migrate/20260711182055_create_system_settings.rb +357a1338a43d5f7ef1cede938fb947ece878903e18ea0357b6eb73e60d99c25b services/console/db/migrate/20260711190035_add_sandbox_repo_cache_to_principals.rb diff --git a/services/console/db/migrate/20260630090000_create_mcp_oauth_clients.rb b/services/console/db/migrate/20260630090000_create_mcp_oauth_clients.rb new file mode 100644 index 000000000..6a0710b91 --- /dev/null +++ b/services/console/db/migrate/20260630090000_create_mcp_oauth_clients.rb @@ -0,0 +1,15 @@ +class CreateMcpOauthClients < ActiveRecord::Migration[8.1] + def change + create_table :mcp_oauth_clients do |t| + t.string :name + t.jsonb :redirect_uris, null: false, default: [] + t.jsonb :grant_types, null: false, default: [] + t.jsonb :response_types, null: false, default: [] + t.jsonb :scopes, null: false, default: [] + t.jsonb :metadata, null: false, default: {} + t.datetime :last_used_at + + t.timestamps + end + end +end diff --git a/services/console/db/migrate/20260630090001_create_mcp_oauth_authorization_codes.rb b/services/console/db/migrate/20260630090001_create_mcp_oauth_authorization_codes.rb new file mode 100644 index 000000000..0f5108b26 --- /dev/null +++ b/services/console/db/migrate/20260630090001_create_mcp_oauth_authorization_codes.rb @@ -0,0 +1,21 @@ +class CreateMcpOauthAuthorizationCodes < ActiveRecord::Migration[8.1] + def change + create_table :mcp_oauth_authorization_codes do |t| + t.references :mcp_oauth_client, null: false, foreign_key: true + t.references :user, null: false, foreign_key: true + t.references :principal, null: false, foreign_key: true + t.string :code_hash, null: false + t.string :redirect_uri, null: false + t.string :code_challenge, null: false + t.string :resource, null: false + t.jsonb :scopes, null: false, default: [] + t.datetime :expires_at, null: false + t.datetime :consumed_at + + t.timestamps + end + + add_index :mcp_oauth_authorization_codes, :code_hash, unique: true + add_index :mcp_oauth_authorization_codes, :expires_at + end +end diff --git a/services/console/db/migrate/20260630090002_create_mcp_oauth_refresh_tokens.rb b/services/console/db/migrate/20260630090002_create_mcp_oauth_refresh_tokens.rb new file mode 100644 index 000000000..03f1c44ed --- /dev/null +++ b/services/console/db/migrate/20260630090002_create_mcp_oauth_refresh_tokens.rb @@ -0,0 +1,20 @@ +class CreateMcpOauthRefreshTokens < ActiveRecord::Migration[8.1] + def change + create_table :mcp_oauth_refresh_tokens do |t| + t.references :mcp_oauth_client, null: false, foreign_key: true + t.references :user, null: false, foreign_key: true + t.references :principal, null: false, foreign_key: true + t.string :token_hash, null: false + t.string :resource, null: false + t.jsonb :scopes, null: false, default: [] + t.datetime :expires_at, null: false + t.datetime :revoked_at + t.datetime :last_used_at + + t.timestamps + end + + add_index :mcp_oauth_refresh_tokens, :token_hash, unique: true + add_index :mcp_oauth_refresh_tokens, :expires_at + end +end diff --git a/services/console/db/migrate/20260702000000_add_sandbox_api_server_capability_to_principals.rb b/services/console/db/migrate/20260702000000_add_sandbox_api_server_capability_to_principals.rb new file mode 100644 index 000000000..b6ab4e207 --- /dev/null +++ b/services/console/db/migrate/20260702000000_add_sandbox_api_server_capability_to_principals.rb @@ -0,0 +1,5 @@ +class AddSandboxApiServerCapabilityToPrincipals < ActiveRecord::Migration[8.1] + def change + add_column :principals, :sandbox_api_server_enabled, :boolean, null: false, default: true + end +end diff --git a/services/console/db/migrate/20260707190000_migrate_sandbox_repo_cache_to_principal_label.rb b/services/console/db/migrate/20260707190000_migrate_sandbox_repo_cache_to_principal_label.rb new file mode 100644 index 000000000..feaf0d47d --- /dev/null +++ b/services/console/db/migrate/20260707190000_migrate_sandbox_repo_cache_to_principal_label.rb @@ -0,0 +1,27 @@ +class MigrateSandboxRepoCacheToPrincipalLabel < ActiveRecord::Migration[8.1] + LABEL_KEY = "centaur.sandbox_repo_cache" + + def up + execute <<~SQL.squish + UPDATE principals + SET labels = COALESCE(labels, '{}'::jsonb) || + jsonb_build_object( + '#{LABEL_KEY}', + CASE WHEN sandbox_repo_cache_enabled THEN 'all' ELSE 'none' END + ), + sync_config_cache_version = COALESCE(sync_config_cache_version, 0) + 1, + updated_at = NOW() + SQL + end + + def down + execute <<~SQL.squish + UPDATE principals + SET sandbox_repo_cache_enabled = (labels ->> '#{LABEL_KEY}' = 'all'), + labels = COALESCE(labels, '{}'::jsonb) - '#{LABEL_KEY}', + sync_config_cache_version = COALESCE(sync_config_cache_version, 0) + 1, + updated_at = NOW() + WHERE labels ? '#{LABEL_KEY}' + SQL + end +end diff --git a/services/console/db/migrate/20260709160000_add_team_id_to_user_identities.rb b/services/console/db/migrate/20260709160000_add_team_id_to_user_identities.rb new file mode 100644 index 000000000..d2383afd3 --- /dev/null +++ b/services/console/db/migrate/20260709160000_add_team_id_to_user_identities.rb @@ -0,0 +1,5 @@ +class AddTeamIdToUserIdentities < ActiveRecord::Migration[8.1] + def change + add_column :user_identities, :team_id, :string + end +end diff --git a/services/console/db/migrate/20260709174916_create_slack_channel_permissions.rb b/services/console/db/migrate/20260709174916_create_slack_channel_permissions.rb new file mode 100644 index 000000000..ae8b28440 --- /dev/null +++ b/services/console/db/migrate/20260709174916_create_slack_channel_permissions.rb @@ -0,0 +1,16 @@ +class CreateSlackChannelPermissions < ActiveRecord::Migration[8.1] + def change + create_table :slack_channel_permissions do |t| + t.references :principal, null: false, foreign_key: true + t.string :channel_id, null: false + t.string :channel_name + t.boolean :upload_enabled, null: false, default: false + t.boolean :download_enabled, null: false, default: false + t.boolean :history_enabled, null: false, default: false + + t.timestamps + end + + add_index :slack_channel_permissions, %i[principal_id channel_id], unique: true + end +end diff --git a/services/console/db/migrate/20260709223000_backfill_slack_channel_permissions_from_labels.rb b/services/console/db/migrate/20260709223000_backfill_slack_channel_permissions_from_labels.rb new file mode 100644 index 000000000..1a8bfaaca --- /dev/null +++ b/services/console/db/migrate/20260709223000_backfill_slack_channel_permissions_from_labels.rb @@ -0,0 +1,31 @@ +class BackfillSlackChannelPermissionsFromLabels < ActiveRecord::Migration[8.1] + def up + execute <<~SQL.squish + INSERT INTO slack_channel_permissions ( + principal_id, + channel_id, + upload_enabled, + download_enabled, + history_enabled, + created_at, + updated_at + ) + SELECT + principals.id, + upper(trim(principals.labels->>'slack_channel_id')), + TRUE, + TRUE, + TRUE, + CURRENT_TIMESTAMP, + CURRENT_TIMESTAMP + FROM principals + WHERE upper(trim(principals.labels->>'slack_channel_id')) ~ '^[CDG][A-Z0-9]{8,}$' + ON CONFLICT (principal_id, channel_id) DO NOTHING + SQL + end + + def down + # One-way data backfill. Existing operators may edit these permissions after + # migration, so rollback should not delete potentially modified rows. + end +end diff --git a/services/console/db/migrate/20260711182055_create_system_settings.rb b/services/console/db/migrate/20260711182055_create_system_settings.rb new file mode 100644 index 000000000..f5fb241eb --- /dev/null +++ b/services/console/db/migrate/20260711182055_create_system_settings.rb @@ -0,0 +1,14 @@ +class CreateSystemSettings < ActiveRecord::Migration[8.1] + def change + create_table :system_settings do |t| + t.boolean :singleton, null: false, default: true + t.string :default_sandbox_repo_cache, null: false, default: "all" + t.boolean :default_sandbox_observability_enabled, null: false, default: true + t.boolean :default_sandbox_api_server_enabled, null: false, default: true + + t.timestamps + end + + add_index :system_settings, :singleton, unique: true + end +end diff --git a/services/console/db/migrate/20260711190035_add_sandbox_repo_cache_to_principals.rb b/services/console/db/migrate/20260711190035_add_sandbox_repo_cache_to_principals.rb new file mode 100644 index 000000000..0b83b98bc --- /dev/null +++ b/services/console/db/migrate/20260711190035_add_sandbox_repo_cache_to_principals.rb @@ -0,0 +1,42 @@ +class AddSandboxRepoCacheToPrincipals < ActiveRecord::Migration[8.1] + LABEL_KEY = "centaur.sandbox_repo_cache" + + def up + add_column :principals, :sandbox_repo_cache, :string + + execute <<~SQL.squish + WITH normalized AS ( + SELECT id, + CASE LOWER(TRIM(COALESCE(labels ->> '#{LABEL_KEY}', ''))) + WHEN 'all' THEN 'all' + WHEN 'public' THEN 'public' + WHEN 'pub' THEN 'public' + WHEN 'none' THEN 'none' + ELSE CASE WHEN sandbox_repo_cache_enabled THEN 'all' ELSE 'none' END + END AS repo_cache + FROM principals + ) + UPDATE principals + SET sandbox_repo_cache = normalized.repo_cache, + labels = (COALESCE(labels, '{}'::jsonb) - '#{LABEL_KEY}') || + jsonb_build_object('#{LABEL_KEY}', normalized.repo_cache) + FROM normalized + WHERE principals.id = normalized.id + SQL + + change_column_default :principals, :sandbox_repo_cache, from: nil, to: "all" + change_column_null :principals, :sandbox_repo_cache, false + remove_column :principals, :sandbox_repo_cache_enabled + end + + def down + add_column :principals, :sandbox_repo_cache_enabled, :boolean, null: false, default: true + + execute <<~SQL.squish + UPDATE principals + SET sandbox_repo_cache_enabled = (sandbox_repo_cache = 'all') + SQL + + remove_column :principals, :sandbox_repo_cache + end +end diff --git a/services/console/db/schema.rb b/services/console/db/schema.rb index 5402028d7..22aeeff31 100644 --- a/services/console/db/schema.rb +++ b/services/console/db/schema.rb @@ -10,7 +10,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema[8.1].define(version: 2026_06_25_030000) do +ActiveRecord::Schema[8.1].define(version: 2026_07_11_190035) do # These are extensions that must be enabled in order to support this database enable_extension "pg_catalog.plpgsql" @@ -176,6 +176,57 @@ t.index ["namespace", "foreign_id"], name: "index_hmac_secrets_on_namespace_and_foreign_id", unique: true end + create_table "mcp_oauth_authorization_codes", force: :cascade do |t| + t.string "code_challenge", null: false + t.string "code_hash", null: false + t.datetime "consumed_at" + t.datetime "created_at", null: false + t.datetime "expires_at", null: false + t.bigint "mcp_oauth_client_id", null: false + t.bigint "principal_id", null: false + t.string "redirect_uri", null: false + t.string "resource", null: false + t.jsonb "scopes", default: [], null: false + t.datetime "updated_at", null: false + t.bigint "user_id", null: false + t.index ["code_hash"], name: "index_mcp_oauth_authorization_codes_on_code_hash", unique: true + t.index ["expires_at"], name: "index_mcp_oauth_authorization_codes_on_expires_at" + t.index ["mcp_oauth_client_id"], name: "index_mcp_oauth_authorization_codes_on_mcp_oauth_client_id" + t.index ["principal_id"], name: "index_mcp_oauth_authorization_codes_on_principal_id" + t.index ["user_id"], name: "index_mcp_oauth_authorization_codes_on_user_id" + end + + create_table "mcp_oauth_clients", force: :cascade do |t| + t.datetime "created_at", null: false + t.jsonb "grant_types", default: [], null: false + t.datetime "last_used_at" + t.jsonb "metadata", default: {}, null: false + t.string "name" + t.jsonb "redirect_uris", default: [], null: false + t.jsonb "response_types", default: [], null: false + t.jsonb "scopes", default: [], null: false + t.datetime "updated_at", null: false + end + + create_table "mcp_oauth_refresh_tokens", force: :cascade do |t| + t.datetime "created_at", null: false + t.datetime "expires_at", null: false + t.datetime "last_used_at" + t.bigint "mcp_oauth_client_id", null: false + t.bigint "principal_id", null: false + t.string "resource", null: false + t.datetime "revoked_at" + t.jsonb "scopes", default: [], null: false + t.string "token_hash", null: false + t.datetime "updated_at", null: false + t.bigint "user_id", null: false + t.index ["expires_at"], name: "index_mcp_oauth_refresh_tokens_on_expires_at" + t.index ["mcp_oauth_client_id"], name: "index_mcp_oauth_refresh_tokens_on_mcp_oauth_client_id" + t.index ["principal_id"], name: "index_mcp_oauth_refresh_tokens_on_principal_id" + t.index ["token_hash"], name: "index_mcp_oauth_refresh_tokens_on_token_hash", unique: true + t.index ["user_id"], name: "index_mcp_oauth_refresh_tokens_on_user_id" + end + create_table "oauth_apps", force: :cascade do |t| t.jsonb "allowed_scopes", default: [], null: false t.string "client_id", null: false @@ -259,8 +310,9 @@ t.jsonb "labels", default: {}, null: false t.string "name" t.string "namespace", default: "default", null: false + t.boolean "sandbox_api_server_enabled", default: true, null: false t.boolean "sandbox_observability_enabled", default: true, null: false - t.boolean "sandbox_repo_cache_enabled", default: true, null: false + t.string "sandbox_repo_cache", default: "all", null: false t.bigint "sync_config_cache_version", default: 0, null: false t.datetime "updated_at", null: false t.index ["created_by_id"], name: "index_principals_on_created_by_id" @@ -343,6 +395,19 @@ t.index ["static_secret_id"], name: "index_secret_sources_on_static_secret_id", unique: true end + create_table "slack_channel_permissions", force: :cascade do |t| + t.string "channel_id", null: false + t.string "channel_name" + t.datetime "created_at", null: false + t.boolean "download_enabled", default: false, null: false + t.boolean "history_enabled", default: false, null: false + t.bigint "principal_id", null: false + t.datetime "updated_at", null: false + t.boolean "upload_enabled", default: false, null: false + t.index ["principal_id", "channel_id"], name: "index_slack_channel_permissions_on_principal_id_and_channel_id", unique: true + t.index ["principal_id"], name: "index_slack_channel_permissions_on_principal_id" + end + create_table "static_secrets", force: :cascade do |t| t.bigint "broker_credential_id" t.datetime "created_at", null: false @@ -361,12 +426,23 @@ t.index ["namespace", "foreign_id"], name: "index_static_secrets_on_namespace_and_foreign_id", unique: true end + create_table "system_settings", force: :cascade do |t| + t.datetime "created_at", null: false + t.boolean "default_sandbox_api_server_enabled", default: true, null: false + t.boolean "default_sandbox_observability_enabled", default: true, null: false + t.string "default_sandbox_repo_cache", default: "all", null: false + t.boolean "singleton", default: true, null: false + t.datetime "updated_at", null: false + t.index ["singleton"], name: "index_system_settings_on_singleton", unique: true + end + create_table "user_identities", force: :cascade do |t| t.datetime "created_at", null: false t.string "email" t.boolean "email_verified", default: false, null: false t.string "provider", null: false t.string "subject", null: false + t.string "team_id" t.datetime "updated_at", null: false t.bigint "user_id", null: false t.index ["provider", "subject"], name: "index_user_identities_on_provider_and_subject", unique: true @@ -404,6 +480,12 @@ add_foreign_key "grants", "static_secrets" add_foreign_key "grants", "users", column: "created_by_id" add_foreign_key "hmac_secrets", "users", column: "created_by_id" + add_foreign_key "mcp_oauth_authorization_codes", "mcp_oauth_clients" + add_foreign_key "mcp_oauth_authorization_codes", "principals" + add_foreign_key "mcp_oauth_authorization_codes", "users" + add_foreign_key "mcp_oauth_refresh_tokens", "mcp_oauth_clients" + add_foreign_key "mcp_oauth_refresh_tokens", "principals" + add_foreign_key "mcp_oauth_refresh_tokens", "users" add_foreign_key "oauth_apps", "users", column: "created_by_id" add_foreign_key "oauth_token_secrets", "users", column: "created_by_id" add_foreign_key "pg_dsn_secrets", "users", column: "created_by_id" @@ -426,6 +508,7 @@ add_foreign_key "secret_sources", "oauth_token_secrets" add_foreign_key "secret_sources", "pg_dsn_secrets" add_foreign_key "secret_sources", "static_secrets" + add_foreign_key "slack_channel_permissions", "principals" add_foreign_key "static_secrets", "broker_credentials" add_foreign_key "static_secrets", "users", column: "created_by_id" add_foreign_key "user_identities", "users" diff --git a/services/console/db/seeds.rb b/services/console/db/seeds.rb index 4fbd6ed97..4b3f5395e 100644 --- a/services/console/db/seeds.rb +++ b/services/console/db/seeds.rb @@ -1,9 +1,88 @@ # This file should ensure the existence of records required to run the application in every environment (production, # development, test). The code here should be idempotent so that it can be executed at any point in every environment. # The data can then be loaded with the bin/rails db:seed command (or created alongside the database with db:setup). -# -# Example: -# -# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name| -# MovieGenre.find_or_create_by!(name: genre_name) -# end + +# Sample OAuth apps for the Integrations page, one per supported provider. +# Dev/test only: the client ids and secrets are placeholders, so the consent +# flows they name will not complete against real providers -- they exist to +# exercise the console UI (Apps list, Integrations cards, start links). +unless Rails.env.production? + seed_user = User.order(:id).first || User.create!( + email: ConsoleEnv["INITIAL_USER_EMAIL"].presence || "dev@iron.local", + password: ConsoleEnv["INITIAL_USER_PASSWORD"].presence || "dev-password-1234", + status: "active", + admin: true + ) + + [ + { + slug: "attio", + provider: "attio", + description: "Attio workspace access for CRM records and object configuration", + # Attio scopes are configured in the Attio developer dashboard; this + # allowlist mirrors the dashboard configuration we expect: user + # management read-only, everything else read-write. + allowed_scopes: %w[ + user_management:read + record_permission:read-write + object_configuration:read-write + list_entry:read-write + list_configuration:read-write + comment:read-write + note:read-write + task:read-write + meeting:read-write + call_recording:read-write + webhook:read-write + file:read-write + ] + }, + { + slug: "google", + provider: "google", + description: "Google Workspace (Gmail, Calendar, Drive)", + allowed_scopes: %w[ + https://www.googleapis.com/auth/gmail.readonly + https://www.googleapis.com/auth/calendar.readonly + https://www.googleapis.com/auth/drive.readonly + ] + }, + { + slug: "slack", + provider: "slack", + description: "Slack workspace access for messages and channels", + allowed_scopes: %w[chat:write channels:history channels:read users:read] + }, + { + slug: "github", + provider: "github", + description: "GitHub repositories and user profile", + allowed_scopes: %w[repo read:user] + }, + { + slug: "granola", + provider: "granola", + description: "Granola MCP access", + # A real client_id/client_secret comes from one-time dynamic registration: + # POST https://mcp-auth.granola.ai/oauth2/register + allowed_scopes: %w[mcp] + }, + { + slug: "linear", + provider: "linear", + description: "Linear workspace access for issues and comments", + allowed_scopes: %w[read write] + } + ].each do |attrs| + OauthApp.find_or_create_by!(slug: attrs[:slug]) do |app| + app.provider = attrs[:provider] + app.description = attrs[:description] + app.allowed_scopes = attrs[:allowed_scopes] + app.client_id = "seed-#{attrs[:slug]}-client-id" + app.client_secret = "seed-#{attrs[:slug]}-client-secret" + app.credential_namespace = "default" + app.enabled = true + app.created_by = seed_user + end + end +end diff --git a/services/console/docs/API.md b/services/console/docs/API.md index e51a540b5..b063099a9 100644 --- a/services/console/docs/API.md +++ b/services/console/docs/API.md @@ -810,6 +810,8 @@ The token credentials it refreshes with are fields on the credential, resolved b GitHub traffic is wired to this broker by the built-in sandbox proxy fragment (`centaur-iron-proxy/src/infra.yaml`): `github.com` and `api.github.com` already carry a `token_broker` source referencing the `github-app` credential, so a deployment only needs to provision that credential — no per-deployment static secret. The Helm chart can provision it at api-rs startup when `tokenBroker.githubApp.enabled=true` and `tokenBroker.githubApp.existingSecretName` points at a Kubernetes Secret containing the App ID, installation ID, and private key. The fragment uses a `replace` (placeholder swap) rather than a header `inject`, which preserves the caller's auth scheme: `git` over HTTPS keeps its Basic `x-access-token:` form (`github.com` rejects `Bearer` for git transport) while the REST API keeps `Bearer`. To provision it manually instead of via Helm, run: +`tokenBroker.githubApp.credentialId` and `extraCredentialIds` are compatibility aliases provisioned alongside the canonical `github-app` credential. They are bootstrap-only unless a separate, explicitly scoped secret references them; the built-in GitHub rules intentionally have one source so equal-priority credentials cannot compete for the same Authorization header. + ``` centaur-perms broker create --namespace default --foreign-id github-app \ --grant github_app_installation --client-id "$GITHUB_APP_ID" \ diff --git a/services/console/lib/api_server/jwt.rb b/services/console/lib/api_server/jwt.rb new file mode 100644 index 000000000..15f027bda --- /dev/null +++ b/services/console/lib/api_server/jwt.rb @@ -0,0 +1,58 @@ +require "zlib" + +module ApiServer + module Jwt + DEFAULT_AUDIENCE = "centaur-api".freeze + DEFAULT_ISSUER = "centaur-console".freeze + DEFAULT_WINDOW_SECONDS = 15.minutes.to_i + DEFAULT_TTL_SECONDS = 1.hour.to_i + + module_function + + def encode_for_principal(principal, now: Time.current) + upload_channels = principal.slack_upload_channel_ids + download_channels = principal.slack_download_channel_ids + history_channels = principal.slack_history_channel_ids + signing_secret = ENV["CENTAUR_JWT_SIGNING_SECRET"].to_s + return nil if signing_secret.blank? + + issued_at = window_start_for(principal, now.to_i) + expires_at = issued_at + DEFAULT_TTL_SECONDS + CentaurJwt::Hs256.encode( + { + "iss" => issuer, + "sub" => principal.oid, + "aud" => audience, + "iat" => issued_at, + "exp" => expires_at, + "slack" => { + "upload_channels" => upload_channels, + "download_channels" => download_channels, + "history_channels" => history_channels + } + }, + signing_secret: signing_secret + ) + end + + # Rotation boundaries are offset per principal (deterministically, from + # the oid) so the fleet's tokens don't all roll over — and force snapshot + # rebuilds — at the same instant. + def window_start_for(principal, timestamp) + offset = rotation_offset(principal) + timestamp - ((timestamp - offset) % DEFAULT_WINDOW_SECONDS) + end + + def rotation_offset(principal) + Zlib.crc32(principal.oid.to_s) % DEFAULT_WINDOW_SECONDS + end + + def audience + ENV["CENTAUR_API_JWT_AUDIENCE"].presence || DEFAULT_AUDIENCE + end + + def issuer + ENV["CENTAUR_API_JWT_ISSUER"].presence || DEFAULT_ISSUER + end + end +end diff --git a/services/console/lib/broker/credential_grants.rb b/services/console/lib/broker/credential_grants.rb index a649023c2..23809357b 100644 --- a/services/console/lib/broker/credential_grants.rb +++ b/services/console/lib/broker/credential_grants.rb @@ -1,3 +1,5 @@ +require "uri" + module Broker # Registry for broker credential token-exchange strategies. BrokerCredential # owns persistence and scheduling; these strategies own provider-specific @@ -5,9 +7,11 @@ module Broker module CredentialGrants PREQIN_TOKEN_ENDPOINT = "https://api.preqin.com/connect/token".freeze PREQIN_REFRESH_TOKEN_ENDPOINT = "https://api.preqin.com/connect/refresh_token".freeze + GITHUB_APP_INSTALLATION = "github_app_installation".freeze + DEFAULT_GITHUB_APP_TOKEN_ENDPOINT_HOSTS = [ "api.github.com" ].freeze - GRANTS = %w[refresh_token password preqin].freeze - REFRESHABLE_WITHOUT_TOKEN_GRANTS = %w[password preqin].freeze + GRANTS = [ "refresh_token", "password", "preqin", GITHUB_APP_INSTALLATION ].freeze + REFRESHABLE_WITHOUT_TOKEN_GRANTS = [ "password", "preqin", GITHUB_APP_INSTALLATION ].freeze Outcome = Data.define(:result, :clear_refresh_token, :dead_reason) @@ -26,20 +30,41 @@ def validate(credential) validate_password(credential) when "preqin" validate_preqin(credential) + when GITHUB_APP_INSTALLATION + validate_github_app_installation(credential) end end - def refresh(credential) + def refresh(credential, now: Time.current) case credential.grant when "password" refresh_password(credential) when "preqin" refresh_preqin(credential) + when GITHUB_APP_INSTALLATION + refresh_github_app_installation(credential, now: now) else refresh_token(credential) end end + def github_app_installation_token_endpoint?(value) + uri = URI.parse(value.to_s) + allowed_hosts = ENV.fetch( + "GITHUB_APP_TOKEN_ENDPOINT_HOSTS", + DEFAULT_GITHUB_APP_TOKEN_ENDPOINT_HOSTS.join(",") + ).split(",").filter_map { |host| host.strip.downcase.presence }.uniq + uri.is_a?(URI::HTTPS) && + allowed_hosts.include?(uri.host.to_s.downcase) && + uri.port == 443 && + uri.userinfo.nil? && + uri.query.nil? && + uri.fragment.nil? && + uri.path.match?(%r{\A/app/installations/[^/]+/access_tokens\z}) + rescue URI::InvalidURIError + false + end + private def success(result, clear_refresh_token: false) @@ -116,6 +141,17 @@ def refresh_preqin(credential) success(result, clear_refresh_token: clear_stale_refresh_token && result.refresh_token.blank?) end + def refresh_github_app_installation(credential, now:) + result = credential.github_app_client.mint( + token_endpoint: credential.token_endpoint, + app_id: credential.effective_client_id, + private_key_pem: credential.effective_client_secret, + timeout: credential.refresh_timeout_seconds, + now: now + ) + success(result, clear_refresh_token: true) + end + def oauth_refresh_token(credential) post_token_form( credential, @@ -199,6 +235,20 @@ def validate_preqin(credential) credential.errors.add(:api_key, "can't be blank for the Preqin broker grant") if credential.api_key.blank? end + def validate_github_app_installation(credential) + if credential.effective_client_secret.blank? + credential.errors.add(:client_secret, "can't be blank for a GitHub App installation credential") + end + return if credential.token_endpoint.blank? + + unless github_app_installation_token_endpoint?(credential.token_endpoint) + credential.errors.add( + :token_endpoint, + "must be an approved HTTPS GitHub App installation access-token endpoint" + ) + end + end + def password_values_present?(credential) credential.username.present? && credential.password.present? end diff --git a/services/console/lib/broker/github_app_installation_client.rb b/services/console/lib/broker/github_app_installation_client.rb index 3d5b667de..e454edbd1 100644 --- a/services/console/lib/broker/github_app_installation_client.rb +++ b/services/console/lib/broker/github_app_installation_client.rb @@ -23,6 +23,14 @@ def mint(token_endpoint:, app_id:, private_key_pem:, timeout: DEFAULT_TIMEOUT, n raise ArgumentError, "token endpoint is required" if token_endpoint.blank? raise ArgumentError, "app_id is required" if app_id.blank? raise ArgumentError, "private_key_pem is required" if private_key_pem.blank? + unless CredentialGrants.github_app_installation_token_endpoint?(token_endpoint) + raise RefreshError.new( + "GitHub App token endpoint is not approved", + stage: "config", + code: "invalid_token_endpoint", + retryable: false + ) + end jwt = app_jwt(app_id: app_id, private_key_pem: private_key_pem, now: now) response = perform(token_endpoint, jwt, timeout) diff --git a/services/console/lib/centaur_jwt/hs256.rb b/services/console/lib/centaur_jwt/hs256.rb new file mode 100644 index 000000000..cef627be5 --- /dev/null +++ b/services/console/lib/centaur_jwt/hs256.rb @@ -0,0 +1,23 @@ +require "base64" +require "json" +require "openssl" + +module CentaurJwt + module Hs256 + module_function + + def encode(payload, signing_secret:) + signing_secret = signing_secret.to_s + raise KeyError, "CENTAUR_JWT_SIGNING_SECRET is not configured" if signing_secret.blank? + + header = { "alg" => "HS256", "typ" => "JWT" } + signing_input = [ base64url_json(header), base64url_json(payload) ].join(".") + signature = OpenSSL::HMAC.digest("SHA256", signing_secret, signing_input) + "#{signing_input}.#{Base64.urlsafe_encode64(signature, padding: false)}" + end + + def base64url_json(value) + Base64.urlsafe_encode64(JSON.generate(value), padding: false) + end + end +end diff --git a/services/console/lib/login/providers/slack.rb b/services/console/lib/login/providers/slack.rb index 2e49cec8f..5b5632178 100644 --- a/services/console/lib/login/providers/slack.rb +++ b/services/console/lib/login/providers/slack.rb @@ -5,6 +5,7 @@ module Providers # endpoint returns an id_token carrying the account identity. class Slack KEY = "slack" + TEAM_ID_CLAIM = "https://slack.com/team_id".freeze AUTHORIZATION_ENDPOINT = "https://slack.com/openid/connect/authorize" TOKEN_ENDPOINT = "https://slack.com/api/openid.connect.token" SCOPES = %w[openid email profile].freeze @@ -17,7 +18,9 @@ def scopes = SCOPES def extra_authorization_params = {} def identity_from(result, client_id:) - Login::IdToken.identity(result.id_token, client_id: client_id, valid_issuers: VALID_ISSUERS) + identity = Login::IdToken.identity(result.id_token, client_id: client_id, valid_issuers: VALID_ISSUERS) + claims = Login::IdToken.decode_claims(result.id_token) + identity.merge(team_id: claims[TEAM_ID_CLAIM].to_s.strip.presence) end end end diff --git a/services/console/lib/mcp/jwt.rb b/services/console/lib/mcp/jwt.rb new file mode 100644 index 000000000..df8e7ff70 --- /dev/null +++ b/services/console/lib/mcp/jwt.rb @@ -0,0 +1,10 @@ +module Mcp + module Jwt + module_function + + def encode(payload) + signing_secret = ENV["CENTAUR_JWT_SIGNING_SECRET"].to_s + CentaurJwt::Hs256.encode(payload, signing_secret: signing_secret) + end + end +end diff --git a/services/console/lib/oauth/providers.rb b/services/console/lib/oauth/providers.rb index f2e457213..10c15a234 100644 --- a/services/console/lib/oauth/providers.rb +++ b/services/console/lib/oauth/providers.rb @@ -10,8 +10,11 @@ module Providers # stateless, so sharing one instance across flows is safe. def self.registry @registry ||= { + Attio::KEY => Attio.new, Github::KEY => Github.new, Google::KEY => Google.new, + Granola::KEY => Granola.new, + Linear::KEY => Linear.new, Slack::KEY => Slack.new }.freeze end diff --git a/services/console/lib/oauth/providers/attio.rb b/services/console/lib/oauth/providers/attio.rb new file mode 100644 index 000000000..60e3c35ad --- /dev/null +++ b/services/console/lib/oauth/providers/attio.rb @@ -0,0 +1,51 @@ +require "digest" + +module Oauth + module Providers + # Attio OAuth consent-flow strategy. Attio app scopes are configured in the + # Attio developer dashboard, not requested on the authorization redirect; + # the token response carries a long-lived workspace-scoped access token with + # no refresh token, expiry, scope, or identity payload. The callback stores a + # deterministic pending workspace identity derived from the token, and + # EnrichAttioCredentialIdentityJob replaces it with the Attio workspace id + # and name from /v2/self. + class Attio + KEY = "attio" + AUTHORIZATION_ENDPOINT = "https://app.attio.com/authorize" + TOKEN_ENDPOINT = "https://app.attio.com/oauth/token" + SELF_ENDPOINT = "https://api.attio.com/v2/self" + IDENTITY_SCOPES = [].freeze + API_HOSTS = %w[api.attio.com].freeze + + def key = KEY + def display_name = "Attio" + def authorization_endpoint = AUTHORIZATION_ENDPOINT + def token_endpoint = TOKEN_ENDPOINT + def identity_scopes = IDENTITY_SCOPES + def api_hosts = API_HOSTS + def authorization_scope_param = "scope" + def scope_separator = " " + def extra_authorization_params = {} + def refreshable? = false + + def parse_granted_scopes(scope) + scope.to_s.split(/[,\s]+/).reject(&:blank?) + end + + def refresh_scopes(_scopes) = [] + + def identity_from(result, client_id:) + if result.access_token.blank? + raise Broker::ExchangeError.new("token response returned an empty access_token", + stage: "parse", code: "missing_access_token") + end + + { + subject: "pending-#{Digest::SHA256.hexdigest(result.access_token)[0, 32]}", + email: nil, + name: "Pending Attio workspace" + } + end + end + end +end diff --git a/services/console/lib/oauth/providers/granola.rb b/services/console/lib/oauth/providers/granola.rb new file mode 100644 index 000000000..edc4336f8 --- /dev/null +++ b/services/console/lib/oauth/providers/granola.rb @@ -0,0 +1,88 @@ +require "base64" +require "json" + +module Oauth + module Providers + # Granola consent-flow strategy. This targets Granola's OAuth server for its + # MCP endpoint (https://mcp.granola.ai/mcp), which Granola documents as + # browser OAuth for MCP rather than as a classic third-party app dashboard. + # Operators obtain the OAuth client once via RFC 7591 dynamic client + # registration at https://mcp-auth.granola.ai/oauth2/register, and Granola may + # change this MCP-backed availability over time. + # + # SECURITY: identity extraction touches the id_token, which carries the + # account identity but no tokens. As elsewhere under Broker/Oauth, nothing + # here logs token material. + class Granola + KEY = "granola" + AUTHORIZATION_ENDPOINT = "https://mcp-auth.granola.ai/oauth2/authorize" + TOKEN_ENDPOINT = "https://mcp-auth.granola.ai/oauth2/token" + # Always requested in addition to the app's API scopes, so the token + # response carries an id_token identifying the Granola account. Granola's + # authorization server requires offline_access to issue a refresh token; + # request it here because the consent flow requires refreshable credentials. + IDENTITY_SCOPES = %w[openid email profile offline_access].freeze + # The access token is for Granola's MCP protected resource. + API_HOSTS = %w[mcp.granola.ai].freeze + VALID_ISSUERS = %w[https://mcp-auth.granola.ai].freeze + + def key = KEY + def display_name = "Granola" + def authorization_endpoint = AUTHORIZATION_ENDPOINT + def token_endpoint = TOKEN_ENDPOINT + def identity_scopes = IDENTITY_SCOPES + def api_hosts = API_HOSTS + def authorization_scope_param = "scope" + def scope_separator = " " + def extra_authorization_params = {} + def refreshable? = true + + def parse_granted_scopes(scope) = scope.to_s.split + def refresh_scopes(scopes) = Array(scopes) + + # Extracts { subject:, email: } from a successful code-exchange result. + # Decodes the id_token payload without verifying its signature: the token + # came directly from Granola's token endpoint over TLS, which OIDC Core + # 3.1.3.7.6 accepts as sufficient. Sanity-checks aud == client_id and + # iss in the known Granola issuer. Raises Broker::ExchangeError on any + # mismatch or a missing/undecodable id_token. + def identity_from(result, client_id:) + if result.id_token.blank? + raise Broker::ExchangeError.new("token response carried no id_token", + stage: "oauth", code: "missing_id_token") + end + + claims = decode_id_token_claims(result.id_token) + + unless claims["aud"] == client_id + raise Broker::ExchangeError.new("id_token aud did not match client_id", + stage: "oauth", code: "id_token_aud_mismatch") + end + unless VALID_ISSUERS.include?(claims["iss"]) + raise Broker::ExchangeError.new("id_token iss was not a Granola issuer", + stage: "oauth", code: "id_token_iss_invalid") + end + + subject = claims["sub"] + if subject.blank? + raise Broker::ExchangeError.new("id_token carried no sub", + stage: "oauth", code: "id_token_missing_sub") + end + + { subject: subject, email: claims["email"] } + end + + private + + # Decodes the JWT payload (second segment), tolerating the unpadded + # base64url JWTs use. No signature verification -- see identity_from. + def decode_id_token_claims(id_token) + seg = id_token.split(".")[1].to_s + seg += "=" * ((4 - seg.length % 4) % 4) + JSON.parse(Base64.urlsafe_decode64(seg)) + rescue ArgumentError, JSON::ParserError + raise Broker::ExchangeError.new("id_token payload did not decode", stage: "parse") + end + end + end +end diff --git a/services/console/lib/oauth/providers/linear.rb b/services/console/lib/oauth/providers/linear.rb new file mode 100644 index 000000000..396fa6ba7 --- /dev/null +++ b/services/console/lib/oauth/providers/linear.rb @@ -0,0 +1,59 @@ +require "digest" + +module Oauth + module Providers + # Linear OAuth consent-flow strategy. Linear's token response carries the + # access token, granted scopes, and rotating refresh token but no account + # identity. To keep the callback path free of external API calls, the flow + # stores a deterministic pending identity derived from the token and + # EnrichLinearCredentialIdentityJob replaces it with the authenticated + # Linear viewer id/name/email. + class Linear + KEY = "linear" + AUTHORIZATION_ENDPOINT = "https://linear.app/oauth/authorize" + TOKEN_ENDPOINT = "https://api.linear.app/oauth/token" + GRAPHQL_ENDPOINT = "https://api.linear.app/graphql" + IDENTITY_SCOPES = [].freeze + API_HOSTS = %w[api.linear.app].freeze + + def key = KEY + def display_name = "Linear" + def authorization_endpoint = AUTHORIZATION_ENDPOINT + def token_endpoint = TOKEN_ENDPOINT + def identity_scopes = IDENTITY_SCOPES + def api_hosts = API_HOSTS + def authorization_scope_param = "scope" + def scope_separator = "," + def extra_authorization_params = {} + def refreshable? = true + + def parse_granted_scopes(scope) + case scope + when Array + scope.map(&:to_s).reject(&:blank?) + else + scope.to_s.split(/[,\s]+/).reject(&:blank?) + end + end + + # Linear accepts an optional scope parameter on refresh. Pass through the + # originally granted scopes so a refresh preserves the consented grant set; + # CredentialGrants serializes this as the token endpoint's space-separated + # scope field. + def refresh_scopes(scopes) = Array(scopes) + + def identity_from(result, client_id:) + if result.access_token.blank? + raise Broker::ExchangeError.new("token response returned an empty access_token", + stage: "parse", code: "missing_access_token") + end + + { + subject: "pending-#{Digest::SHA256.hexdigest(result.access_token)[0, 32]}", + email: nil, + name: "Pending Linear account" + } + end + end + end +end diff --git a/services/console/lib/oauth/providers/slack.rb b/services/console/lib/oauth/providers/slack.rb index 439cd69dc..affd9aea9 100644 --- a/services/console/lib/oauth/providers/slack.rb +++ b/services/console/lib/oauth/providers/slack.rb @@ -36,7 +36,8 @@ def identity_from(result, client_id:) return { subject: user_id, email: result.response.dig("authed_user", "email"), - name: slack_user_name(result.response) + name: slack_user_name(result.response), + team_id: slack_team_id(result.response) } end @@ -51,6 +52,11 @@ def slack_user_name(response) response.dig("authed_user", "name").presence || response.dig("authed_user", "user").presence end + + def slack_team_id(response) + response.dig("team", "id").presence || + response.dig("authed_user", "team_id").presence + end end end end diff --git a/services/console/lib/security_credential_validation.rb b/services/console/lib/security_credential_validation.rb new file mode 100644 index 000000000..d44494877 --- /dev/null +++ b/services/console/lib/security_credential_validation.rb @@ -0,0 +1,47 @@ +# frozen_string_literal: true + +module SecurityCredentialValidation + MINIMUM_BYTES = 32 + CREDENTIAL_ENV_LANES = [ + %w[CENTAUR_CONSOLE_CENTAUR_API_KEY IRON_CONTROL_CENTAUR_API_KEY], + %w[SLACKBOT_API_KEY], + %w[GITHUBBOT_API_KEY], + %w[LINEARBOT_API_KEY], + %w[DISCORDBOT_API_KEY], + %w[TEAMSBOT_API_KEY], + %w[WORKFLOW_API_KEY], + %w[SLACK_FEEDBACK_API_KEY], + %w[CENTAUR_JWT_SIGNING_SECRET] + ].freeze + + def self.validate!(environment = ENV) + owners = {} + + CREDENTIAL_ENV_LANES.each do |env_names| + env_name = env_names.first + value = configured_value(environment, env_names).to_s.strip + next if value.empty? + + if value.bytesize < MINIMUM_BYTES + raise ArgumentError, "#{env_name} must contain at least #{MINIMUM_BYTES} bytes" + end + + existing = owners[value] + if existing + raise ArgumentError, "#{existing} and #{env_name} must contain distinct service credentials" + end + + owners[value] = env_name + end + + true + end + + def self.configured_value(environment, env_names) + env_names.each do |env_name| + return environment[env_name] if environment.key?(env_name) + end + nil + end + private_class_method :configured_value +end diff --git a/services/console/public/icon-dark.svg b/services/console/public/icon-dark.svg new file mode 100644 index 000000000..4bdda32e2 --- /dev/null +++ b/services/console/public/icon-dark.svg @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + + diff --git a/services/console/public/icon-light.svg b/services/console/public/icon-light.svg new file mode 100644 index 000000000..a1f184c23 --- /dev/null +++ b/services/console/public/icon-light.svg @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + + diff --git a/services/console/public/offline.html b/services/console/public/offline.html new file mode 100644 index 000000000..f58b132e0 --- /dev/null +++ b/services/console/public/offline.html @@ -0,0 +1,46 @@ + + + + Offline · Centaur Console + + + + + + + +
+ +

Console unreachable

+

No connection to the Centaur Console. Check your network — and that you are on the Tailnet — then try again.

+ +
+ + diff --git a/services/console/public/pwa-icon-192.png b/services/console/public/pwa-icon-192.png new file mode 100644 index 000000000..5a75050c0 Binary files /dev/null and b/services/console/public/pwa-icon-192.png differ diff --git a/services/console/public/pwa-icon-512.png b/services/console/public/pwa-icon-512.png new file mode 100644 index 000000000..3b71b8cf0 Binary files /dev/null and b/services/console/public/pwa-icon-512.png differ diff --git a/services/console/public/pwa-icon.svg b/services/console/public/pwa-icon.svg new file mode 100644 index 000000000..1af2c6d62 --- /dev/null +++ b/services/console/public/pwa-icon.svg @@ -0,0 +1,19 @@ + + + + + + + + + + + + + + + + + + + diff --git a/services/console/test/controllers/api/v1/principals_controller_test.rb b/services/console/test/controllers/api/v1/principals_controller_test.rb index ac934bb37..e9d3f91d3 100644 --- a/services/console/test/controllers/api/v1/principals_controller_test.rb +++ b/services/console/test/controllers/api/v1/principals_controller_test.rb @@ -41,9 +41,18 @@ def json_body assert_equal principal.oid, data["id"] assert_equal "acme", data["namespace"] assert_equal "C0123456789", data["foreign_id"] - assert_equal({ "kind" => "slack_channel", "team" => "platform" }, data["labels"]) + assert_equal( + { + "kind" => "slack_channel", + "team" => "platform", + Principal::SANDBOX_REPO_CACHE_LABEL => "all" + }, + data["labels"] + ) + assert_equal "all", data["sandbox_repo_cache"] assert_equal true, data["sandbox_repo_cache_enabled"] assert_equal true, data["sandbox_observability_enabled"] + assert_equal true, data["sandbox_api_server_enabled"] end test "GET returns 404 for an unknown oid" do @@ -63,7 +72,16 @@ def json_body data: { namespace: "acme", foreign_id: "U-new-id", - labels: { "kind" => "user", "team" => "platform" } + labels: { "kind" => "user", "team" => "platform" }, + slack_channel_permissions: [ + { + channel_id: "C0123456789", + channel_name: "general", + upload_enabled: true, + download_enabled: false, + history_enabled: true + } + ] } } @@ -76,9 +94,114 @@ def json_body assert_match(/\Aprn_/, data["id"]) assert_equal "acme", data["namespace"] assert_equal "U-new-id", data["foreign_id"] - assert_equal({ "kind" => "user", "team" => "platform" }, data["labels"]) + assert_equal( + { + "kind" => "user", + "team" => "platform", + Principal::SANDBOX_REPO_CACHE_LABEL => "all" + }, + data["labels"] + ) + assert_equal( + [ + { + "channel_id" => "C0123456789", + "channel_name" => "general", + "upload_enabled" => true, + "download_enabled" => false, + "history_enabled" => true + } + ], + data["slack_channel_permissions"] + ) + assert_equal "all", data["sandbox_repo_cache"] assert_equal true, data["sandbox_repo_cache_enabled"] assert_equal true, data["sandbox_observability_enabled"] + assert_equal true, data["sandbox_api_server_enabled"] + end + + test "POST applies system sandbox defaults when omitted" do + system_settings(:default).update!( + default_sandbox_repo_cache: "public", + default_sandbox_observability_enabled: false, + default_sandbox_api_server_enabled: false + ) + body = { + data: { + namespace: "acme", + foreign_id: "U-defaulted" + } + } + + post api_v1_principals_url, params: body.to_json, headers: auth_headers + assert_response :created + + data = json_body.fetch("data") + assert_equal "public", data["sandbox_repo_cache"] + assert_equal false, data["sandbox_observability_enabled"] + assert_equal false, data["sandbox_api_server_enabled"] + end + + test "POST keeps explicit sandbox capabilities over system defaults" do + system_settings(:default).update!( + default_sandbox_repo_cache: "none", + default_sandbox_observability_enabled: false, + default_sandbox_api_server_enabled: false + ) + body = { + data: { + namespace: "acme", + foreign_id: "U-explicit-capabilities", + sandbox_repo_cache: "all", + sandbox_observability_enabled: true, + sandbox_api_server_enabled: true + } + } + + post api_v1_principals_url, params: body.to_json, headers: auth_headers + assert_response :created + + data = json_body.fetch("data") + assert_equal "all", data["sandbox_repo_cache"] + assert_equal true, data["sandbox_observability_enabled"] + assert_equal true, data["sandbox_api_server_enabled"] + end + + test "POST overwrites explicit repo-cache label with system default" do + system_settings(:default).update!(default_sandbox_repo_cache: "all") + body = { + data: { + namespace: "acme", + foreign_id: "U-explicit-repo-cache-label", + labels: { Principal::SANDBOX_REPO_CACHE_LABEL => "none" } + } + } + + post api_v1_principals_url, params: body.to_json, headers: auth_headers + assert_response :created + + data = json_body.fetch("data") + assert_equal "all", data["sandbox_repo_cache"] + assert_equal({ Principal::SANDBOX_REPO_CACHE_LABEL => "all" }, data["labels"]) + end + + test "POST uses repo-cache param over conflicting label" do + system_settings(:default).update!(default_sandbox_repo_cache: "all") + body = { + data: { + namespace: "acme", + foreign_id: "U-repo-cache-param-wins", + sandbox_repo_cache: "public", + labels: { Principal::SANDBOX_REPO_CACHE_LABEL => "none" } + } + } + + post api_v1_principals_url, params: body.to_json, headers: auth_headers + assert_response :created + + data = json_body.fetch("data") + assert_equal "public", data["sandbox_repo_cache"] + assert_equal({ Principal::SANDBOX_REPO_CACHE_LABEL => "public" }, data["labels"]) end test "POST creates a Principal with only a human-readable name" do @@ -98,8 +221,9 @@ def json_body test "PUT updates the human-readable name" do principal = principals(:acme_channel) principal.update!( - sandbox_repo_cache_enabled: false, - sandbox_observability_enabled: false + sandbox_repo_cache: "none", + sandbox_observability_enabled: false, + sandbox_api_server_enabled: false ) body = { data: { name: "Acme Slack channel" } } @@ -108,16 +232,18 @@ def json_body principal.reload assert_equal "Acme Slack channel", principal.name - assert_equal false, principal.sandbox_repo_cache_enabled + assert_equal "none", principal.sandbox_repo_cache assert_equal false, principal.sandbox_observability_enabled + assert_equal false, principal.sandbox_api_server_enabled end test "PUT updates sandbox access flags" do principal = principals(:acme_channel) body = { data: { - sandbox_repo_cache_enabled: false, - sandbox_observability_enabled: false + sandbox_repo_cache: "public", + sandbox_observability_enabled: false, + sandbox_api_server_enabled: false } } @@ -125,12 +251,15 @@ def json_body assert_response :ok principal.reload - assert_equal false, principal.sandbox_repo_cache_enabled + assert_equal "public", principal.sandbox_repo_cache assert_equal false, principal.sandbox_observability_enabled + assert_equal false, principal.sandbox_api_server_enabled data = json_body.fetch("data") + assert_equal "public", data["sandbox_repo_cache"] assert_equal false, data["sandbox_repo_cache_enabled"] assert_equal false, data["sandbox_observability_enabled"] + assert_equal false, data["sandbox_api_server_enabled"] end test "POST returns 422 when (namespace, foreign_id) already exists" do @@ -160,7 +289,264 @@ def json_body assert_response :ok principal.reload - assert_equal({ "kind" => "slack_channel", "team" => "ops" }, principal.labels) + assert_equal( + { + "kind" => "slack_channel", + "team" => "ops", + Principal::SANDBOX_REPO_CACHE_LABEL => "all" + }, + principal.labels + ) + end + + test "PUT overwrites explicit repo-cache label" do + principal = principals(:acme_channel) + body = { + data: { + labels: { + "kind" => "slack_channel", + Principal::SANDBOX_REPO_CACHE_LABEL => "none" + } + } + } + + put api_v1_principal_url(id: principal.oid), params: body.to_json, headers: auth_headers + assert_response :ok + assert_equal "all", principal.reload.sandbox_repo_cache + assert_equal "all", principal.labels[Principal::SANDBOX_REPO_CACHE_LABEL] + end + + test "PUT replaces Slack channel permission rows" do + principal = principals(:acme_channel) + SlackChannelPermission.create!( + principal: principal, + channel_id: "C1111111111", + upload_enabled: true + ) + body = { + data: { + slack_channel_permissions: [ + { + channel_id: "C0123456789", + upload_enabled: true, + download_enabled: true, + history_enabled: false + }, + { + channel_id: "G9876543210", + upload_enabled: false, + download_enabled: false, + history_enabled: true + } + ] + } + } + + put api_v1_principal_url(id: principal.oid), params: body.to_json, headers: auth_headers + assert_response :ok + + assert_equal( + [ + { + "channel_id" => "C0123456789", + "channel_name" => nil, + "upload_enabled" => true, + "download_enabled" => true, + "history_enabled" => false + }, + { + "channel_id" => "G9876543210", + "channel_name" => nil, + "upload_enabled" => false, + "download_enabled" => false, + "history_enabled" => true + } + ], + principal.reload.slack_channel_permissions_payload + ) + end + + test "PUT rejects a single Slack channel permission object" do + principal = principals(:acme_channel) + body = { + data: { + slack_channel_permissions: { + channel_id: "C0123456789", + upload_enabled: true, + download_enabled: false, + history_enabled: true + } + } + } + + put api_v1_principal_url(id: principal.oid), params: body.to_json, headers: auth_headers + assert_response :unprocessable_content + assert_equal "slack_channel_permissions must be an array", json_body.dig("error", "message") + end + + test "PUT rejects malformed Slack channel permission rows" do + principal = principals(:acme_channel) + body = { data: { slack_channel_permissions: [ "not-an-object" ] } } + + put api_v1_principal_url(id: principal.oid), params: body.to_json, headers: auth_headers + assert_response :unprocessable_content + assert_equal "slack_channel_permissions rows must be objects", json_body.dig("error", "message") + end + + test "PUT can clear Slack channel permission rows" do + principal = principals(:acme_channel) + principal.update!(labels: { Principal::SLACK_CHANNEL_ID_LABEL => "C0123456789" }) + SlackChannelPermission.create!( + principal: principal, + channel_id: "C0123456789", + upload_enabled: true, + download_enabled: true, + history_enabled: true + ) + body = { data: { slack_channel_permissions: [] } } + + put api_v1_principal_url(id: principal.oid), params: body.to_json, headers: auth_headers + assert_response :ok + + assert_empty principal.reload.slack_channel_permissions + assert_equal [], json_body.dig("data", "slack_channel_permissions") + end + + test "POST upserts one Slack channel permission without replacing other rows" do + principal = principals(:acme_channel) + SlackChannelPermission.create!( + principal: principal, + channel_id: "G9876543210", + upload_enabled: true, + download_enabled: false, + history_enabled: false + ) + body = { + data: { + channel_id: "C0123456789", + channel_name: "general", + upload_enabled: true, + download_enabled: true, + history_enabled: true + } + } + + post "/api/v1/principals/#{principal.oid}/slack_channel_permissions", + params: body.to_json, + headers: auth_headers + assert_response :created + + assert_equal( + [ "C0123456789", "G9876543210" ], + principal.reload.slack_channel_permissions.ordered.pluck(:channel_id) + ) + assert_equal "general", json_body.dig("data", "channel_name") + end + + test "POST updates an existing Slack channel permission with normalized channel id" do + principal = principals(:acme_channel) + SlackChannelPermission.create!( + principal: principal, + channel_id: "C0123456789", + channel_name: "general", + upload_enabled: true, + download_enabled: false, + history_enabled: false + ) + body = { + data: { + channel_id: " c0123456789 ", + channel_name: "general", + upload_enabled: false, + download_enabled: true, + history_enabled: true + } + } + + assert_no_difference -> { principal.slack_channel_permissions.count } do + post "/api/v1/principals/#{principal.oid}/slack_channel_permissions", + params: body.to_json, + headers: auth_headers + end + assert_response :ok + + permission = principal.reload.slack_channel_permissions.sole + assert_equal "C0123456789", permission.channel_id + assert_not permission.upload_enabled + assert_predicate permission, :download_enabled + assert_predicate permission, :history_enabled + end + + test "POST retries after concurrent Slack channel permission create wins" do + principal = principals(:acme_channel) + body = { + data: { + channel_id: "C0123456789", + channel_name: "new-name", + upload_enabled: false, + download_enabled: true, + history_enabled: false + } + } + calls = 0 + original = Api::V1::PrincipalsController.instance_method(:save_slack_channel_permission!) + + Api::V1::PrincipalsController.define_method(:save_slack_channel_permission!) do |target_principal, attrs| + calls += 1 + if calls == 1 + target_principal.slack_channel_permissions.create!( + channel_id: attrs[:channel_id], + channel_name: "winner", + upload_enabled: true, + download_enabled: false, + history_enabled: true + ) + raise ActiveRecord::RecordNotUnique, "duplicate key value violates unique constraint" + end + + original.bind_call(self, target_principal, attrs) + end + Api::V1::PrincipalsController.send(:private, :save_slack_channel_permission!) + + assert_difference -> { principal.slack_channel_permissions.count } => 1 do + post "/api/v1/principals/#{principal.oid}/slack_channel_permissions", + params: body.to_json, + headers: auth_headers + end + assert_response :ok + + permission = principal.reload.slack_channel_permissions.sole + assert_equal "C0123456789", permission.channel_id + assert_equal "new-name", permission.channel_name + assert_not permission.upload_enabled + assert_predicate permission, :download_enabled + assert_not permission.history_enabled + assert_equal 1, calls + ensure + Api::V1::PrincipalsController.define_method(:save_slack_channel_permission!, original) + Api::V1::PrincipalsController.send(:private, :save_slack_channel_permission!) + end + + test "POST upserts one Slack DM permission" do + principal = principals(:acme_user_bob) + body = { + data: { + channel_id: "D0123456789", + channel_name: "U0123456789" + } + } + + post "/api/v1/principals/#{principal.oid}/slack_channel_permissions", + params: body.to_json, + headers: auth_headers + assert_response :created + + permission = principal.reload.slack_channel_permissions.sole + assert_equal "D0123456789", permission.channel_id + assert_equal "U0123456789", permission.channel_name + assert_predicate permission, :upload_enabled + assert_predicate permission, :download_enabled + assert_predicate permission, :history_enabled end test "PUT ignores attempts to change immutable namespace and foreign_id" do @@ -192,6 +578,11 @@ def json_body end test "PUT upserts a new principal by foreign_id" do + system_settings(:default).update!( + default_sandbox_repo_cache: "public", + default_sandbox_observability_enabled: false, + default_sandbox_api_server_enabled: false + ) body = { data: { namespace: "acme", name: "Upserted" } } assert_difference -> { Principal.count } => 1 do put api_v1_principal_url(id: "U-upsert"), params: body.to_json, headers: auth_headers @@ -202,6 +593,9 @@ def json_body assert_equal "acme", data["namespace"] assert_equal "U-upsert", data["foreign_id"] assert_equal "Upserted", data["name"] + assert_equal "public", data["sandbox_repo_cache"] + assert_equal false, data["sandbox_observability_enabled"] + assert_equal false, data["sandbox_api_server_enabled"] end test "PUT by foreign_id updates an existing principal without creating" do @@ -246,6 +640,16 @@ def json_body assert_equal %w[U-alice U-bob].sort, foreign_ids.sort end + test "GET index filters by sandbox repo-cache label" do + get api_v1_principals_url, + params: { namespace: "acme", labels: { Principal::SANDBOX_REPO_CACHE_LABEL => "all" } }, + headers: auth_headers + assert_response :ok + + foreign_ids = json_body.fetch("data").map { |p| p["foreign_id"] } + assert_equal %w[C0123456789 U-alice U-bob].sort, foreign_ids.sort + end + test "GET index ANDs multiple label filters" do get api_v1_principals_url, params: { namespace: "acme", labels: { kind: "user", team: "platform" } }, @@ -362,7 +766,6 @@ def grant_sources_to_acme_channel data = json_body.fetch("data") assert_equal principal.oid, data["id"] - assert_equal "120s", data.dig("proxy", "upstream_response_header_timeout") assert_equal 2, data.fetch("secrets").length assert_kind_of Array, data.fetch("transforms") assert_kind_of Array, data.fetch("postgres") diff --git a/services/console/test/controllers/api/v1/proxy_sync_controller_test.rb b/services/console/test/controllers/api/v1/proxy_sync_controller_test.rb index b82e0b536..d09ca6804 100644 --- a/services/console/test/controllers/api/v1/proxy_sync_controller_test.rb +++ b/services/console/test/controllers/api/v1/proxy_sync_controller_test.rb @@ -43,11 +43,10 @@ def json_body body = json_body assert_match(/\Asha256:[0-9a-f]{64}\z/, body.fetch("config_hash")) - assert_equal "120s", body.dig("proxy", "upstream_response_header_timeout") secrets = body.fetch("secrets") assert_equal 2, secrets.length - # Unsupported top-level fields stay absent so the proxy no-ops on them. + # Omitted top-level fields stay absent so the proxy no-ops on them. refute body.key?("rules") refute body.key?("mcp") refute body.key?("ingest_token") @@ -69,22 +68,6 @@ def json_body refute_includes raw, "s3cr3t-db-pass" end - test "sync overlays managed proxy settings onto cached snapshots" do - legacy_payload = Principal::EMPTY_CONFIG.deep_dup - legacy_payload.delete("proxy") - - PrincipalSyncConfigSnapshot.create!( - principal: @proxy.principal, - principal_cache_version: @proxy.principal.sync_config_cache_version, - payload: legacy_payload - ) - - post api_v1_proxy_sync_url, params: {}.to_json, headers: auth_headers - assert_response :ok - - assert_equal "120s", json_body.dig("proxy", "upstream_response_header_timeout") - end - test "secret changes bump principal cache version and build a new snapshot" do post api_v1_proxy_sync_url, params: {}.to_json, headers: auth_headers assert_response :ok @@ -321,6 +304,61 @@ def json_body assert_equal "PROD_API_KEY", bumped.last end + test "infra role delivers the built-in GitHub broker replacement to sandbox principals" do + admin = users(:acme_admin) + credential = BrokerCredential.create!( + namespace: "acme", + foreign_id: "github-app", + name: "GitHub App installation token", + grant: BrokerCredential::GITHUB_APP_INSTALLATION, + token_endpoint: "https://api.github.com/app/installations/42/access_tokens", + client_id: "12345", + client_secret: "private-key", + access_token: "ghs-live-installation-token", + expires_at: 1.hour.from_now, + last_refresh: Time.current, + created_by: admin + ) + secret = StaticSecret.new( + namespace: "acme", + foreign_id: "infra-github-app", + name: "github-app", + replace_config: { + "proxy_value" => "GITHUB_TOKEN", + "match_headers" => [ "Authorization" ] + }, + labels: { "managed-by" => "centaur" }, + created_by: admin + ) + secret.build_source( + source_type: "token_broker", + config: { + "credential_id" => credential.foreign_id, + "credential_namespace" => credential.namespace + } + ) + secret.rules.build(host: "github.com", position: 0) + secret.rules.build(host: "api.github.com", position: 1) + secret.save! + Grant.create!(role: roles(:acme_infra), static_secret: secret, created_by: admin) + + assert_equal secret, secret.source.static_secret + assert_includes @proxy.principal.roles, roles(:acme_infra) + assert_includes @proxy.principal.granted_static_secrets, secret + + post api_v1_proxy_sync_url, params: {}.to_json, headers: auth_headers + assert_response :ok + + entry = json_body.fetch("secrets").find do |candidate| + candidate.dig("replace", "proxy_value") == "GITHUB_TOKEN" + end + refute_nil entry + assert_equal "ghs-live-installation-token", entry.dig("source", "value") + assert_equal "control_plane", entry.dig("source", "type") + assert_equal [ "Authorization" ], entry.dig("replace", "match_headers") + assert_equal [ "github.com", "api.github.com" ], entry.fetch("rules").map { |rule| rule.fetch("host") } + end + test "an unassigned proxy syncs an empty config with unassigned status" do unassigned_token = "iprx_#{'c' * 64}" post api_v1_proxy_sync_url, params: {}.to_json, headers: auth_headers(unassigned_token) @@ -329,7 +367,6 @@ def json_body body = json_body assert_equal "unassigned", body.fetch("status") assert_nil body.fetch("principal_id") - assert_equal "120s", body.dig("proxy", "upstream_response_header_timeout") assert_empty body.fetch("secrets") assert_empty body.fetch("transforms") end diff --git a/services/console/test/controllers/console/descopes_controller_test.rb b/services/console/test/controllers/console/descopes_controller_test.rb new file mode 100644 index 000000000..35ef36154 --- /dev/null +++ b/services/console/test/controllers/console/descopes_controller_test.rb @@ -0,0 +1,76 @@ +require "test_helper" + +module Console + # Covers admin self-descope ("view as operator"): who can start it, that admin + # gates and admin chrome disappear while descoped, how it's restored, and the + # self-healing session cleanup when the user is no longer an admin. + class DescopesControllerTest < ActionDispatch::IntegrationTest + def sign_in(user) + post login_url, params: { email: user.email, password: "password123456" } + end + + test "a non-admin cannot descope" do + sign_in users(:member_user) + post console_descope_url + assert_redirected_to console_threads_path + assert_nil flash[:alert] + end + + test "a descoped admin loses admin pages and chrome, and sees the banner" do + sign_in users(:acme_admin) + post console_descope_url + assert_redirected_to console_threads_path + + get console_users_url + assert_redirected_to console_threads_path + assert_nil flash[:alert] + + get console_threads_url + assert_response :ok + assert_select ".console-descope-banner", /Admin permissions paused/ + assert_select ".console-nav-link", text: "Control", count: 0 + assert_select "form[action=?]", console_descope_path do + assert_select "button", text: /Restore admin/ + end + end + + test "restore brings back admin permissions" do + sign_in users(:acme_admin) + post console_descope_url + + delete console_descope_url + assert_redirected_to console_principals_path + + get console_users_url + assert_response :ok + assert_select ".console-descope-banner", count: 0 + end + + test "descope ends automatically when the user is no longer an admin" do + admin = users(:acme_admin) + sign_in admin + post console_descope_url + + admin.update!(admin: false) + get console_threads_url + assert_response :ok + assert_select ".console-descope-banner", count: 0 + end + + test "restore is a no-op redirect when not descoped" do + sign_in users(:acme_admin) + delete console_descope_url + assert_redirected_to console_principals_path + end + + test "the account menu offers descope only to acting admins" do + sign_in users(:acme_admin) + get console_threads_url + assert_select ".console-signout-label", text: "View as operator" + + sign_in users(:member_user) + get console_threads_url + assert_select ".console-signout-label", text: "View as operator", count: 0 + end + end +end diff --git a/services/console/test/controllers/console/etls_controller_test.rb b/services/console/test/controllers/console/etls_controller_test.rb index 885a77d42..8fdb3108e 100644 --- a/services/console/test/controllers/console/etls_controller_test.rb +++ b/services/console/test/controllers/console/etls_controller_test.rb @@ -60,7 +60,22 @@ def delete_slack_archive_import(import_id) assert_redirected_to login_path end - test "renders Slack archive imports on the ETLs page" do + test "an active non-admin is redirected away from the Data Sync page" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + get console_etls_url + assert_redirected_to console_threads_path + assert_nil flash[:alert] + # The gate fires before the action, so the api client is never touched. + assert_empty @client.calls + + post console_slack_archive_imports_url, params: { filename: "export.zip" } + assert_redirected_to console_threads_path + assert_empty @client.calls + end + + test "renders Slack archive imports on the Data Sync page" do @client.imports = [ { "import_id" => "sai_uploaded", @@ -85,8 +100,8 @@ def delete_slack_archive_import(import_id) get console_etls_url assert_response :ok - assert_select "h1", text: "ETLs" - assert_select "nav a[href=?]", console_etls_path, text: "ETLs" + assert_select "h1", text: "Data Sync" + assert_select "nav a[href=?]", console_etls_path, text: "Data Sync" assert_select "td", text: /export\.zip/ assert_select "th", text: "Workspace", count: 0 assert_select "span", text: "uploaded" diff --git a/services/console/test/controllers/console/integrations_controller_test.rb b/services/console/test/controllers/console/integrations_controller_test.rb new file mode 100644 index 000000000..d20b96b0c --- /dev/null +++ b/services/console/test/controllers/console/integrations_controller_test.rb @@ -0,0 +1,104 @@ +require "test_helper" + +class Console::IntegrationsControllerTest < ActionDispatch::IntegrationTest + test "redirects to login when not signed in" do + get console_integrations_url + assert_redirected_to login_path + end + + test "a non-admin sees enabled apps with their start links, logos, and no disabled apps" do + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + get console_integrations_url + assert_response :ok + + # Enabled apps show up with their consent start links. + %w[google slack github].each do |slug| + assert_select "a[href=?]", "http://www.example.com/oauth/#{slug}/start" + end + # Disabled apps are hidden. + assert_no_match "google-disabled", response.body + + # Known providers render a brand logo (inline SVG). + assert_select "svg path[fill='#4285F4']" # Google + assert_select "svg path[fill='#E01E5A']" # Slack + end + + test "an app already connected under the user's email shows Reconnect and its status" do + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + credential = BrokerCredential.create!( + oauth_app: oauth_apps(:acme_google), + namespace: "acme", + foreign_id: "google-google-member-sub", + name: "Google – Member", + token_endpoint: "https://oauth2.googleapis.com/token", + client_id: "google-client-id", + provider_subject: "member-sub", + provider_email: users(:member_user).email, + external_user_key: "member-key" + ) + + get console_integrations_url + assert_response :ok + assert_select "a.btn-secondary[href=?]", "http://www.example.com/oauth/google/start", text: "Reconnect" + assert_match "Connected", response.body + # The other apps are still unconnected. + assert_select "a.btn-primary[href=?]", "http://www.example.com/oauth/slack/start", text: "Connect" + + # A dead credential asks the user to reconnect rather than claiming success. + credential.update!(dead: true, dead_reason: "invalid_grant") + get console_integrations_url + assert_match "Needs reconnecting", response.body + end + + test "a credential the user minted shows connected even when the provider email differs" do + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + BrokerCredential.create!( + oauth_app: oauth_apps(:acme_google), + namespace: "acme", + foreign_id: "google-google-personal-sub", + name: "Google – Personal", + token_endpoint: "https://oauth2.googleapis.com/token", + client_id: "google-client-id", + provider_subject: "personal-sub", + provider_email: "personal@gmail.example", + external_user_key: "personal-key", + created_by: users(:member_user) + ) + + get console_integrations_url + assert_response :ok + assert_select "a.btn-secondary[href=?]", "http://www.example.com/oauth/google/start", text: "Reconnect" + end + + test "a credential minted for someone else's email does not mark the app connected" do + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + BrokerCredential.create!( + oauth_app: oauth_apps(:acme_google), + namespace: "acme", + foreign_id: "google-google-other-sub", + name: "Google – Other", + token_endpoint: "https://oauth2.googleapis.com/token", + client_id: "google-client-id", + provider_subject: "other-sub", + provider_email: users(:acme_admin).email, + external_user_key: "other-key" + ) + + get console_integrations_url + assert_response :ok + assert_select "a.btn-primary[href=?]", "http://www.example.com/oauth/google/start", text: "Connect" + assert_no_match "Reconnect", response.body + end + + test "an admin sees the same page" do + post login_url, params: { email: users(:acme_admin).email, password: "password123456" } + + get console_integrations_url + assert_response :ok + assert_select "a[href=?]", "http://www.example.com/oauth/google/start" + end +end diff --git a/services/console/test/controllers/console/principals_controller_test.rb b/services/console/test/controllers/console/principals_controller_test.rb index f4c6f8dbe..47f9b39f9 100644 --- a/services/console/test/controllers/console/principals_controller_test.rb +++ b/services/console/test/controllers/console/principals_controller_test.rb @@ -17,20 +17,202 @@ class PrincipalsControllerTest < ActionDispatch::IntegrationTest assert_redirected_to login_path end - test "update_sandbox_access toggles repo cache and observability access" do + test "new renders the create form" do + get console_new_principal_url + assert_response :ok + assert_select "form[action=?][method=?]", console_create_principal_path, "post" do + assert_select "input[name='principal[namespace]'][value=default]" + assert_select "input[name='principal[foreign_id]']" + assert_select "input[name='principal[name]']" + assert_select "button", "Add label" + assert_select "input[type=submit][value='Add Principal']" + end + end + + test "create persists a principal and redirects to its detail page" do + system_settings(:default).update!( + default_sandbox_repo_cache: "public", + default_sandbox_observability_enabled: false, + default_sandbox_api_server_enabled: false + ) + + assert_difference -> { Principal.count }, 1 do + post console_create_principal_url, + params: { + principal: { namespace: "acme", foreign_id: "C-new-console", name: "New console principal" }, + labels: { + "0" => { key: "kind", value: "slack_channel" }, + "1" => { key: "team", value: "platform" } + } + } + end + + principal = Principal.find_by!(namespace: "acme", foreign_id: "C-new-console") + assert_redirected_to console_principal_path(principal.oid) + assert_equal "Principal created.", flash[:notice] + assert_equal "New console principal", principal.name + assert_equal( + { + "kind" => "slack_channel", + "team" => "platform", + Principal::SANDBOX_REPO_CACHE_LABEL => "public" + }, + principal.labels + ) + assert_equal "public", principal.sandbox_repo_cache + assert_equal false, principal.sandbox_observability_enabled + assert_equal false, principal.sandbox_api_server_enabled + assert_equal @operator, principal.created_by + end + + test "create re-renders validation errors" do + existing = principals(:acme_channel) + + assert_no_difference -> { Principal.count } do + post console_create_principal_url, + params: { + principal: { namespace: existing.namespace, foreign_id: existing.foreign_id, name: "Duplicate" } + } + end + + assert_response :unprocessable_entity + assert_select ".alert-error", text: /Principal could not be saved/ + assert_select ".field-error", text: /has already been taken/ + end + + test "update_sandbox_access toggles sandbox capabilities" do principal = principals(:acme_user_bob) patch console_principal_sandbox_access_url(principal.oid), params: { - sandbox_repo_cache_enabled: "0", - sandbox_observability_enabled: "0" + sandbox_repo_cache: "public", + sandbox_observability_enabled: "0", + sandbox_api_server_enabled: "0" } assert_redirected_to console_principal_path(principal.oid) assert_equal "Updated sandbox access.", flash[:notice] principal.reload - assert_equal false, principal.sandbox_repo_cache_enabled + assert_equal "public", principal.sandbox_repo_cache + assert_equal "public", principal.labels[Principal::SANDBOX_REPO_CACHE_LABEL] assert_equal false, principal.sandbox_observability_enabled + assert_equal false, principal.sandbox_api_server_enabled + end + + test "update_slack_channel_permissions stores selected Slack channel permissions" do + principal = principals(:acme_user_bob) + + patch console_principal_slack_channel_permissions_url(principal.oid), + params: { + principal: { + slack_channel_permissions_attributes: { + "0" => { + channel_id: "C0123456789", + upload_enabled: "1", + download_enabled: "0", + history_enabled: "1" + }, + "1" => { + channel_id: "G9876543210", + upload_enabled: "0", + download_enabled: "1", + history_enabled: "0" + } + } + } + } + + assert_redirected_to console_principal_path(principal.oid) + assert_equal( + [ + { + "channel_id" => "C0123456789", + "channel_name" => nil, + "upload_enabled" => true, + "download_enabled" => false, + "history_enabled" => true + }, + { + "channel_id" => "G9876543210", + "channel_name" => nil, + "upload_enabled" => false, + "download_enabled" => true, + "history_enabled" => false + } + ], + principal.reload.slack_channel_permissions_payload + ) + end + + test "update_slack_channel_permissions clears stale channel names when changing channels" do + principal = principals(:acme_user_bob) + permission = SlackChannelPermission.create!( + principal: principal, + channel_id: "C0123456789", + channel_name: "old-channel", + upload_enabled: true, + download_enabled: true, + history_enabled: true + ) + + patch console_principal_slack_channel_permissions_url(principal.oid), + params: { + principal: { + slack_channel_permissions_attributes: { + "0" => { + id: permission.id, + channel_id: "G9876543210", + channel_name: "", + upload_enabled: "1", + download_enabled: "1", + history_enabled: "1" + } + } + } + } + + assert_redirected_to console_principal_path(principal.oid) + permission.reload + assert_equal "G9876543210", permission.channel_id + assert_nil permission.channel_name + end + + test "destroy deletes the principal and dependent access records" do + principal = principals(:acme_channel) + proxy = proxies(:acme_proxy) + client = McpOauthClient.create!(redirect_uris: [ "http://localhost/callback" ]) + McpOauthAuthorizationCode.create!( + mcp_oauth_client: client, + user: users(:acme_admin), + principal: principal, + redirect_uri: "http://localhost/callback", + code_challenge: "challenge", + resource: "https://api.example.test", + scopes: %w[mcp:tools] + ) + McpOauthRefreshToken.create!( + mcp_oauth_client: client, + user: users(:acme_admin), + principal: principal, + resource: "https://api.example.test", + scopes: %w[mcp:tools] + ) + + assert_difference -> { Principal.count }, -1 do + assert_difference -> { Grant.where(principal: principal).count }, -3 do + assert_difference -> { PrincipalRole.where(principal: principal).count }, -1 do + assert_difference -> { McpOauthAuthorizationCode.where(principal: principal).count }, -1 do + assert_difference -> { McpOauthRefreshToken.where(principal: principal).count }, -1 do + delete console_delete_principal_url(principal.oid) + end + end + end + end + end + + assert_redirected_to console_principals_path + assert_equal "Deleted principal #{principal.foreign_id}.", flash[:notice] + assert_nil proxy.reload.principal end test "assign_role attaches the role and redirects with a notice" do diff --git a/services/console/test/controllers/console/system_settings_controller_test.rb b/services/console/test/controllers/console/system_settings_controller_test.rb new file mode 100644 index 000000000..ee5583c48 --- /dev/null +++ b/services/console/test/controllers/console/system_settings_controller_test.rb @@ -0,0 +1,52 @@ +require "test_helper" + +module Console + class SystemSettingsControllerTest < ActionDispatch::IntegrationTest + def sign_in(user) + post login_url, params: { email: user.email, password: "password123456" } + end + + test "redirects to login when signed out" do + get edit_console_system_settings_url + assert_redirected_to login_path + end + + test "non-admin users cannot edit settings" do + sign_in users(:member_user) + get edit_console_system_settings_url + assert_redirected_to console_threads_path + end + + test "admin can edit system settings" do + sign_in users(:acme_admin) + + get edit_console_system_settings_url + assert_response :ok + + assert_select ".console-control-tab-active", text: "Settings" + assert_select "select[name='system_setting[default_sandbox_repo_cache]']" + assert_select "input[name='system_setting[default_sandbox_observability_enabled]']" + assert_select "input[name='system_setting[default_sandbox_api_server_enabled]']" + end + + test "admin updates default sandbox capabilities" do + sign_in users(:acme_admin) + + patch console_system_settings_url, + params: { + system_setting: { + default_sandbox_repo_cache: "public", + default_sandbox_observability_enabled: "0", + default_sandbox_api_server_enabled: "0" + } + } + + assert_redirected_to edit_console_system_settings_path + assert_equal "System settings updated.", flash[:notice] + settings = system_settings(:default).reload + assert_equal "public", settings.default_sandbox_repo_cache + assert_equal false, settings.default_sandbox_observability_enabled + assert_equal false, settings.default_sandbox_api_server_enabled + end + end +end diff --git a/services/console/test/controllers/console/threads_controller_test.rb b/services/console/test/controllers/console/threads_controller_test.rb new file mode 100644 index 000000000..14e58c8ba --- /dev/null +++ b/services/console/test/controllers/console/threads_controller_test.rb @@ -0,0 +1,1580 @@ +require "test_helper" +require "tmpdir" + +class Console::ThreadsControllerTest < ActionDispatch::IntegrationTest + TranscriptMessage = Struct.new(:role, :parts_array, :metadata_hash, :created_at, keyword_init: true) + TranscriptSession = Struct.new(:metadata_hash, :harness_type, :title, keyword_init: true) + ModelSession = Struct.new(:thread_key, :metadata_hash, :harness_type, keyword_init: true) + ModelExecution = Struct.new(:metadata, keyword_init: true) + TranscriptEvent = Struct.new(:event_type, :payload_hash, :created_at, keyword_init: true) + SelectedSession = Struct.new(:thread_key, keyword_init: true) + + setup do + @operator = users(:acme_admin) + post login_url, params: { email: @operator.email, password: "password123456" } + end + + test "an admin sees the Control and Data Sync nav items" do + with_recent_first_error do + get console_threads_url + end + + assert_response :ok + assert_select ".console-nav-link", text: "Control" + assert_select ".console-nav-link", text: "Data Sync" + end + + test "a non-admin sees only the Integrations nav item, not Control or Data Sync" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + with_recent_first_error do + get console_threads_url + end + + assert_response :ok + assert_select ".console-nav-link", count: 1, text: /Integrations/ + assert_select ".console-thread-group-title", text: /Chats/ + end + + test "threads page falls back to the new chat screen when session database is unavailable" do + with_recent_first_error do + get console_threads_url + end + + assert_response :ok + # No chat selected: the new-chat composer renders (posting goes through + # the API, not the sessions DB), alongside the unavailability note. + assert_select ".console-thread-detail-header", count: 0 + assert_select "a[aria-label=?]", "New chat", count: 1 + assert_select "textarea[name=prompt]", count: 1 + assert_select "body", text: /Chat database is unavailable/ + end + + test "plain threads page redirects to first visible thread" do + skip_unless_session_table + + thread_key = "console:auto-select-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + + get console_threads_url + + assert_redirected_to console_threads_path(thread: thread_key) + end + + test "direct selected thread renders chat not found when the current user did not start it" do + skip_unless_session_table + + thread_key = "slack:C0DIRECT:#{SecureRandom.hex(6)}" + insert_slack_session( + thread_key, + slack_user_id: "U_OTHER", + slack_user_name: "someone-else" + ) + + # @operator has no Slack OAuth credential matching U_OTHER, so this thread is + # outside their owner scope. A direct ?thread= link must render a 404 chat + # not found state instead of surfacing it or falling back to another chat. + get console_threads_url(thread: thread_key) + + assert_response :not_found + assert_select "body", text: /Chat not found/ + # The not-found rendering carries no page header and no explainer copy — + # just the centered "Chat not found" state. + assert_select ".console-thread-detail-header", count: 0 + assert_select "body", text: /may not exist/, count: 0 + assert_select "[data-thread-panel]", count: 0 + assert_select ".console-thread-list a.console-thread-link-active[href=?]", + console_threads_path(thread: thread_key), + count: 0 + end + + test "direct link to a nonexistent thread renders chat not found" do + skip_unless_session_table + + # Even with an owned chat present, a bogus key must 404 rather than fall + # back to the first visible chat. + insert_console_session("console:owned-#{SecureRandom.hex(6)}") + + get console_threads_url(thread: "console:missing-#{SecureRandom.hex(6)}") + + assert_response :not_found + assert_select "body", text: /Chat not found/ + end + + test "slack assistant-role messages from the current Slack user render as user authored" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [ "u123" ] } + controller.instance_variable_set( + :@selected_session, + TranscriptSession.new( + metadata_hash: { + "slack_user_id" => "U123", + "slack_display_name" => "Goksu Toprak", + "slack_user_name" => "goksu" + } + ) + ) + message = TranscriptMessage.new( + role: "assistant", + parts_array: [ { "type" => "text", "text" => "Root Slack bot post" } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "slack_user_id" => "U123", + "slack_display_name" => "U123" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + + item = controller.send(:transcript_item_for_message, message) + + assert_equal "assistant", item[:role] + assert_equal "Goksu Toprak", item[:label] + assert_equal :end, item[:align] + assert_equal "Root Slack bot post", item[:text] + end + + test "slack message text resolves mentions from bot identity and selected actor metadata" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [ "u123" ] } + controller.instance_variable_set( + :@selected_session, + TranscriptSession.new( + metadata_hash: { + "slack_user_id" => "U123", + "slack_display_name" => "Goksu Toprak", + "slack_user_name" => "goksu" + } + ) + ) + message = TranscriptMessage.new( + role: "user", + parts_array: [ + { + "type" => "text", + "text" => "@UBOT Are you working? Also loop in <@U123>." + } + ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "is_mention" => true, + "slack_user_id" => "U123", + "slack_display_name" => "Goksu Toprak", + "slack_user_name" => "goksu" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + controller.instance_variable_set(:@selected_messages, [ message ]) + controller.instance_variable_set(:@selected_events, []) + + item = controller.send(:transcript_item_for_message, message) + + assert_equal "@ai Are you working? Also loop in @goksu.", item[:text] + end + + test "slack mention resolution prefers synced user names when available" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [] } + controller.define_singleton_method(:slack_user_display_labels_from_database) do |_user_ids| + { "u456" => "@alice" } + end + message = TranscriptMessage.new( + role: "user", + parts_array: [ { "type" => "text", "text" => "cc @U456" } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "slack_user_id" => "U123" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + controller.instance_variable_set(:@selected_session, TranscriptSession.new(metadata_hash: {})) + controller.instance_variable_set(:@selected_messages, [ message ]) + controller.instance_variable_set(:@selected_events, []) + + item = controller.send(:transcript_item_for_message, message) + + assert_equal "cc @alice", item[:text] + end + + test "slack messages from other actors keep their author label" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [ "u123" ] } + controller.instance_variable_set( + :@selected_session, + TranscriptSession.new(metadata_hash: { "slack_user_id" => "U123" }) + ) + message = TranscriptMessage.new( + role: "user", + parts_array: [ { "type" => "text", "text" => "Another person replied" } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "slack_user_id" => "U456", + "slack_display_name" => "Alice" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + + item = controller.send(:transcript_item_for_message, message) + + assert_equal "Alice", item[:label] + assert_equal :start, item[:align] + end + + test "slack messages from selected thread owner still show author when not current Slack user" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [ "u999" ] } + controller.define_singleton_method(:slack_mention_labels_by_id) { { "u123" => "@goksu" } } + controller.instance_variable_set( + :@selected_session, + TranscriptSession.new( + metadata_hash: { + "slack_user_id" => "U123", + "slack_display_name" => "Goksu Toprak", + "slack_user_name" => "goksu" + } + ) + ) + message = TranscriptMessage.new( + role: "user", + parts_array: [ { "type" => "text", "text" => "Owner message in a direct linked thread" } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "slack_user_id" => "U123", + "slack_display_name" => "U123" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + + item = controller.send(:transcript_item_for_message, message) + + assert_equal "@goksu", item[:label] + assert_equal :start, item[:align] + end + + test "slack bot messages use configured bot username as author label" do + controller = Console::ThreadsController.new + controller.define_singleton_method(:current_slack_user_ids) { [] } + mention = TranscriptMessage.new( + role: "user", + parts_array: [ { "type" => "text", "text" => "@UBOT Please check this." } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "is_mention" => true, + "slack_user_id" => "U123" + }, + created_at: Time.zone.parse("2026-06-26 17:15:58 UTC") + ) + bot_message = TranscriptMessage.new( + role: "user", + parts_array: [ { "type" => "text", "text" => "Working on it." } ], + metadata_hash: { + "source" => "slackbotv2", + "platform" => "slack", + "slack_user_id" => "UBOT", + "slack_display_name" => "UBOT" + }, + created_at: Time.zone.parse("2026-06-26 17:16:58 UTC") + ) + controller.instance_variable_set(:@selected_session, TranscriptSession.new(metadata_hash: {})) + controller.instance_variable_set(:@selected_messages, [ mention, bot_message ]) + controller.instance_variable_set(:@selected_events, []) + + item = controller.send(:transcript_item_for_message, bot_message) + + assert_equal "@ai", item[:label] + assert_equal :start, item[:align] + end + + test "terminal execution events render as bot output" do + controller = Console::ThreadsController.new + event = TranscriptEvent.new( + event_type: "session.execution_completed", + payload_hash: { "result_text" => "The issue is real for @U123." }, + created_at: Time.zone.parse("2026-06-26 17:16:44 UTC") + ) + controller.define_singleton_method(:slack_user_display_labels_from_database) do |_user_ids| + { "u123" => "@goksu" } + end + controller.instance_variable_set(:@selected_session, TranscriptSession.new(metadata_hash: {})) + controller.instance_variable_set(:@selected_messages, []) + controller.instance_variable_set(:@selected_events, [ event ]) + + item = controller.send(:transcript_item_for_event, event) + + assert_equal "assistant", item[:role] + assert_equal "@ai", item[:label] + assert_equal :start, item[:align] + assert_equal "The issue is real for @goksu.", item[:text] + end + + test "generated thread title strips slack mentions and clips to assistant title length" do + controller = Console::ThreadsController.new + title = controller.send( + :generated_thread_title, + "@U0ANX3AM5RR Approach truth-seeking to max and let me know if this is actually " \ + "a legit issue with extra context that should not fit" + ) + + assert_not_includes title, "@U0ANX3AM5RR" + assert title.start_with?("Approach truth-seeking") + assert_operator title.length, :<=, 80 + assert title.end_with?("...") + end + + test "thread title prefers the stored generated title over metadata" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "summary" => { "title" => "metadata title" } }, + harness_type: "codex", + title: "Fix worker memory leak" + ) + + assert_equal "Fix worker memory leak", controller.send(:thread_title, session) + end + + test "thread title ignores a blank stored title" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "subject" => "Fallback subject" }, + harness_type: "codex", + title: " " + ) + + assert_equal "Fallback subject", controller.send(:thread_title, session) + end + + test "thread title prefers stored summary metadata when present" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "summary" => { "title" => "Investigate rollout failure" } }, + harness_type: "codex" + ) + + assert_equal "Investigate rollout failure", controller.send(:thread_title, session) + end + + test "thread title tolerates a plain string summary without raising" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "summary" => "a plain string" }, + harness_type: "codex" + ) + + assert_nothing_raised do + assert_equal "a plain string", controller.send(:thread_title, session) + end + end + + test "thread title tolerates a string thread metadata without raising" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "thread" => "x", "subject" => "Fallback subject" }, + harness_type: "codex" + ) + + assert_nothing_raised do + assert_equal "Fallback subject", controller.send(:thread_title, session) + end + end + + test "thread source and harness labels are display cased" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "platform" => "slack" }, + harness_type: "codex" + ) + + assert_equal "Slack", controller.send(:thread_source_label, session) + assert_equal "slack", controller.send(:thread_source_icon, session) + assert_equal "Codex", controller.send(:thread_harness_label, session) + end + + test "thread model label prefers the latest execution's recorded model override" do + controller = Console::ThreadsController.new + session = ModelSession.new( + thread_key: "slack:C1:1", + metadata_hash: {}, + harness_type: "claudecode" + ) + execution = ModelExecution.new(metadata: { "model" => "claude-sonnet-4-6" }) + controller.instance_variable_set(:@latest_executions, { "slack:C1:1" => execution }) + + assert_equal "CLAUDE-SONNET-4-6", controller.send(:thread_model_label, session) + end + + test "thread model label reads session metadata before the harness default" do + controller = Console::ThreadsController.new + session = TranscriptSession.new( + metadata_hash: { "model" => "claude-fable-5" }, + harness_type: "claudecode" + ) + + assert_equal "CLAUDE-FABLE-5", controller.send(:thread_model_label, session) + end + + test "thread model label falls back to the deployment's model env override" do + controller = Console::ThreadsController.new + + with_env("CLAUDE_MODEL" => "claude-fable-5", "CODEX_MODEL" => "gpt-6") do + assert_equal "CLAUDE-FABLE-5", controller.send( + :thread_model_label, + TranscriptSession.new(metadata_hash: {}, harness_type: "claudecode") + ) + assert_equal "GPT-6", controller.send( + :thread_model_label, + TranscriptSession.new(metadata_hash: {}, harness_type: "codex") + ) + end + end + + test "thread model label falls back to the models pinned in the harness config files" do + controller = Console::ThreadsController.new + + Dir.mktmpdir do |dir| + FileUtils.mkdir_p(File.join(dir, "claude")) + FileUtils.mkdir_p(File.join(dir, "codex")) + File.write(File.join(dir, "claude", "settings.json"), { model: "claude-baked-1" }.to_json) + File.write(File.join(dir, "codex", "config.toml"), <<~TOML) + model = "gpt-baked-1" + model_reasoning_effort = "low" + TOML + + with_env("CLAUDE_MODEL" => nil, "CODEX_MODEL" => nil, "CENTAUR_HARNESS_CONFIG_DIR" => dir) do + assert_equal "CLAUDE-BAKED-1", controller.send( + :thread_model_label, + TranscriptSession.new(metadata_hash: {}, harness_type: "claudecode") + ) + assert_equal "GPT-BAKED-1", controller.send( + :thread_model_label, + TranscriptSession.new(metadata_hash: {}, harness_type: "codex") + ) + end + end + end + + test "thread model label is nil for harnesses without a fixed default" do + controller = Console::ThreadsController.new + + assert_nil controller.send( + :thread_model_label, + TranscriptSession.new(metadata_hash: {}, harness_type: "amp") + ) + end + + test "visible thread scope matches Slack threads owned by the current user's Slack OAuth record" do + app = oauth_apps(:acme_slack) + app.update!(client_secret: "slack-secret", labels: { "slack_team_id" => "T123" }) + create_slack_oauth_credential( + app, + subject: "UOWNER", + email: @operator.email, + labels: { "slack_team_id" => "T123" } + ) + controller = threads_controller_for(@operator) + + sql = controller.send(:visible_thread_scope).to_sql + + assert_includes sql, "thread_key LIKE 'slack:%'" + assert_includes sql, "metadata ->> 'slack_user_id'" + assert_includes sql, "uowner" + assert_includes sql, "split_part(thread_key, ':', 2)" + assert_includes sql, "t123" + end + + test "visible thread scope keeps current user's console threads without Slack OAuth" do + controller = threads_controller_for(@operator) + sql = controller.send(:visible_thread_scope).to_sql + + assert_includes sql, "thread_key LIKE 'console:%'" + assert_includes sql, @operator.email + refute_includes sql, "slack_user_id" + end + + test "visible thread scope matches Slack threads by user id when the credential has no team" do + app = oauth_apps(:acme_slack) + app.update!(client_secret: "slack-secret", labels: {}) + create_slack_oauth_credential( + app, + subject: "UOWNER", + email: @operator.email, + labels: {} + ) + controller = threads_controller_for(@operator) + + sql = controller.send(:visible_thread_scope).to_sql + + # slackbotv2 threads carry no team (slack:CHANNEL:TS keys, no slack_team_id), + # so a team-less credential still matches on slack_user_id alone; team scoping + # is added only when the credential exposes a team. + assert_includes sql, "thread_key LIKE 'slack:%'" + assert_includes sql, "metadata ->> 'slack_user_id'" + assert_includes sql, "uowner" + refute_includes sql, "split_part(thread_key, ':', 2)" + end + + test "visible thread scope matches Slack threads via the SSO identity without a broker credential" do + UserIdentity.create!( + user: @operator, + provider: "slack", + subject: "USSOONLY", + email: @operator.email, + email_verified: true + ) + controller = threads_controller_for(@operator) + + sql = controller.send(:visible_thread_scope).to_sql + + # The Slack OIDC subject is the workspace user id, so signing in with + # Slack is enough to own the threads slackbotv2 attributed to that id — + # no broker credential required. + assert_includes sql, "thread_key LIKE 'slack:%'" + assert_includes sql, "metadata ->> 'slack_user_id'" + assert_includes sql, "ussoonly" + refute_includes sql, "split_part(thread_key, ':', 2)" + end + + test "visible thread scope dedupes an SSO identity that matches its broker credential" do + app = oauth_apps(:acme_slack) + app.update!(client_secret: "slack-secret", labels: {}) + create_slack_oauth_credential(app, subject: "UOWNER", email: @operator.email, labels: {}) + UserIdentity.create!( + user: @operator, + provider: "slack", + subject: "UOWNER", + email: @operator.email, + email_verified: true + ) + controller = threads_controller_for(@operator) + + owners = controller.send(:slack_thread_owners_for_current_user) + + assert_equal [ "UOWNER" ], owners.map { |owner| owner.user_id.upcase } + end + + test "sidebar thread scope matches Slack threads via the SSO identity without a broker credential" do + UserIdentity.create!( + user: @operator, + provider: "slack", + subject: "USSOONLY", + email: @operator.email, + email_verified: true + ) + controller = threads_controller_for(@operator) + + sql = controller.send(:console_sidebar_visible_thread_scope).to_sql + + assert_includes sql, "thread_key LIKE 'slack:%'" + assert_includes sql, "metadata ->> 'slack_user_id'" + assert_includes sql, "ussoonly" + end + + test "selected session resolves a directly linked thread only within the owner scope" do + controller = Console::ThreadsController.new + owned_thread = SelectedSession.new(thread_key: "slack:C123:1782339173.755169") + scoped_relation = Object.new + scoped_relation.define_singleton_method(:where) do |thread_key:| + thread_key == owned_thread.thread_key ? [ owned_thread ] : [] + end + controller.instance_variable_set(:@starting_new_thread, false) + controller.instance_variable_set(:@sessions, []) + + # An owned key outside the base window is recovered through the scope. + controller.instance_variable_set(:@selected_thread_key, owned_thread.thread_key) + assert_equal owned_thread, controller.send(:selected_session, scoped_relation, []) + + # A key the scope does not own has no unscoped fallback, so it stays hidden. + controller.instance_variable_set(:@selected_thread_key, "slack:C999:1782339173.999999") + assert_nil controller.send(:selected_session, scoped_relation, []) + end + + test "renders the sidebar New chat link and the full-page composer" do + with_composer do + with_recent_first_error do + get console_threads_url(new: 1) + end + end + + assert_response :ok + assert_select "a[aria-label=?]", "New chat", count: 1 + assert_select "form[action=?]", console_threads_path do + assert_select "textarea[name=prompt]", count: 1 + # The model picker is a custom menu (account-dropdown style) posting + # through a hidden field, not a native select. + assert_select "input[type=hidden][name=model]", count: 1 + assert_select "[data-console-model-option][data-value=?]", "amp" + assert_select "select", count: 0 + end + # Submitting replaces the centered empty state with a full-height, + # bottom-aligned optimistic transcript while the request is in flight. + assert_includes response.body, 'container.classList.add("console-new-chat--optimistic")' + assert_includes response.body, ".console-new-chat--optimistic" + end + + test "shows the new chat screen when nothing is selected" do + with_composer do + with_recent_first_error do + get console_threads_url + end + end + + assert_response :ok + assert_select "textarea[name=prompt]", count: 1 + assert_select "body", text: /No chats yet/, count: 0 + end + + test "an active execution renders a thinking indicator" do + skip_unless_session_table + insert_console_session("console:thinking-active") + insert_session_execution("console:thinking-active", status: "running") + + get console_threads_url(thread: "console:thinking-active") + + assert_response :ok + assert_select "[data-console-thinking-indicator]", count: 1 + end + + test "a completed execution renders no thinking indicator" do + skip_unless_session_table + insert_console_session("console:thinking-done") + insert_session_execution("console:thinking-done", status: "completed") + + get console_threads_url(thread: "console:thinking-done") + + assert_response :ok + assert_select "[data-console-thinking-indicator]", count: 0 + end + + test "a new sentinel pane opens a composer panel alongside a thread" do + skip_unless_session_table + insert_console_session("console:with-new-pane") + + with_composer do + get console_threads_url(thread: "console:with-new-pane,new") + end + + assert_response :ok + assert_select "[data-thread-panel]", count: 2 + assert_select "[data-thread-panel=new]", count: 1 + assert_select "[data-thread-panel=new] textarea[name=prompt]", count: 1 + assert_select "[data-thread-panel=new] [data-console-model-picker]", count: 1 + end + + test "the new sentinel alone renders the full-page new chat screen" do + with_composer do + with_recent_first_error do + get console_threads_url(thread: "new") + end + end + + assert_response :ok + assert_select "[data-thread-panel]", count: 0 + assert_select "textarea[name=prompt]", count: 1 + end + + test "starting a chat from a pane swaps the sentinel for the created thread" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, + params: { + prompt: "Reply with PONG.", + model: "gpt-5.5", + open_threads: "console:other,new" + } + end + + thread_key = client.calls[0].last[:thread_key] + assert_redirected_to console_threads_path(thread: "console:other,#{thread_key}") + end + + test "renders a follow-up composer on an open chat" do + skip_unless_session_table + insert_console_session("console:composer-open") + + with_composer do + get console_threads_url(thread: "console:composer-open") + end + + assert_response :ok + assert_select "form[action=?]", console_threads_path do + assert_select "input[type=hidden][name=thread_key][value=?]", "console:composer-open" + assert_select "textarea[name=prompt]", count: 1 + # Follow-ups stay on the chat's existing harness/model: no picker. + assert_select "[data-console-model-picker]", count: 0 + end + # Optimistic rendering must not clear the textarea until Turbo has copied + # its value into FormData, or the controller receives a blank prompt. + assert_includes response.body, 'form.addEventListener("formdata"' + end + + test "starting a chat creates a session, appends the prompt, and executes it" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, + params: { prompt: "Reply with PONG.", model: "claude-opus-4-8" } + end + + assert_equal %i[create_session append_session_messages execute_session], client.calls.map(&:first) + + create = client.calls[0].last + assert create[:thread_key].start_with?("console:"), "expected a console:-namespaced thread key" + assert_equal "claudecode", create[:harness_type] + assert_equal "console", create[:metadata][:platform] + assert_equal "console", create[:metadata][:source] + assert_equal @operator.email, create[:metadata][:actor_email] + assert_equal "claude-opus-4-8", create[:metadata][:model] + + append = client.calls[1].last + assert_equal create[:thread_key], append[:thread_key] + message = append[:messages].first + assert_equal "user", message[:role] + assert_equal "Reply with PONG.", message[:parts].first[:text] + assert_equal @operator.email, message[:metadata][:user_email] + + execute = client.calls[2].last + assert_equal create[:thread_key], execute[:thread_key] + assert execute[:idempotency_key].present? + assert_equal "claude-opus-4-8", execute[:metadata][:model] + line = JSON.parse(execute[:input_lines].first) + assert_equal "user", line["type"] + assert_equal create[:thread_key], line["thread_key"] + assert_equal "claude-opus-4-8", line["model"] + assert_equal message[:client_message_id], line["client_user_message_id"] + assert_equal "Reply with PONG.", line.dig("message", "content", 0, "text") + + assert_redirected_to console_threads_path(thread: create[:thread_key]) + end + + test "picking Amp starts an amp chat and sends no model" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, params: { prompt: "Reply with PONG.", model: "amp" } + end + + create = client.calls[0].last + assert_equal "amp", create[:harness_type] + assert_not create[:metadata].key?(:model) + + execute = client.calls[2].last + assert_not execute[:metadata].key?(:model) + line = JSON.parse(execute[:input_lines].first) + assert_not line.key?("model") + end + + test "starting a chat with an unknown model is rejected" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, params: { prompt: "Reply with PONG.", model: "hal9000" } + end + + assert_empty client.calls + assert_redirected_to console_threads_path(new: 1) + assert_match(/Unknown model/, flash[:alert]) + end + + test "a gpt model pick starts a codex chat" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, params: { prompt: "Reply with PONG.", model: "gpt-5.5" } + end + + create = client.calls[0].last + assert_equal "codex", create[:harness_type] + assert_equal "gpt-5.5", create[:metadata][:model] + end + + test "a codex chat carries the picked reasoning effort" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, + params: { prompt: "Reply with PONG.", model: "gpt-5.6-sol", effort: "max" } + end + + execute = client.calls[2].last + assert_equal "max", execute[:metadata][:reasoning] + line = JSON.parse(execute[:input_lines].first) + assert_equal "max", line["reasoning"] + end + + test "an effort the model does not offer is dropped" do + client = RecordingApiClient.new + with_composer(client: client) do + # max is 5.6-only; claude models take no effort at all. + post console_threads_url, + params: { prompt: "Reply with PONG.", model: "gpt-5.5", effort: "max" } + post console_threads_url, + params: { prompt: "Reply with PONG.", model: "claude-opus-4-8", effort: "high" } + end + + [ 2, 5 ].each do |index| + execute = client.calls[index].last + assert_not execute[:metadata].key?(:reasoning) + assert_not JSON.parse(execute[:input_lines].first).key?("reasoning") + end + end + + test "a blank prompt asks for a message" do + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, params: { prompt: " " } + end + + assert_empty client.calls + assert_redirected_to console_threads_path(new: 1) + assert_equal "Type a message first.", flash[:alert] + end + + test "replying appends and executes on an owned chat without creating a session" do + skip_unless_session_table + insert_console_session("console:composer-reply") + + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, + params: { + prompt: "Continue from here.", + thread_key: "console:composer-reply", + open_threads: "console:composer-reply,console:other" + } + end + + assert_equal %i[append_session_messages execute_session], client.calls.map(&:first) + assert_equal "console:composer-reply", client.calls[0].last[:thread_key] + assert_redirected_to console_threads_path(thread: "console:composer-reply,console:other") + end + + test "replying into a chat outside the owner scope is rejected" do + skip_unless_session_table + + client = RecordingApiClient.new + with_composer(client: client) do + post console_threads_url, + params: { prompt: "Continue from here.", thread_key: "console:not-mine" } + end + + assert_empty client.calls + assert_redirected_to console_threads_path + assert_equal "Chat not found.", flash[:alert] + end + + test "a session api error surfaces as a flash alert" do + client = RecordingApiClient.new(error: CentaurApiClient::Error.new("boom")) + with_composer(client: client) do + post console_threads_url, params: { prompt: "Reply with PONG.", harness_type: "codex" } + end + + assert_redirected_to console_threads_path(new: 1) + assert_match(/boom/, flash[:alert]) + end + + # Fix 6: the sidebar thread list is loaded lazily via a Turbo Frame so the + # cross-database sessions query never runs during the primary page render. + test "console pages defer the sidebar thread list to a lazy turbo frame" do + # A non-thread page must not run the sessions query during its render: if it + # did, load_console_sidebar_threads would be invoked. Track invocations and + # assert none happen while rendering the primary page. + original = ApplicationController.instance_method(:load_console_sidebar_threads) + Thread.current[:sidebar_loaded] = false + ApplicationController.send(:define_method, :load_console_sidebar_threads) do + Thread.current[:sidebar_loaded] = true + original.bind(self).call + end + + begin + get console_principals_url + + assert_response :ok + assert_not Thread.current[:sidebar_loaded], + "primary page render must not load the sidebar thread list" + assert_select "turbo-frame#console_sidebar_threads[src=?]", console_sidebar_threads_path + assert_select "turbo-frame#console_sidebar_threads[loading=?]", "lazy" + ensure + ApplicationController.send(:define_method, :load_console_sidebar_threads, original) + Thread.current[:sidebar_loaded] = nil + end + end + + test "sidebar action renders the empty thread list when the session DB is unavailable" do + with_recent_first_error do + get console_sidebar_threads_url + end + + assert_response :ok + assert_select "turbo-frame#console_sidebar_threads" + assert_select ".console-thread-empty", text: /No recent chats/ + end + + # Fix 5: selected_messages must return the NEWEST MESSAGE_LIMIT messages, in + # oldest-first display order. A previous ascending order + limit returned the + # oldest N and dropped the newest for long threads. + test "selected_messages query fetches newest messages first with a limit" do + # Building the SQL type-casts against the session_messages schema, which + # only exists where the api-rs session tables are present. + skip_unless_session_table + + relation = CentaurSessionMessage + .where(thread_key: "console:ordering") + .order(created_at: :desc, message_id: :desc) + .limit(Console::ThreadsController::MESSAGE_LIMIT) + sql = relation.to_sql + + assert_match(/ORDER BY.*created_at.*DESC.*message_id.*DESC/i, sql) + assert_match(/LIMIT #{Console::ThreadsController::MESSAGE_LIMIT}\b/, sql) + end + + test "selected_messages returns newest messages in ascending display order" do + skip_unless_session_table + + thread_key = "console:transcript-order" + insert_console_session(thread_key) + + limit = Console::ThreadsController::MESSAGE_LIMIT + total = limit + 5 + total.times do |i| + insert_session_message(thread_key, index: i) + end + + controller = Console::ThreadsController.new + controller.instance_variable_set(:@selected_session, SelectedSession.new(thread_key: thread_key)) + + messages = controller.send(:selected_messages) + + assert_equal limit, messages.size + indices = messages.map { |m| m.message_id.split("-").last.to_i } + # Oldest-first display order over the newest `limit` messages: the earliest + # (index 0..4) are dropped, and what remains is ascending. + assert_equal (total - limit...total).to_a, indices + assert_equal indices, indices.sort + end + + OutputLineEvent = Struct.new(:payload, :created_at, :execution_id, :event_id, keyword_init: true) + + test "thinking transcript item is extracted from a completed reasoning output line" do + controller = Console::ThreadsController.new + line = { + method: "item/completed", + params: { + item: { + type: "reasoning", + content: [ "First I will check the schema.", "Then write the query." ] + } + } + }.to_json + event = OutputLineEvent.new(payload: line, created_at: Time.zone.parse("2026-06-26 17:15:58 UTC")) + + item = controller.send(:thinking_transcript_item, event) + + assert_equal "thinking", item[:role] + assert_equal "Thinking", item[:label] + assert_equal :thinking, item[:source] + assert_equal :start, item[:align] + assert_equal "First I will check the schema.\nThen write the query.", item[:text] + assert_equal event.created_at, item[:created_at] + end + + test "thinking extraction accepts dot-form types and summary-only reasoning" do + controller = Console::ThreadsController.new + line = { + type: "item.completed", + item: { type: "reasoning", summary: [ { text: "Condensed thought." } ] } + }.to_json + event = OutputLineEvent.new(payload: line, created_at: Time.zone.now) + + item = controller.send(:thinking_transcript_item, event) + + assert_equal "Condensed thought.", item[:text] + end + + test "thinking extraction formats completed command execution output lines" do + controller = Console::ThreadsController.new + line = { + method: "item/completed", + params: { + item: { + id: "cmd-1", + type: "commandExecution", + command: "pnpm test", + status: "completed", + aggregatedOutput: "ok\n", + exitCode: 0 + } + } + }.to_json + event = OutputLineEvent.new(payload: line, created_at: Time.zone.parse("2026-06-26 17:15:58 UTC")) + + item = controller.send(:thinking_transcript_item, event) + + assert_equal "thinking", item[:role] + assert_equal "Ran 1 command", item[:label] + assert_equal :thinking, item[:source] + assert_equal "command", item[:trace_kind] + assert_equal 1, item[:commands].length + assert_equal "pnpm test", item[:commands].first[:command] + assert_equal "ok\n", item[:commands].first[:output] + assert_equal 0, item[:commands].first[:exit_code] + assert_not item[:commands].first[:failed] + assert_includes item[:text], "Status: completed" + assert_includes item[:text], "Exit code: 0" + assert_includes item[:text], "```sh\npnpm test\n```" + assert_includes item[:text], "Output:" + assert_includes item[:text], "```text\nok\n```" + end + + test "compact trace grouping combines adjacent command executions for one run" do + controller = Console::ThreadsController.new + now = Time.zone.now + first = { + role: "thinking", + label: "Ran 1 command", + text: "$ pnpm test", + trace_kind: "command", + commands: [ { command: "pnpm test", output: "ok\n", exit_code: 0, status: "completed", failed: false } ], + execution_id: "exe-1", + created_at: now, + source: :thinking + } + second = { + role: "thinking", + label: "Ran 1 command", + text: "$ curl bad", + trace_kind: "command", + commands: [ { command: "curl bad", output: "failed\n", exit_code: 22, status: "completed", failed: true } ], + execution_id: "exe-1", + created_at: now + 1.second, + source: :thinking + } + thought = { + role: "thinking", + label: "Thinking", + text: "Need one more check.", + trace_kind: "thinking", + created_at: now + 2.seconds, + source: :thinking + } + + grouped = controller.send(:compact_trace_items, [ first, second, thought ]) + + assert_equal 2, grouped.length + assert_equal "commands", grouped.first[:trace_kind] + assert_equal "Ran 2 commands", grouped.first[:label] + assert_equal "1 failed", grouped.first[:failed_label] + assert_equal [ "pnpm test", "curl bad" ], grouped.first[:commands].map { |command| command[:command] } + assert_equal thought, grouped.second + end + + test "activity summaries attach to the latest trace item at or before their source line" do + controller = Console::ThreadsController.new + items = [ + { event_id: 10, text: "first" }, + { event_id: 20, text: "second" } + ] + summaries = [ + TranscriptEvent.new(event_type: "session.activity_summary", payload_hash: { "summary" => "I found the bug", "source_event_id" => 9 }), + TranscriptEvent.new(event_type: "session.activity_summary", payload_hash: { "summary" => "I'm reading the schema", "source_event_id" => 11 }), + TranscriptEvent.new(event_type: "session.activity_summary", payload_hash: { "summary" => "I'm writing the query", "source_event_id" => 15 }), + TranscriptEvent.new(event_type: "session.activity_summary", payload_hash: { "summary" => "", "source_event_id" => 21 }) + ] + controller.define_singleton_method(:selected_activity_summaries) { summaries } + + controller.send(:apply_activity_summaries, items) + + # The newest summary in an item's window wins; blank summaries and + # summaries preceding every trace item are dropped. + assert_equal "I'm writing the query", items[0][:summary] + assert_nil items[1][:summary] + end + + test "thinking extraction formats claude stream-json tool calls" do + controller = Console::ThreadsController.new + line = { + type: "assistant", + message: { + content: [ + { type: "tool_use", id: "toolu_1", name: "websearch", input: { query: "centaur" } } + ] + } + }.to_json + event = OutputLineEvent.new(payload: line, created_at: Time.zone.now) + + item = controller.send(:thinking_transcript_item, event) + + assert_equal "Tool call", item[:label] + assert_includes item[:text], "Use websearch" + assert_includes item[:text], '"query": "centaur"' + end + + test "thinking extraction ignores partial and unrelated output lines" do + controller = Console::ThreadsController.new + now = Time.zone.now + + delta = { method: "item/reasoning/textDelta", params: { delta: "partial" } }.to_json + started_tool = { + method: "item/started", + params: { item: { type: "commandExecution", command: "pnpm test" } } + }.to_json + non_json = "plain stdout noise mentioning reasoning" + non_string = { "result" => "reasoning" } + + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: delta, created_at: now)) + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: started_tool, created_at: now)) + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: non_json, created_at: now)) + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: non_string, created_at: now)) + end + + test "thinking transcript item is extracted from a claude stream-json assistant line" do + controller = Console::ThreadsController.new + line = { + type: "assistant", + message: { + role: "assistant", + content: [ + { type: "thinking", thinking: "The schema mismatch explains the failure.", signature: "sig" }, + { type: "text", text: "Here is the fix." } + ] + } + }.to_json + event = OutputLineEvent.new(payload: line, created_at: Time.zone.parse("2026-06-26 17:15:58 UTC")) + + item = controller.send(:thinking_transcript_item, event) + + assert_equal "thinking", item[:role] + assert_equal :thinking, item[:source] + assert_equal "The schema mismatch explains the failure.", item[:text] + assert_equal event.created_at, item[:created_at] + end + + test "thinking extraction joins multiple claude thinking blocks and skips thinking-free assistant lines" do + controller = Console::ThreadsController.new + now = Time.zone.now + + multi = { + type: "assistant", + message: { + content: [ + { type: "thinking", thinking: "First thought." }, + { type: "thinking", thinking: "Second thought." } + ] + } + }.to_json + text_only = { + type: "assistant", + message: { content: [ { type: "text", text: "No thinking here." } ] } + }.to_json + stream_event = { + type: "stream_event", + event: { delta: { type: "thinking_delta", thinking: "partial" } } + }.to_json + + item = controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: multi, created_at: now)) + assert_equal "First thought.\nSecond thought.", item[:text] + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: text_only, created_at: now)) + assert_nil controller.send(:thinking_transcript_item, OutputLineEvent.new(payload: stream_event, created_at: now)) + end + + test "requested thread keys are deduped, stripped, and capped at the panel limit" do + controller = Console::ThreadsController.new + controller.params = ActionController::Parameters.new( + thread: " a , b,a,, c ,d,e " + ) + + assert_equal %w[a b c d], controller.send(:requested_thread_keys) + end + + test "thinking trace renders as a collapsed disclosure in the transcript" do + skip_unless_session_table + + thread_key = "console:thinking-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + insert_session_message(thread_key, index: 0) + insert_reasoning_event(thread_key, text: "I should compare the two schemas before answering.") + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "details.console-thinking summary", text: /Thinking/ + assert_select "details.console-thinking", text: /compare the two schemas/ + end + + test "tool trace renders as a collapsed disclosure in the transcript" do + skip_unless_session_table + + thread_key = "console:tool-trace-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + insert_session_message(thread_key, index: 0) + insert_command_trace_event(thread_key, command: "pnpm test", output: "ok\n") + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "details.console-thinking summary", text: /Ran 1 command/ + assert_select ".console-thinking-command-row", text: /pnpm test/ + assert_select ".console-thinking-command-full", text: /\$ pnpm test/ + assert_select ".console-thinking-command-result", text: /ok/ + assert_select ".console-thinking-command-meta", count: 0 + assert_select "details.console-thinking", text: /Status:/, count: 0 + assert_select "details.console-thinking", text: /pnpm test/ + assert_select "details.console-thinking", text: /ok/ + end + + test "thinking preview shows the activity summary covering its block" do + skip_unless_session_table + + thread_key = "console:activity-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + insert_session_message(thread_key, index: 0) + source_event_id = insert_reasoning_event(thread_key, text: "I should compare the two schemas before answering.") + insert_activity_summary_event( + thread_key, + summary: "I'm comparing the two schemas", + source_event_id: source_event_id + ) + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "details.console-thinking .console-thinking-preview", + text: /I'm comparing the two schemas/ + # The full thinking text stays available in the disclosure body. + assert_select "details.console-thinking", text: /compare the two schemas before answering/ + end + + test "command trace group shows the activity summary as its collapsed preview" do + skip_unless_session_table + + thread_key = "console:activity-cmd-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + insert_session_message(thread_key, index: 0) + source_event_id = insert_command_trace_event(thread_key, command: "pnpm test", output: "ok\n") + insert_activity_summary_event( + thread_key, + summary: "I'm running the test suite", + source_event_id: source_event_id + ) + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "details.console-thinking summary", text: /Ran 1 command/ + assert_select "details.console-thinking .console-thinking-preview", + text: /I'm running the test suite/ + end + + test "split view renders owned panes as panels and drops unowned keys" do + skip_unless_session_table + + primary_key = "console:panel-a-#{SecureRandom.hex(6)}" + pane_key = "console:panel-b-#{SecureRandom.hex(6)}" + unowned_key = "slack:C0PANEL:#{SecureRandom.hex(6)}" + insert_console_session(primary_key) + insert_console_session(pane_key) + insert_slack_session(unowned_key, slack_user_id: "U_OTHER", slack_user_name: "someone-else") + + get console_threads_url(thread: [ primary_key, pane_key, unowned_key ].join(",")) + + assert_response :ok + assert_select "[data-thread-panel]", count: 2 + assert_select "[data-thread-panel=?]", primary_key + assert_select "[data-thread-panel=?]", pane_key + assert_select "[data-thread-panel=?]", unowned_key, count: 0 + # Each panel exposes a close control back to the remaining threads. + assert_select "[data-thread-panel] a[aria-label='Close panel']", count: 2 + end + + test "split view caps the grid at four panels" do + skip_unless_session_table + + keys = Array.new(5) { |i| "console:panel-cap-#{i}-#{SecureRandom.hex(4)}" } + keys.each { |key| insert_console_session(key) } + + get console_threads_url(thread: keys.join(",")) + + assert_response :ok + assert_select "[data-thread-panel]", count: Console::ThreadsController::PANEL_LIMIT + end + + test "single thread view does not render the split grid" do + skip_unless_session_table + + thread_key = "console:solo-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "[data-thread-panel]", count: 0 + # column-reverse scroll container opens the thread at its newest message. + assert_select "#thread-transcript-scroll.console-transcript-scroll" + end + + test "sidebar thread links carry the cmd-click split view hook" do + skip_unless_session_table + + thread_key = "console:sidebar-split-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + + get console_sidebar_threads_url + + assert_response :ok + # The layout's Cmd/Ctrl-click handler targets this attribute to add the + # thread to the split-view grid. + assert_select "a[data-console-thread-link][href=?]", + console_threads_path(thread: thread_key) + end + + # The sidebar list loads out of band via a lazy Turbo Frame, so the page must + # forward the current thread selection on the frame src for the active + # highlight to render. + test "threads page forwards the thread selection to the sidebar frame src" do + skip_unless_session_table + + thread_key = "console:sidebar-active-#{SecureRandom.hex(8)}" + insert_console_session(thread_key) + + get console_threads_url(thread: thread_key) + + assert_response :ok + assert_select "turbo-frame#console_sidebar_threads[src=?]", + console_sidebar_threads_path(thread: thread_key) + end + + test "sidebar highlights every open thread of a split view" do + skip_unless_session_table + + keys = Array.new(2) { |i| "console:sidebar-open-#{i}-#{SecureRandom.hex(4)}" } + keys.each { |key| insert_console_session(key) } + + get console_sidebar_threads_url(thread: keys.join(",")) + + assert_response :ok + # Open threads carry their 1-based pane number in grid order; no filled + # pill on thread rows. + assert_select "a.console-thread-link-open[data-console-pane-index='1'][href=?]", + console_threads_path(thread: keys.first) + assert_select "a.console-thread-link-open[data-console-pane-index='2'][href=?]", + console_threads_path(thread: keys.last) + assert_select "a.console-thread-link-active", count: 0 + end + + test "split view close control drops one thread and keeps the rest open" do + skip_unless_session_table + + keys = Array.new(3) { |i| "console:panel-close-#{i}-#{SecureRandom.hex(4)}" } + keys.each { |key| insert_console_session(key) } + + get console_threads_url(thread: keys.join(",")) + + assert_response :ok + # Closing the middle panel keeps the primary and the last pane. + assert_select "[data-thread-panel=?] a[aria-label='Close panel'][href=?]", + keys[1], + console_threads_path(thread: [ keys[0], keys[2] ].join(",")) + # Closing the primary panel promotes the next thread to primary. + assert_select "[data-thread-panel=?] a[aria-label='Close panel'][href=?]", + keys[0], + console_threads_path(thread: [ keys[1], keys[2] ].join(",")) + end + + private + + # Fake CentaurApiClient recording every composer call; raises `error` from + # each method instead when given, to exercise the failure paths. + class RecordingApiClient + attr_reader :calls + + def initialize(error: nil) + @calls = [] + @error = error + end + + def create_session(**kwargs) = record(:create_session, kwargs) + def append_session_messages(**kwargs) = record(:append_session_messages, kwargs) + def execute_session(**kwargs) = record(:execute_session, kwargs) + + private + + def record(name, kwargs) + raise @error if @error + + @calls << [ name, kwargs ] + {} + end + end + + # Runs the block with the injected fake session client. + def with_composer(client: RecordingApiClient.new) + original_factory = Console::ThreadsController.client_factory + Console::ThreadsController.client_factory = -> { client } + yield client + ensure + Console::ThreadsController.client_factory = original_factory + end + + # Sets each env var for the block (nil deletes) and restores the previous + # values afterwards. + def with_env(overrides) + previous = overrides.keys.index_with { |name| ENV[name] } + overrides.each { |name, value| value.nil? ? ENV.delete(name) : ENV[name] = value } + yield + ensure + previous.each { |name, value| value.nil? ? ENV.delete(name) : ENV[name] = value } + end + + def with_recent_first_error + singleton = class << CentaurSession; self; end + original = CentaurSession.method(:recent_first) + singleton.define_method(:recent_first) { raise ActiveRecord::ConnectionNotEstablished } + yield + ensure + singleton.define_method(:recent_first, original) + end + + def threads_controller_for(user) + Console::ThreadsController.new.tap do |controller| + controller.define_singleton_method(:current_user) { user } + end + end + + def create_slack_oauth_credential(app, subject:, email:, labels: {}) + BrokerCredential.create!( + namespace: app.credential_namespace, + oauth_app: app, + provider_subject: subject, + provider_email: email, + labels: labels, + token_endpoint: app.provider_strategy.token_endpoint, + refresh_token: "refresh-#{subject}", + access_token: "access-#{subject}", + expires_at: 1.hour.from_now, + last_refresh: Time.current, + external_user_key: "user-#{subject}" + ) + end + + def insert_console_session(thread_key) + connection = CentaurSession.connection + metadata = { platform: "console", actor_email: @operator.email }.to_json + insert_session(thread_key, metadata) + end + + def skip_unless_session_table + skip("api-rs session tables are unavailable") unless CentaurSession.connection.data_source_exists?("sessions") + end + + def insert_slack_session(thread_key, slack_user_id:, slack_user_name:) + metadata = { + source: "slackbotv2", + platform: "slack", + thread_id: thread_key, + slack_user_id: slack_user_id, + slack_user_name: slack_user_name + }.to_json + insert_session(thread_key, metadata) + end + + def insert_session_execution(thread_key, status:) + connection = CentaurSession.connection + connection.execute(<<~SQL.squish) + insert into session_executions (execution_id, thread_key, status, metadata, created_at, updated_at) + values ( + #{connection.quote("#{thread_key}-exec")}, + #{connection.quote(thread_key)}, + #{connection.quote(status)}, + '{}'::jsonb, + now(), + now() + ) + SQL + end + + def insert_session_message(thread_key, index:) + connection = CentaurSession.connection + parts = [ { type: "text", text: "message #{index}" } ].to_json + connection.execute(<<~SQL.squish) + insert into session_messages (message_id, thread_key, role, parts, metadata, created_at) + values ( + #{connection.quote("#{thread_key}-msg-#{index}")}, + #{connection.quote(thread_key)}, + 'user', + #{connection.quote(parts)}::jsonb, + '{}'::jsonb, + now() + (#{index} * interval '1 second') + ) + SQL + end + + # Mirrors how api-rs persists harness stdout: the payload column is a + # JSON-encoded *string* holding one protocol notification line. + def insert_reasoning_event(thread_key, text:) + insert_output_line_event( + thread_key, + method: "item/completed", + params: { item: { type: "reasoning", content: [ text ] } } + ) + end + + def insert_command_trace_event(thread_key, command:, output:) + insert_output_line_event( + thread_key, + method: "item/completed", + params: { + item: { + type: "commandExecution", + command: command, + status: "completed", + aggregatedOutput: output, + exitCode: 0 + } + } + ) + end + + def insert_output_line_event(thread_key, method:, params:) + connection = CentaurSession.connection + line = { method: method, params: params }.to_json + connection.select_value(<<~SQL.squish).to_i + insert into session_events (thread_key, event_type, payload, created_at) + values ( + #{connection.quote(thread_key)}, + 'session.output.line', + #{connection.quote(line.to_json)}::jsonb, + now() + ) + returning event_id + SQL + end + + # Mirrors api-rs's activity-summary worker: the payload is a JSON object + # whose source_event_id points at the output line that triggered it. + def insert_activity_summary_event(thread_key, summary:, source_event_id:) + connection = CentaurSession.connection + payload = { summary: summary, source_event_id: source_event_id }.to_json + connection.execute(<<~SQL.squish) + insert into session_events (thread_key, event_type, payload, created_at) + values ( + #{connection.quote(thread_key)}, + 'session.activity_summary', + #{connection.quote(payload)}::jsonb, + now() + ) + SQL + end + + def insert_session(thread_key, metadata) + connection = CentaurSession.connection + connection.execute(<<~SQL.squish) + insert into sessions (thread_key, harness_type, status, metadata, created_at, updated_at) + values ( + #{connection.quote(thread_key)}, + 'codex', + 'active', + #{connection.quote(metadata)}::jsonb, + now() + interval '1 day', + now() + interval '1 day' + ) + SQL + end +end diff --git a/services/console/test/controllers/console/users_controller_test.rb b/services/console/test/controllers/console/users_controller_test.rb index d4bcfc681..e09fa2bc3 100644 --- a/services/console/test/controllers/console/users_controller_test.rb +++ b/services/console/test/controllers/console/users_controller_test.rb @@ -16,8 +16,8 @@ def sign_in(user) test "an active non-admin is forbidden" do sign_in users(:member_user) get console_users_url - assert_redirected_to root_path - assert_equal "That page is restricted to admins.", flash[:alert] + assert_redirected_to console_threads_path + assert_nil flash[:alert] end test "an admin sees the index with pending users listed" do @@ -25,6 +25,16 @@ def sign_in(user) get console_users_url assert_response :ok assert_select "td", /pending@acme.example/ + assert_select ".console-nav-link", text: "Control" + assert_select ".console-nav-link", text: "Apps", count: 0 + assert_select ".console-nav-link", text: "Users", count: 0 + assert_select ".console-control-tab", text: "Apps" + assert_select ".console-control-tab-active", text: "Users" + assert_select "button[data-console-theme-toggle]", text: "Light mode" + assert_select "link[data-console-favicon][href=?]", "/icon-dark.svg" + assert_includes response.body, "/icon-light.svg" + assert_includes response.body, "prefers-color-scheme: light" + assert_includes response.body, "centaur-console-theme-source" end test "the index shows IdP chips for linked identities and a password chip otherwise" do @@ -53,6 +63,30 @@ def sign_in(user) assert target.reload.disabled? end + test "disable revokes outstanding MCP OAuth refresh tokens" do + sign_in users(:acme_admin) + target = users(:member_user) + refresh = McpOauthRefreshToken.create!( + mcp_oauth_client: McpOauthClient.create!( + name: "Amp", + redirect_uris: [ "http://127.0.0.1:49152/callback" ], + grant_types: McpOauthClient::DEFAULT_GRANT_TYPES, + response_types: McpOauthClient::DEFAULT_RESPONSE_TYPES, + scopes: McpOauthClient::DEFAULT_SCOPES + ), + user: target, + principal: principals(:acme_channel), + resource: "http://localhost:3000/mcp", + scopes: [ "mcp:tools" ] + ) + + post disable_console_user_url(target.oid) + + assert_redirected_to console_users_path + assert target.reload.disabled? + assert refresh.reload.revoked_at.present? + end + test "an admin cannot disable their own account" do admin = users(:acme_admin) sign_in admin @@ -76,7 +110,7 @@ def sign_in(user) sign_in users(:member_user) target = users(:pending_user) post approve_console_user_url(target.oid) - assert_redirected_to root_path + assert_redirected_to console_threads_path assert target.reload.pending? end end diff --git a/services/console/test/controllers/console/workflows_controller_test.rb b/services/console/test/controllers/console/workflows_controller_test.rb new file mode 100644 index 000000000..071f1b4bb --- /dev/null +++ b/services/console/test/controllers/console/workflows_controller_test.rb @@ -0,0 +1,447 @@ +require "test_helper" + +class Console::WorkflowsControllerTest < ActionDispatch::IntegrationTest + FakeWorkflowRun = Struct.new( + :workflow_name, + :workflow_name_label, + :task_name, + :display_status, + :queue_name, + :queue_label, + :attempts, + :max_attempts, + :started_or_created_at, + :created_at, + :terminal_at, + :recency_at, + :run_id, + :task_id, + :harness_type, + :queue_run_count, + keyword_init: true + ) do + def workflow_name_label + self[:workflow_name_label].presence || workflow_name.presence || task_name.presence || "unknown workflow" + end + + def workflow_key + workflow_name.presence || task_name.presence + end + + def recency_at + self[:recency_at] || terminal_at || started_or_created_at + end + end + + # Stands in for CentaurApiClient: schedules/run details for show-page + # enrichment, plus a capture of force-started runs. + class FakeApiClient + attr_reader :created_runs + + def initialize(schedules: [], run_details: {}, create_result: nil, create_error: nil) + @schedules = schedules + @run_details = run_details + @create_result = create_result || { "ok" => true, "run_id" => "run-new", "created" => true } + @create_error = create_error + @created_runs = [] + end + + def list_workflow_schedules + { "ok" => true, "schedules" => @schedules } + end + + def get_workflow_run(run_id) + detail = @run_details[run_id] + raise CentaurApiClient::Error, "run not found" unless detail + + { "ok" => true, "run" => detail } + end + + def create_workflow_run(workflow_name:, input: nil) + raise CentaurApiClient::Error, @create_error if @create_error + + @created_runs << { workflow_name: workflow_name, input: input } + @create_result + end + end + + setup do + @original_client_factory = Console::WorkflowsController.client_factory + with_api_client(FakeApiClient.new) + @operator = users(:acme_admin) + post login_url, params: { email: @operator.email, password: "password123456" } + end + + teardown do + Console::WorkflowsController.client_factory = @original_client_factory + end + + test "an admin sees one row per workflow" do + run = fake_run(workflow_name: "slack_sync", display_status: "running") + + with_workflow_index(runs: [ run ]) do + get console_workflows_url + end + + assert_response :ok + assert_select "h1", count: 0 + assert_select ".console-thread-group-title-active", text: /Workflows/ + assert_select "a[href=?]", console_workflow_path("slack_sync"), text: /slack_sync/ + assert_select "span", text: "running" + assert_select "a[href=?]", console_workflows_path + assert response.body.index('href="/console/workflows"') < response.body.index('href="/console/threads"') + end + + test "a workflow with runs in several queues lists each queue on its own line" do + run = fake_run(workflow_name: "slack_backfill", queue_name: "centaur_workflows_etl_backfill", queue_label: "etl backfill") + queue_runs = [ + fake_run(workflow_name: "slack_backfill", queue_name: "centaur_workflows_etl_backfill", queue_label: "etl backfill", queue_run_count: 7), + fake_run(workflow_name: "slack_backfill", queue_name: "centaur_workflows_slack_live", queue_label: "slack live", display_status: "running", queue_run_count: 2) + ] + + with_workflow_index(runs: [ run ], queue_breakdown: { "slack_backfill" => queue_runs }) do + get console_workflows_url + end + + assert_response :ok + assert_select "tbody tr", count: 1 + assert_match "etl backfill", response.body + assert_match "slack live", response.body + assert_match "├", response.body + assert_match "└", response.body + assert_match "7 runs", response.body + end + + test "the workflow index does not show run ids" do + run = fake_run(workflow_name: "slack_sync") + + with_workflow_index(runs: [ run ]) do + get console_workflows_url + end + + assert_response :ok + assert_no_match run.run_id, response.body + assert_no_match run.task_id, response.body + end + + test "the workflow index is paginated" do + runs = 3.times.map { |i| fake_run(workflow_name: "wf_#{i}") } + + with_workflow_index(runs: runs, workflow_count: 120) do + get console_workflows_url, params: { page: 2 } + end + + assert_response :ok + assert_match "120 workflows", response.body + assert_match "page 2 of 3", response.body + assert_select "a", text: "Previous" + assert_select "a", text: "Next" + end + + test "a non-admin is redirected away from the workflow dashboard" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + get console_workflows_url + + assert_redirected_to console_threads_path + assert_nil flash[:alert] + end + + test "a non-admin does not see the workflows tab" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + get console_threads_url + + assert_response :ok + assert_select ".console-nav-link", text: "Control", count: 0 + assert_select ".console-nav-link", text: "Data Sync", count: 0 + assert_select ".console-thread-group-title", text: /Chats/ + assert_select ".console-thread-group-title", text: /Workflows/, count: 0 + end + + test "workflow show page lists core metadata and historical runs" do + run = fake_run(workflow_name: "slack_sync", display_status: "completed", harness_type: "codex") + + with_workflow_history("slack_sync", runs: [ run ]) do + get console_workflow_url("slack_sync") + end + + assert_response :ok + assert_select "h1.page-title", text: /slack_sync/ + assert_select "dt", text: "Engine" + assert_select "dd", text: "Codex" + assert_select "h2", "Historical Runs" + assert_select "tbody tr", count: 1 + assert_select "form[action=?]", run_console_workflow_path("slack_sync") + end + + test "workflow show page renders status filter tabs with counts" do + run = fake_run(workflow_name: "slack_sync", display_status: "completed") + + with_workflow_history( + "slack_sync", + runs: [ run ], + status_counts: { "completed" => 9, "failed" => 1 } + ) do + get console_workflow_url("slack_sync") + end + + assert_response :ok + assert_select "a.chip", text: /all\s*10/ + assert_select "a.chip", text: /completed\s*9/ + assert_select "a.chip", text: /failed\s*1/ + assert_select "dd", text: /10 runs/ + end + + test "workflow show page marks the active status tab and passes the filter through" do + run = fake_run(workflow_name: "slack_sync", display_status: "failed") + seen = {} + + with_workflow_history( + "slack_sync", + runs: [ run ], + status_counts: { "completed" => 9, "failed" => 1 }, + capture: seen + ) do + get console_workflow_url("slack_sync"), params: { status: "failed" } + end + + assert_response :ok + assert_equal "failed", seen[:status] + assert_select "a.chip-on", text: /failed\s*1/ + end + + test "workflow show page renders queue tabs when several queues exist" do + run = fake_run(workflow_name: "slack_sync") + + with_workflow_history( + "slack_sync", + runs: [ run ], + queue_names: %w[centaur_workflows_etl centaur_workflows_slack_live] + ) do + get console_workflow_url("slack_sync"), params: { queue: "centaur_workflows_slack_live" } + end + + assert_response :ok + assert_select "a.chip", text: "etl" + assert_select "a.chip-on", text: "slack live" + end + + test "workflow show page paginates historical runs" do + runs = 2.times.map { |i| fake_run(workflow_name: "slack_sync", run_id: "run-#{i}") } + + with_workflow_history("slack_sync", runs: runs, run_count: 130) do + get console_workflow_url("slack_sync"), params: { page: 2 } + end + + assert_response :ok + assert_match "130 runs", response.body + assert_match "page 2 of 3", response.body + end + + test "workflow show page shows the schedule and source link when registered" do + run = fake_run(workflow_name: "slack_sync", harness_type: "codex") + with_api_client(FakeApiClient.new(schedules: [ slack_sync_schedule ])) + + with_workflow_history("slack_sync", runs: [ run ]) do + get console_workflow_url("slack_sync") + end + + assert_response :ok + assert_select "dt", text: "Schedule" + assert_select "dd", text: /cron \*\/5 \* \* \* \* · America\/Los_Angeles/ + assert_select "a[href=?]", + "https://github.com/paradigmxyz/centaur/blob/main/workflows/slack/sync.py", + text: /workflows\/slack\/sync\.py/ + end + + test "workflow show page links overlay-repo workflow sources to the overlay repo" do + run = fake_run(workflow_name: "consensus_ci_triage") + schedule = slack_sync_schedule.merge( + "workflow_name" => "consensus_ci_triage", + "source_path" => "centaur-tempo/workflows/consensus_ci_triage.py" + ) + with_api_client(FakeApiClient.new(schedules: [ schedule ])) + + with_workflow_history("consensus_ci_triage", runs: [ run ]) do + get console_workflow_url("consensus_ci_triage") + end + + assert_response :ok + assert_select "a[href=?]", + "https://github.com/tempoxyz/centaur-tempo/blob/main/workflows/consensus_ci_triage.py" + end + + test "workflow show page surfaces the latest run's input and failure for debugging" do + run = fake_run(workflow_name: "slack_sync", display_status: "failed") + with_api_client( + FakeApiClient.new( + run_details: { + run.run_id => { + "run_id" => run.run_id, + "input" => { "mode" => "full" }, + "failure" => { "error" => "boom exploded" } + } + } + ) + ) + + with_workflow_history("slack_sync", runs: [ run ]) do + get console_workflow_url("slack_sync") + end + + assert_response :ok + assert_select "h2", text: "Debugging" + assert_select "dt", text: "Input" + assert_select "dt", text: "Failure" + assert_match "boom exploded", response.body + end + + test "workflow show page renders without api enrichment when the api is down" do + run = fake_run(workflow_name: "slack_sync") + with_api_client(FakeApiClient.new(run_details: {})) + + with_workflow_history("slack_sync", runs: [ run ]) do + get console_workflow_url("slack_sync") + end + + assert_response :ok + assert_select "h2", text: "Debugging", count: 0 + assert_select "dt", text: "Schedule", count: 0 + end + + test "force starting a workflow queues a run with the schedule input" do + client = FakeApiClient.new(schedules: [ slack_sync_schedule ]) + with_api_client(client) + + post run_console_workflow_url("slack_sync") + + assert_redirected_to console_workflow_path("slack_sync") + assert_match(/Run queued \(run-new\)/, flash[:notice]) + assert_equal [ { workflow_name: "slack_sync", input: { "mode" => "incremental" } } ], client.created_runs + end + + test "force starting a workflow surfaces api errors" do + with_api_client(FakeApiClient.new(create_error: "workflow runtime is not enabled")) + + post run_console_workflow_url("slack_sync") + + assert_redirected_to console_workflow_path("slack_sync") + assert_match(/workflow runtime is not enabled/, flash[:alert]) + end + + test "a non-admin cannot force start a workflow" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + client = FakeApiClient.new + with_api_client(client) + + post run_console_workflow_url("slack_sync") + + assert_redirected_to console_threads_path + assert_empty client.created_runs + end + + test "workflow show page returns not found for unknown workflow" do + with_workflow_history("missing") do + get console_workflow_url("missing") + end + + assert_response :not_found + assert_select "body", text: /No workflow runs found for missing/ + end + + test "workflows page handles unavailable workflow database" do + with_centaur_workflow_run_methods(available?: -> { false }) do + get console_workflows_url + end + + assert_response :ok + assert_select "body", text: /Workflow database is unavailable/ + assert_select "body", text: /No workflow runs available/ + end + + private + + def with_api_client(client) + Console::WorkflowsController.client_factory = -> { client } + end + + def slack_sync_schedule + { + "schedule_id" => "slack_sync", + "workflow_name" => "slack_sync", + "source_path" => "workflows/slack/sync.py", + "kind" => { "type" => "cron", "cron" => "*/5 * * * *" }, + "timezone" => "America/Los_Angeles", + "input" => { "mode" => "incremental" }, + "enabled" => true, + "no_delivery" => false + } + end + + def fake_run(attrs = {}) + now = Time.zone.parse("2026-07-06 12:00:00 UTC") + FakeWorkflowRun.new({ + workflow_name: "echo", + workflow_name_label: nil, + task_name: "centaur_workflow", + display_status: "completed", + queue_name: "centaur_workflows", + queue_label: "default", + attempts: 1, + max_attempts: 3, + started_or_created_at: now, + created_at: now, + terminal_at: now + 2.minutes, + recency_at: nil, + run_id: "00000000-0000-0000-0000-000000000001", + task_id: "00000000-0000-0000-0000-000000000002", + harness_type: nil, + queue_run_count: 1 + }.merge(attrs)) + end + + def with_workflow_index(runs:, queue_breakdown: {}, workflow_count: nil) + with_centaur_workflow_run_methods( + available?: -> { true }, + workflow_count: -> { workflow_count || runs.size }, + latest_per_workflow: ->(limit:, offset: 0) { runs }, + latest_per_queue: ->(keys) { queue_breakdown } + ) do + yield + end + end + + def with_workflow_history(workflow_name, runs: [], status_counts: nil, queue_names: [], run_count: nil, capture: nil) + status_counts ||= runs.group_by(&:display_status).transform_values(&:size) + with_centaur_workflow_run_methods( + available?: -> { true }, + for_workflow: ->(name, limit:, offset: 0, status: nil, queue: nil) { + capture&.merge!(status: status, queue: queue, offset: offset) + name == workflow_name && limit.positive? ? runs : [] + }, + status_counts: ->(name) { name == workflow_name ? status_counts : {} }, + queue_names: ->(name) { name == workflow_name ? queue_names : [] }, + run_count: ->(name, status: nil, queue: nil) { run_count || runs.size } + ) do + yield + end + end + + def with_centaur_workflow_run_methods(overrides) + originals = overrides.keys.to_h { |name| [ name, CentaurWorkflowRun.method(name) ] } + + overrides.each do |name, implementation| + CentaurWorkflowRun.define_singleton_method(name, &implementation) + end + + yield + ensure + originals&.each do |name, original| + CentaurWorkflowRun.define_singleton_method(name, original) + end + end +end diff --git a/services/console/test/controllers/console_controller_test.rb b/services/console/test/controllers/console_controller_test.rb index 97640f2db..dbc20434a 100644 --- a/services/console/test/controllers/console_controller_test.rb +++ b/services/console/test/controllers/console_controller_test.rb @@ -12,6 +12,28 @@ class ConsoleControllerTest < ActionDispatch::IntegrationTest assert_redirected_to login_path end + test "an active non-admin is redirected away from every Control page" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + [ root_url, console_principals_url, console_roles_url, console_secrets_url, + console_credentials_url, console_oauth_apps_url ].each do |url| + get url + assert_redirected_to console_threads_path + assert_nil flash[:alert] + end + end + + test "a non-admin cannot mutate through the Control form controllers" do + delete logout_url + post login_url, params: { email: users(:member_user).email, password: "password123456" } + + assert_no_difference -> { Role.count } do + post console_roles_url, params: { role: { foreign_id: "sneaky", namespace: "default" } } + end + assert_redirected_to console_threads_path + end + test "secrets table shows backend labels (not refs) and links to detail" do secret = static_secrets(:acme_prod_api_key) get console_secrets_url @@ -121,6 +143,40 @@ class ConsoleControllerTest < ActionDispatch::IntegrationTest assert_select "div", text: /#{Regexp.escape(principal.oid)}.*#{Regexp.escape(principal.namespace)}/ end + test "principals table links to add principal" do + get console_principals_url + assert_response :ok + assert_select "a[href=?]", console_new_principal_path, text: "Add Principal" + end + + test "principal detail page offers delete" do + principal = principals(:acme_channel) + get console_principal_url(principal.oid) + assert_response :ok + assert_select "form[action=?][method=?]", console_delete_principal_path(principal.oid), "post" do + assert_select "input[name=_method][value=delete]" + assert_select "button[type=submit]", "Delete" + end + end + + test "principal detail page renders DM permissions as API-managed rows" do + principal = principals(:acme_user_bob) + SlackChannelPermission.create!( + principal: principal, + channel_id: "D0123456789", + channel_name: "U0123456789", + upload_enabled: true, + download_enabled: false, + history_enabled: true + ) + + get console_principal_url(principal.oid) + assert_response :ok + + assert_select "td", text: /DM U0123456789/ + assert_select "td", text: "API-managed" + end + test "credentials table combines id, shows status, and links to detail" do credential = broker_credentials(:acme_managed_gmail) get console_credentials_url diff --git a/services/console/test/controllers/mcp/oauth_controller_test.rb b/services/console/test/controllers/mcp/oauth_controller_test.rb new file mode 100644 index 000000000..d991b8a26 --- /dev/null +++ b/services/console/test/controllers/mcp/oauth_controller_test.rb @@ -0,0 +1,339 @@ +require "test_helper" +require "base64" +require "digest" +require "uri" + +module Mcp + class OauthControllerTest < ActionDispatch::IntegrationTest + setup do + @operator = users(:acme_admin) + @saved_env = { + "CENTAUR_JWT_SIGNING_SECRET" => ENV["CENTAUR_JWT_SIGNING_SECRET"], + "CENTAUR_MCP_PUBLIC_URL" => ENV["CENTAUR_MCP_PUBLIC_URL"], + "CENTAUR_CONSOLE_PUBLIC_URL" => ENV["CENTAUR_CONSOLE_PUBLIC_URL"] + } + ENV["CENTAUR_JWT_SIGNING_SECRET"] = "test-secret" + ENV["CENTAUR_MCP_PUBLIC_URL"] = "http://localhost:3000/mcp" + ENV["CENTAUR_CONSOLE_PUBLIC_URL"] = "http://www.example.com" + end + + teardown do + @saved_env.each do |key, value| + if value.nil? + ENV.delete(key) + else + ENV[key] = value + end + end + end + + test "metadata advertises MCP OAuth endpoints" do + get "/.well-known/oauth-authorization-server" + + assert_response :ok + body = JSON.parse(response.body) + assert_equal "http://www.example.com", body.fetch("issuer") + assert_equal "http://www.example.com/mcp/oauth/authorize", body.fetch("authorization_endpoint") + assert_equal "http://www.example.com/mcp/oauth/token", body.fetch("token_endpoint") + assert_equal "http://www.example.com/mcp/oauth/register", body.fetch("registration_endpoint") + assert_includes body.fetch("code_challenge_methods_supported"), "S256" + end + + test "dynamic client registration creates a public PKCE client" do + assert_difference -> { McpOauthClient.count }, 1 do + post "/mcp/oauth/register", + params: { + client_name: "Amp", + redirect_uris: [ "http://127.0.0.1:49152/callback" ], + scope: "mcp:tools" + }, + as: :json + end + + assert_response :created + body = JSON.parse(response.body) + assert_match(/\Amoc_/, body.fetch("client_id")) + assert_equal "none", body.fetch("token_endpoint_auth_method") + assert_equal "mcp:tools", body.fetch("scope") + end + + test "dynamic client registration rejects non-loopback redirect URIs" do + assert_no_difference -> { McpOauthClient.count } do + post "/mcp/oauth/register", + params: { + client_name: "Attacker", + redirect_uris: [ "https://evil.example/callback" ], + scope: "mcp:tools" + }, + as: :json + end + + assert_response :bad_request + assert_equal "invalid_client_metadata", JSON.parse(response.body).fetch("error") + end + + test "authorize rejects non-loopback redirect URIs even when already stored" do + client = create_client + client.update_column(:redirect_uris, [ "https://evil.example/callback" ]) + post login_url, params: { email: @operator.email, password: "password123456" } + + assert_no_difference -> { McpOauthAuthorizationCode.count } do + get "/mcp/oauth/authorize", + params: authorize_params(client).merge(redirect_uri: "https://evil.example/callback") + end + + assert_response :bad_request + assert_includes response.body, "redirect_uri is not registered" + end + + test "authorize redirects signed-out users through login and preserves the request" do + client = create_client + get "/mcp/oauth/authorize", params: authorize_params(client) + + assert_redirected_to login_path + + post login_url, params: { email: @operator.email, password: "password123456" } + assert_match %r{\Ahttp://www.example.com/mcp/oauth/authorize\?}, response.location + end + + test "authorize accepts dynamic loopback redirect ports" do + client = create_client(redirect_uris: [ "http://localhost/callback" ]) + approval_params = authorize_params(client).merge( + redirect_uri: "http://localhost:49153/callback" + ) + post login_url, params: { email: @operator.email, password: "password123456" } + + assert_no_difference -> { McpOauthAuthorizationCode.count } do + get "/mcp/oauth/authorize", params: approval_params + end + + assert_response :ok + assert_select "form[action=?]", "/mcp/oauth/authorize" + + post "/mcp/oauth/authorize", params: approval_params.merge(decision: "approve") + assert_response :redirect + redirect = URI.parse(response.location) + assert_equal "localhost", redirect.host + assert_equal 49153, redirect.port + assert Rack::Utils.parse_nested_query(redirect.query).key?("code") + end + + test "authorization approval denial redirects without issuing a code" do + client = create_client + post login_url, params: { email: @operator.email, password: "password123456" } + + assert_no_difference -> { McpOauthAuthorizationCode.count } do + post "/mcp/oauth/authorize", params: authorize_params(client).merge(decision: "deny") + end + + assert_response :redirect + redirect = URI.parse(response.location) + query = Rack::Utils.parse_nested_query(redirect.query) + assert_equal "access_denied", query.fetch("error") + assert_equal "state-test", query.fetch("state") + end + + test "authorization code exchange returns a JWT access token for the console principal" do + client = create_client + code = authorize_code(client) + stored_code = McpOauthAuthorizationCode.find_usable(code) + assert_equal @operator, stored_code.user + assert_equal "http://localhost:3000/mcp", stored_code.resource + assert_match(/\Aprn_/, stored_code.principal.oid) + + exchange_authorization_code(client, code) + + assert_response :ok + body = JSON.parse(response.body) + assert_equal "Bearer", body.fetch("token_type") + assert_equal "mcp:tools", body.fetch("scope") + assert_match(/\Amcprt_/, body.fetch("refresh_token")) + + jwt_payload = decode_jwt_payload(body.fetch("access_token")) + assert_equal "http://www.example.com", jwt_payload.fetch("iss") + assert_equal "http://localhost:3000/mcp", jwt_payload.fetch("aud") + assert_equal stored_code.principal.oid, jwt_payload.fetch("principal_id") + assert_equal @operator.email, jwt_payload.fetch("email") + assert_equal "mcp:tools", jwt_payload.fetch("scope") + end + + test "authorization approval seeds new console principals with the user-mcp role" do + client = create_client + + code = authorize_code(client) + + principal = McpOauthAuthorizationCode.find_usable(code).principal + role = Role.find_by(namespace: principal.namespace, foreign_id: "user-mcp") + assert role, "expected the user-mcp role to be created" + assert_equal "User MCP", role.name + assert_equal "centaur", role.labels["managed-by"] + assert_includes principal.roles, role + end + + test "authorization approval labels a principal from one Slack SSO identity" do + @operator.user_identities.create!( + provider: "slack", subject: "U123", team_id: "T123", email: @operator.email, email_verified: true + ) + client = create_client + + code = authorize_code(client) + + principal = McpOauthAuthorizationCode.find_usable(code).principal + assert_equal "U123", principal.labels["slack_user_id"] + assert_equal "T123", principal.labels["slack_team_id"] + end + + test "authorization approval leaves Slack labels unset for ambiguous Slack SSO identities" do + @operator.user_identities.create!( + provider: "slack", subject: "U123", team_id: "T123", email: @operator.email, email_verified: true + ) + @operator.user_identities.create!( + provider: "slack", subject: "U456", team_id: "T456", email: @operator.email, email_verified: true + ) + client = create_client + + code = authorize_code(client) + + principal = McpOauthAuthorizationCode.find_usable(code).principal + assert_nil principal.labels["slack_user_id"] + assert_nil principal.labels["slack_team_id"] + end + + test "authorization approval reuses an existing user-mcp role" do + existing = Role.create!( + namespace: "default", + foreign_id: "user-mcp", + name: "Custom user role", + created_by: @operator + ) + client = create_client + + assert_no_difference -> { Role.count } do + code = authorize_code(client) + principal = McpOauthAuthorizationCode.find_usable(code).principal + assert_includes principal.roles, existing + end + end + + test "authorization approval does not restore a removed user-mcp role on existing principals" do + client = create_client + code = authorize_code(client) + principal = McpOauthAuthorizationCode.find_usable(code).principal + principal.principal_roles.destroy_all + + post "/mcp/oauth/authorize", params: authorize_params(client).merge(decision: "approve") + + assert_response :redirect + assert_empty principal.reload.roles + end + + test "authorization code exchange rejects users disabled after consent" do + client = create_client + code = authorize_code(client) + stored_code = McpOauthAuthorizationCode.find_usable(code) + @operator.update!(status: :disabled) + + assert_no_difference -> { McpOauthRefreshToken.count } do + exchange_authorization_code(client, code) + end + + assert_response :bad_request + assert_equal "invalid_grant", JSON.parse(response.body).fetch("error") + assert stored_code.reload.consumed_at.present? + end + + test "refresh token exchange rejects inactive users and revokes their tokens" do + client = create_client + code = authorize_code(client) + exchange_authorization_code(client, code) + refresh_token = JSON.parse(response.body).fetch("refresh_token") + issued = McpOauthRefreshToken.find_usable(refresh_token) + extra = McpOauthRefreshToken.create!( + mcp_oauth_client: client, + user: @operator, + principal: issued.principal, + resource: issued.resource, + scopes: issued.scopes + ) + @operator.update_column(:status, "disabled") + + post "/mcp/oauth/token", + params: { + grant_type: "refresh_token", + client_id: client.public_client_id, + refresh_token: refresh_token + } + + assert_response :bad_request + assert_equal "invalid_grant", JSON.parse(response.body).fetch("error") + assert issued.reload.revoked_at.present? + assert extra.reload.revoked_at.present? + assert_equal 0, @operator.mcp_oauth_refresh_tokens.usable.count + end + + private + + def create_client(redirect_uris: [ redirect_uri ]) + McpOauthClient.create!( + name: "Amp", + redirect_uris: redirect_uris, + grant_types: McpOauthClient::DEFAULT_GRANT_TYPES, + response_types: McpOauthClient::DEFAULT_RESPONSE_TYPES, + scopes: McpOauthClient::DEFAULT_SCOPES + ) + end + + def authorize_params(client) + { + response_type: "code", + client_id: client.public_client_id, + redirect_uri: redirect_uri, + scope: "mcp:tools", + state: "state-test", + resource: "http://localhost:3000/mcp", + code_challenge: code_challenge, + code_challenge_method: "S256" + } + end + + def authorize_code(client) + approval_params = authorize_params(client) + post login_url, params: { email: @operator.email, password: "password123456" } + + assert_no_difference -> { McpOauthAuthorizationCode.count } do + get "/mcp/oauth/authorize", params: approval_params + end + assert_response :ok + assert_select "form[action=?]", "/mcp/oauth/authorize" + + post "/mcp/oauth/authorize", params: approval_params.merge(decision: "approve") + assert_response :redirect + redirect = URI.parse(response.location) + Rack::Utils.parse_nested_query(redirect.query).fetch("code") + end + + def exchange_authorization_code(client, code) + post "/mcp/oauth/token", + params: { + grant_type: "authorization_code", + client_id: client.public_client_id, + code: code, + redirect_uri: redirect_uri, + code_verifier: code_verifier + } + end + + def redirect_uri = "http://127.0.0.1:49152/callback" + + def code_verifier = "test-code-verifier" + + def code_challenge + Base64.urlsafe_encode64(Digest::SHA256.digest(code_verifier), padding: false) + end + + def decode_jwt_payload(token) + _header, payload, _signature = token.split(".") + JSON.parse(Base64.urlsafe_decode64(payload)) + end + end +end diff --git a/services/console/test/controllers/oauth/flows_controller_test.rb b/services/console/test/controllers/oauth/flows_controller_test.rb index 3fd4e2f1a..ec30bd5ab 100644 --- a/services/console/test/controllers/oauth/flows_controller_test.rb +++ b/services/console/test/controllers/oauth/flows_controller_test.rb @@ -12,12 +12,16 @@ class FlowsControllerTest < ActionDispatch::IntegrationTest CLIENT_ID = "acme-google-client-id".freeze SLACK_CLIENT_ID = "acme-slack-client-id".freeze GITHUB_CLIENT_ID = "acme-github-client-id".freeze + ATTIO_CLIENT_ID = "acme-attio-client-id".freeze + LINEAR_CLIENT_ID = "acme-linear-client-id".freeze setup do @app = oauth_apps(:acme_google) # slug "google" @app.update!(client_secret: "app-secret") oauth_apps(:acme_slack).update!(client_secret: "slack-secret") oauth_apps(:acme_github).update!(client_secret: "github-secret") + oauth_apps(:acme_attio).update!(client_secret: "attio-secret") + oauth_apps(:acme_linear).update!(client_secret: "linear-secret") clear_enqueued_jobs end @@ -58,6 +62,7 @@ def slack_token_body(sub: "U0R7MFMJM", scope: "chat:write", id_token_value: nil, ok: true, access_token: "xoxe.xoxb-1-bot", refresh_token: "xoxe-1-bot-refresh", expires_in: 43_200, token_type: "bot", scope: "commands", id_token: id_token_value, + team: { id: "TACME", name: "Acme" }, authed_user: { id: sub, user: "grace", @@ -78,6 +83,23 @@ def github_token_body(scope: "repo,read:user", **overrides) }.merge(overrides).to_json end + def attio_token_body(**overrides) + { + access_token: "attio-user-token", + token_type: "Bearer" + }.merge(overrides).to_json + end + + def linear_token_body(scope: "read write", **overrides) + { + access_token: "lin-user-token", + refresh_token: "lin-refresh-token", + token_type: "Bearer", + expires_in: 86_399, + scope: scope + }.merge(overrides).to_json + end + def sign_in(user) post login_url, params: { email: user.email, password: "password123456" } end @@ -93,6 +115,23 @@ def start_flow(slug: "google", **params) # --- start ---------------------------------------------------------------- + test "start redirects to Attio with dashboard-configured scopes" do + get oauth_start_url(slug: "attio") + assert_response :redirect + uri = URI.parse(response.location) + assert_equal "app.attio.com", uri.host + assert_equal "/authorize", uri.path + q = URI.decode_www_form(uri.query).to_h + assert_equal ATTIO_CLIENT_ID, q["client_id"] + assert_equal "http://www.example.com/oauth/attio/callback", q["redirect_uri"] + assert_equal "code", q["response_type"] + assert_equal "S256", q["code_challenge_method"] + assert q["code_challenge"].present? + # The Attio developer dashboard owns the effective scopes; the generic + # flow still sends the sample app allowlist as a harmless scope param. + assert_equal "record_permission:read object_configuration:read", q["scope"] + end + test "start redirects to Google with the right params and sets the flow cookie" do get oauth_start_url(slug: "google") assert_response :redirect @@ -153,6 +192,24 @@ def start_flow(slug: "google", **params) assert_includes scopes, "read:user" end + test "start redirects to Linear with comma separated scopes" do + get oauth_start_url(slug: "linear") + assert_response :redirect + uri = URI.parse(response.location) + assert_equal "linear.app", uri.host + assert_equal "/oauth/authorize", uri.path + q = URI.decode_www_form(uri.query).to_h + assert_equal LINEAR_CLIENT_ID, q["client_id"] + assert_equal "http://www.example.com/oauth/linear/callback", q["redirect_uri"] + assert_equal "code", q["response_type"] + assert_equal "S256", q["code_challenge_method"] + assert_nil q["user_scope"] + assert_nil q["prompt"] + scopes = q["scope"].split(",") + assert_includes scopes, "read" + assert_includes scopes, "write" + end + test "start works without any session" do get oauth_start_url(slug: "google") assert_response :redirect @@ -195,16 +252,15 @@ def start_flow(slug: "google", **params) # --- callback ------------------------------------------------------------- - test "callback happy path mints a live credential and renders a success page" do + test "callback happy path mints a live credential and redirects to the Integrations page" do state = start_flow stub_exchange(status: 200, body: token_body) assert_difference -> { BrokerCredential.count } => 1 do get oauth_callback_url(slug: "google"), params: { state: state, code: "auth-code" } end - assert_response :ok - assert_match "Connected", response.body - assert_match "user@example.com", response.body + assert_redirected_to console_integrations_path + assert_equal "google connected as user@example.com.", flash[:notice] cred = BrokerCredential.find_by(oauth_app: @app, provider_subject: "google-sub-1") assert_equal "acme", cred.namespace @@ -218,7 +274,6 @@ def start_flow(slug: "google", **params) assert_equal "RT", cred.refresh_token assert cred.next_attempt_at.present? assert_nil cred.created_by - assert_includes response.body, cred.oid end test "callback happy path supports Slack user tokens" do @@ -228,8 +283,8 @@ def start_flow(slug: "google", **params) assert_difference -> { BrokerCredential.count } => 1 do get oauth_callback_url(slug: "slack"), params: { state: state, code: "auth-code" } end - assert_response :ok - assert_match "Connected", response.body + assert_redirected_to console_integrations_path + assert_match(/\Aslack connected/, flash[:notice]) app = oauth_apps(:acme_slack) cred = BrokerCredential.find_by(oauth_app: app, provider_subject: "U0R7MFMJM") @@ -241,10 +296,40 @@ def start_flow(slug: "google", **params) assert_equal %w[chat:write], cred.scopes assert_equal "xoxe.xoxp-1-user", cred.access_token assert_equal "xoxe-1-refresh", cred.refresh_token + assert_equal "TACME", cred.labels["slack_team_id"] assert_equal [ "slack.com" ], cred.static_secret.rules.map(&:host) assert_equal "Slack – grace token", cred.static_secret.name end + test "callback happy path supports Attio workspace tokens" do + state = start_flow(slug: "attio", scopes: "record_permission:read") + stub_exchange(status: 200, body: attio_token_body) + + assert_enqueued_with(job: Oauth::EnrichAttioCredentialIdentityJob) do + assert_difference -> { BrokerCredential.count } => 1 do + get oauth_callback_url(slug: "attio"), params: { state: state, code: "auth-code" } + end + end + assert_redirected_to console_integrations_path + assert_match(/\Aattio connected/, flash[:notice]) + + app = oauth_apps(:acme_attio) + cred = BrokerCredential.find_by(oauth_app: app) + assert_equal "acme", cred.namespace + assert_match(/\Aattio-attio-pending-[a-f0-9]{32}\z/, cred.foreign_id) + assert_match(/\Apending-[a-f0-9]{32}\z/, cred.provider_subject) + assert_equal "Attio – Pending Attio workspace", cred.name + assert_equal "https://app.attio.com/oauth/token", cred.token_endpoint + assert_nil cred.provider_email + assert_equal %w[record_permission:read], cred.scopes + assert_equal "attio-user-token", cred.access_token + assert_nil cred.refresh_token + assert_nil cred.next_attempt_at + assert_equal [ "api.attio.com" ], cred.static_secret.rules.map(&:host) + assert_equal "Attio – Pending Attio workspace token", cred.static_secret.name + refute_includes BrokerCredential.refreshable, cred + end + test "callback happy path supports GitHub OAuth app tokens" do state = start_flow(slug: "github", scopes: "repo read:user") stub_exchange(status: 200, body: github_token_body) @@ -254,8 +339,8 @@ def start_flow(slug: "google", **params) get oauth_callback_url(slug: "github"), params: { state: state, code: "auth-code" } end end - assert_response :ok - assert_match "Connected", response.body + assert_redirected_to console_integrations_path + assert_match(/\Agithub connected/, flash[:notice]) app = oauth_apps(:acme_github) cred = BrokerCredential.find_by(oauth_app: app) @@ -274,6 +359,34 @@ def start_flow(slug: "google", **params) refute_includes BrokerCredential.refreshable, cred end + test "callback happy path supports Linear OAuth app tokens" do + state = start_flow(slug: "linear", scopes: "read write") + stub_exchange(status: 200, body: linear_token_body) + + assert_enqueued_with(job: Oauth::EnrichLinearCredentialIdentityJob) do + assert_difference -> { BrokerCredential.count } => 1 do + get oauth_callback_url(slug: "linear"), params: { state: state, code: "auth-code" } + end + end + assert_redirected_to console_integrations_path + assert_match(/\Alinear connected/, flash[:notice]) + + app = oauth_apps(:acme_linear) + cred = BrokerCredential.find_by(oauth_app: app) + assert_equal "acme", cred.namespace + assert_match(/\Alinear-linear-pending-[a-f0-9]{32}\z/, cred.foreign_id) + assert_match(/\Apending-[a-f0-9]{32}\z/, cred.provider_subject) + assert_equal "Linear – Pending Linear account", cred.name + assert_equal "https://api.linear.app/oauth/token", cred.token_endpoint + assert_nil cred.provider_email + assert_equal %w[read write], cred.scopes + assert_equal "lin-user-token", cred.access_token + assert_equal "lin-refresh-token", cred.refresh_token + assert cred.next_attempt_at.present? + assert_equal [ "api.linear.app" ], cred.static_secret.rules.map(&:host) + assert_equal "Linear – Pending Linear account token", cred.static_secret.name + end + test "callback wraps the minted credential in a grantable static secret" do state = start_flow stub_exchange(status: 200, body: token_body) @@ -313,6 +426,43 @@ def start_flow(slug: "google", **params) assert_equal "operator-renamed", secret.reload.name end + test "callback records the signed-in user on the credential and keeps the original owner on re-consent" do + user = users(:member_user) + sign_in user + state = start_flow + stub_exchange(status: 200, body: token_body) + get oauth_callback_url(slug: "google"), params: { state: state, code: "auth-code" } + + cred = BrokerCredential.find_by(oauth_app: @app, provider_subject: "google-sub-1") + assert_equal user, cred.created_by + + # Someone else re-consenting for the same provider account does not steal + # the credential. + sign_in users(:acme_admin) + state = start_flow + stub_exchange(status: 200, body: token_body) + get oauth_callback_url(slug: "google"), params: { state: state, code: "auth-code" } + assert_equal user, cred.reload.created_by + end + + test "a Slack consent with no email in the token response still shows connected on Integrations" do + user = users(:member_user) + sign_in user + state = start_flow(slug: "slack", scopes: "chat:write") + stub_exchange(status: 200, body: slack_token_body) + get oauth_callback_url(slug: "slack"), params: { state: state, code: "auth-code" } + assert_redirected_to console_integrations_path + + # Slack's token response carries no email (enrichment fills it in later), + # so the connected state must come from the created_by link. + cred = BrokerCredential.find_by(oauth_app: oauth_apps(:acme_slack), provider_subject: "U0R7MFMJM") + assert_nil cred.provider_email + assert_equal user, cred.created_by + + get console_integrations_url + assert_select "a.btn-secondary[href=?]", "http://www.example.com/oauth/slack/start", text: "Reconnect" + end + test "callback works with a disabled console session" do user = users(:member_user) sign_in user @@ -324,8 +474,8 @@ def start_flow(slug: "google", **params) get oauth_callback_url(slug: "google"), params: { state: state, code: "auth-code" } end - assert_response :ok - assert_match "Connected", response.body + assert_redirected_to console_integrations_path + assert_match(/connected/, flash[:notice]) assert_equal user.id, session[:user_id] end diff --git a/services/console/test/controllers/session_oauth_controller_test.rb b/services/console/test/controllers/session_oauth_controller_test.rb index 5381d6ce6..7ba73379c 100644 --- a/services/console/test/controllers/session_oauth_controller_test.rb +++ b/services/console/test/controllers/session_oauth_controller_test.rb @@ -101,13 +101,13 @@ def run_callback(sub:, email:, provider: "google", **token_overrides) # --- callback: provisioning ------------------------------------------------ - test "callback provisions a pending user for a non-bootstrap email and signs them in" do + test "callback provisions an active user for a non-bootstrap email and lands on the console" do assert_difference -> { User.count }, 1 do run_callback(sub: "new-sub", email: "newcomer@example.com") end - assert_redirected_to pending_path + assert_redirected_to console_threads_path user = User.find_by(email: "newcomer@example.com") - assert user.pending? + assert user.active? assert_not user.admin? assert_equal "Test User", user.name assert_equal user.id, session[:user_id] @@ -144,12 +144,12 @@ def run_callback(sub:, email:, provider: "google", **token_overrides) assert_nil session[:user_id] end - test "callback creates a pending user for an unverified, unrecognized email" do + test "callback creates an active user for an unverified, unrecognized email" do assert_difference -> { User.count }, 1 do run_callback(sub: "unv-sub", email: "stranger@example.com", email_verified: false) end user = User.find_by(email: "stranger@example.com") - assert user.pending? + assert user.active? assert_not user.user_identities.first.email_verified end diff --git a/services/console/test/controllers/sessions_controller_test.rb b/services/console/test/controllers/sessions_controller_test.rb index d49e1faf7..f2edd6699 100644 --- a/services/console/test/controllers/sessions_controller_test.rb +++ b/services/console/test/controllers/sessions_controller_test.rb @@ -15,6 +15,13 @@ class SessionsControllerTest < ActionDispatch::IntegrationTest assert_equal @operator.id, session[:user_id] end + test "a non-admin lands on the threads view after login" do + member = users(:member_user) + post login_url, params: { email: member.email, password: "password123456" } + assert_redirected_to console_threads_path + assert_equal member.id, session[:user_id] + end + test "email match is case-insensitive" do post login_url, params: { email: @operator.email.upcase, password: "password123456" } assert_equal @operator.id, session[:user_id] diff --git a/services/console/test/fixtures/oauth_apps.yml b/services/console/test/fixtures/oauth_apps.yml index f7dd50cbf..079bdd591 100644 --- a/services/console/test/fixtures/oauth_apps.yml +++ b/services/console/test/fixtures/oauth_apps.yml @@ -1,6 +1,18 @@ # client_secret is encrypted and is set in test setup via the model, not here -- # encrypt_fixtures is off (same as broker_credentials.yml). client_id is not # encrypted, so it lives here. +acme_attio: + slug: attio + description: Acme Attio integration + provider: attio + client_id: acme-attio-client-id + allowed_scopes: + - record_permission:read + - object_configuration:read + credential_namespace: acme + enabled: true + created_by: acme_admin + acme_google: slug: google description: Acme Google integration @@ -49,3 +61,15 @@ acme_github: credential_namespace: acme enabled: true created_by: acme_admin + +acme_linear: + slug: linear + description: Acme Linear integration + provider: linear + client_id: acme-linear-client-id + allowed_scopes: + - read + - write + credential_namespace: acme + enabled: true + created_by: acme_admin diff --git a/services/console/test/fixtures/principals.yml b/services/console/test/fixtures/principals.yml index 91e23366f..7341b4fdc 100644 --- a/services/console/test/fixtures/principals.yml +++ b/services/console/test/fixtures/principals.yml @@ -4,6 +4,7 @@ acme_channel: labels: kind: slack_channel team: platform + centaur.sandbox_repo_cache: all created_by: acme_admin globex_user: @@ -11,6 +12,7 @@ globex_user: foreign_id: U987654321 labels: kind: user + centaur.sandbox_repo_cache: all created_by: globex_admin acme_user_alice: @@ -19,6 +21,7 @@ acme_user_alice: labels: kind: user team: platform + centaur.sandbox_repo_cache: all created_by: acme_admin acme_user_bob: @@ -27,6 +30,7 @@ acme_user_bob: labels: kind: user team: ops + centaur.sandbox_repo_cache: all created_by: acme_admin globex_user_overlap: @@ -35,4 +39,5 @@ globex_user_overlap: labels: kind: user team: platform + centaur.sandbox_repo_cache: all created_by: globex_admin diff --git a/services/console/test/fixtures/slack_channel_permissions.yml b/services/console/test/fixtures/slack_channel_permissions.yml new file mode 100644 index 000000000..06ce246cd --- /dev/null +++ b/services/console/test/fixtures/slack_channel_permissions.yml @@ -0,0 +1,2 @@ +# Empty by default. Tests create Slack channel permission rows explicitly so legacy +# slack_channel_id label fallback remains covered by existing principal fixtures. diff --git a/services/console/test/fixtures/system_settings.yml b/services/console/test/fixtures/system_settings.yml new file mode 100644 index 000000000..f86e37652 --- /dev/null +++ b/services/console/test/fixtures/system_settings.yml @@ -0,0 +1,5 @@ +default: + singleton: true + default_sandbox_repo_cache: all + default_sandbox_observability_enabled: true + default_sandbox_api_server_enabled: true diff --git a/services/console/test/helpers/application_helper_test.rb b/services/console/test/helpers/application_helper_test.rb index a04d35d3b..f7e00d46e 100644 --- a/services/console/test/helpers/application_helper_test.rb +++ b/services/console/test/helpers/application_helper_test.rb @@ -1,4 +1,5 @@ require "test_helper" +require "timeout" class ApplicationHelperTest < ActionView::TestCase test "truncate_middle leaves short values unchanged" do @@ -33,10 +34,128 @@ class ApplicationHelperTest < ActionView::TestCase assert_select_in html, "time[data-localtime-relative-value=true]" end + test "local_time can request compact relative formatting" do + html = local_time(Time.utc(2026, 6, 4, 18, 30, 0), relative: true, format: :compact) + + assert_select_in html, "time[data-localtime-relative-value=true]" + assert_select_in html, "time[data-localtime-format-value=compact]" + end + test "local_time renders a placeholder for nil" do assert_select_in local_time(nil), "span", text: "—" end + test "console_markdown renders common github-flavored markdown" do + html = console_markdown(<<~MARKDOWN) + Yes, **partially legit**. + + Issue 1 is real on current `main`. + + - one + - two + + https://github.com/paradigmxyz/centaur/issues/792 + MARKDOWN + + assert_select_in html, "p", text: /Yes, partially legit/ + assert_select_in html, "strong", text: "partially legit" + assert_select_in html, "code", text: "main" + assert_select_in html, "ul li", count: 2 + assert_select_in html, "a.console-markdown-link[href='https://github.com/paradigmxyz/centaur/issues/792']", + text: "https://github.com/paradigmxyz/centaur/issues/792" + end + + test "console_markdown renders gfm tables with alignment" do + html = console_markdown(<<~MARKDOWN) + Before the table. + + | Name | Count | Status | + | :--- | ---: | :---: | + | `api-rs` | 12 | **ok** | + | console | 3 | pending | + + After the table. + MARKDOWN + + assert_select_in html, "table thead tr th", count: 3 + assert_select_in html, "table tbody tr", count: 2 + assert_select_in html, "th.text-right", text: "Count" + assert_select_in html, "th.text-center", text: "Status" + assert_select_in html, "td.text-right", text: "12" + assert_select_in html, "tbody code", text: "api-rs" + assert_select_in html, "tbody strong", text: "ok" + assert_select_in html, "p", text: "Before the table." + assert_select_in html, "p", text: "After the table." + end + + test "console_markdown pads and truncates ragged table rows to the header width" do + html = console_markdown(<<~MARKDOWN) + | a | b | + | --- | --- | + | only | + | one | two | three | + MARKDOWN + + assert_select_in html, "tbody tr", count: 2 + assert_select_in html, "tbody tr:first-child td", count: 2 + assert_select_in html, "tbody tr:last-child td", count: 2 + refute_includes html, "three" + end + + test "console_markdown escapes html inside table cells" do + html = console_markdown("| h |\n| --- |\n| |") + + refute_includes html, " **safe**") + + refute_includes html, "