diff --git a/README.md b/README.md index 9242169..9078e4f 100644 --- a/README.md +++ b/README.md @@ -28,12 +28,11 @@ drive it over HTTP (Topology 1). no LLM and embeds no agent framework — the brain and governance (breakwater/flowsafe) live in the consumers. See its README. -M4 status: both packages are publish-ready (`@understudy/protocol` 0.3.0, -`@understudy/connector` 0.1.0 — MIT, `files`-scoped tarballs, `publishConfig.access: -public`) and the CI release flow is in place (see Release below); publishing waits only on -the npm `understudy` org + `NPM_TOKEN` secret. The cross-repo consumer e2e (a metamind / -smart-compliance Mastra agent + flowsafe approvals driving understudy) is the remaining -M4 step. The agent loop and governance stay consumer-side, per Topology 1. +M4 status: `@understudy/protocol@0.5.0` and `@understudy/connector@0.3.0` are +published on npm. Metamind contains the cross-repository Mastra workflow, +flowsafe approval gate, breakwater browser connectors, and attended runbook. +The remaining M4 step is running that proof with a connected Chromium +extension. The agent loop and governance stay in the consumer, per Topology 1. ## Develop @@ -50,13 +49,12 @@ freshly-published malicious versions; first-party `@proofoftech/*` packages are ## Release (npm) -Changesets + GitHub Actions, single-branch (see `.changeset/README.md`): a PR touching a -published package adds a changeset (`pnpm changeset`); on push to `master`, -`.github/workflows/release.yml` opens/updates the "Version Packages" PR — or, when nothing -is pending, publishes any package version not yet on npm (tags + GitHub releases, with -provenance). Requires the `NPM_TOKEN` repo secret with publish rights on the `@understudy` -scope. The first publish (`@understudy/protocol` 0.3.0, `@understudy/connector` 0.1.0) -needs no changeset — those versions are already set and unpublished. +Changesets and GitHub Actions manage releases from `master`; see +`.changeset/README.md`. A pull request that changes a published package adds a +changeset with `pnpm changeset`. The release workflow opens or updates the +“Version Packages” pull request, then publishes the approved versions with npm +provenance. The repository secret `NPM_TOKEN` needs publish access to the +`@understudy` scope. The M0 harness needs no build — load `apps/cdp-spike` unpacked in a Chromium browser (`apps/cdp-spike/README.md`). diff --git a/apps/backend/CLAUDE.md b/apps/backend/CLAUDE.md index ef70f21..acbad4e 100644 --- a/apps/backend/CLAUDE.md +++ b/apps/backend/CLAUDE.md @@ -19,14 +19,14 @@ Cloudflare Worker (Hono) + one Agents-SDK Durable Object per session (`SessionAg | `src/session.ts` | `SessionAgent` — the per-session Durable Object: WS auth, event routing, `dispatch`/`fillSecret` RPCs (typed `DispatchOutcome`, write-replay cache) | Changing session lifecycle, dryRun behavior, fill_secret dispatch, idempotent replay | | `src/coordinator.ts` | `SessionCoordinator` — portable command↔event correlation interface + failure-prefix constants, no Cloudflare imports | Understanding the portable seam, swapping the CF impl | | `src/coordinator-cf.ts` | `CfSessionCoordinator` — CF impl: pending map (+ duplicate-in-flight guard) + persisted awaiting-marker + hibernation reconciliation | Debugging a stuck/timed-out command, hibernation edge cases | -| `src/auth.ts` | Caller bearer-token auth, sessionId mint/scope (HMAC), extension token verify | Changing auth, adding a token type, debugging 401/404 | +| `src/auth.ts` | Caller bearer-token auth, fresh/idempotent sessionId minting, HMAC tenant scope, extension-token verification | Changing auth, session creation, token types, or 401/404 behavior | | `src/secrets.ts` | `resolveSecret` — vault lookup only, no dispatch | Changing the vault backend, debugging secret resolution failures | | `src/vault.ts` | AES-256-GCM envelope codec + `EncryptedKvVault`/`createVault` — KV holds ciphertext only | Changing the envelope format/key handling (mirror `scripts/vault-put.mjs`) | | `src/base64url.ts` | base64url codec shared by auth.ts and vault.ts | Rarely — codec changes | | `src/types.ts` | Shared `Env`, `SessionState` (incl. `completedWrites`), `SessionStatus`, `VaultBinding`, `DispatchOutcome` | Adding a binding, changing DO state shape, changing the RPC outcome union | | `scripts/stub-consumer.mjs` | Throwaway Node runbook harness (not a workspace member) driving the API against a real extension | Running the attended M3 end-to-end verification | | `scripts/vault-put.mjs` | Seeds one vault secret as an envelope via `wrangler kv key put` (plaintext from stdin; `--local` for dev) | Seeding/rotating vault values (never raw `kv key put`) | -| `test/service.test.ts` | Hono route tests: auth, tenant scoping, dryRun, fill_secret routing, pre-accept WS gate, idempotent write replay | Verifying/extending the command API | +| `test/service.test.ts` | Hono route tests: auth, tenant scoping, idempotent session minting, dryRun, fill_secret routing, pre-accept WS gate, write replay | Verifying/extending the command API | | `test/session.test.ts` | `SessionAgent`/coordinator tests: in-DO WS auth (defense in depth), onClose stamping, resolve correlation, hibernation-resume | Verifying/extending DO behavior | | `test/auth.test.ts` | Auth module unit tests | Verifying/extending auth.ts | | `test/coordinator.test.ts` | Coordinator unit tests (timeout, duplicate guard, abandon, no-leak logging) | Verifying/extending coordinator-cf.ts | diff --git a/apps/backend/README.md b/apps/backend/README.md index b11d020..9257600 100644 --- a/apps/backend/README.md +++ b/apps/backend/README.md @@ -120,6 +120,11 @@ during that window. Four things close this gap: - **sessionIds are minted, not looked up**: `mintSessionId` HMAC-signs a payload containing the tenant; `scopeSession` verifies the signature and payload rather than querying a table. This makes tenant-ownership verification stateless. + `POST /v1/sessions` also accepts an optional `Idempotency-Key` UUID. The UUID is + hashed with the authenticated tenant before signing, so retries and concurrent + requests from one consumer converge on the same tenant-scoped session without + exposing the caller's key in the session id. Omitting the header preserves the + fresh-session behavior. - **`fill_secret` resolves service-side, DO-scoped**: the agent (consumer-side) only ever sees an opaque `secretRef`. `secrets.ts::resolveSecret` performs vault lookup only — it imports neither `session.ts` nor the coordinator, so it cannot diff --git a/apps/backend/src/auth.ts b/apps/backend/src/auth.ts index 750e2cf..848b557 100644 --- a/apps/backend/src/auth.ts +++ b/apps/backend/src/auth.ts @@ -3,7 +3,8 @@ * * Three independent verification paths: authenticate() maps a caller's * bearer token to an {actor, tenantId} (who is calling the command API); - * mintSessionId/scopeSession bind a sessionId to its owning tenant so a + * mintSessionId/scopeSession bind a fresh or idempotently replayed sessionId + * to its owning tenant so a * cross-tenant request is refused with 404, never 403 - a 403 would confirm * the session exists for someone who does not own it, an existence oracle * (DL-008); verifyExtensionToken authenticates the extension's own WebSocket @@ -69,13 +70,38 @@ export function isValidTenantId(tenantId: string): boolean { /** * Mints a sessionId with the owning tenant embedded and HMAC-signed, so * scopeSession can verify ownership statelessly - no lookup table maps - * sessionId -> tenant; the id carries its own proof (DL-008). + * sessionId -> tenant; the id carries its own proof (DL-008). An idempotency + * UUID is tenant-salted and hashed into the nonce so concurrent consumer + * retries converge without exposing the caller's key in the returned id. */ -export async function mintSessionId(tenantId: string, env: Env): Promise { +export const SESSION_IDEMPOTENCY_KEY_PATTERN = + /^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i; + +export async function mintSessionId( + tenantId: string, + env: Env, + idempotencyKey?: string, +): Promise { if (!isValidTenantId(tenantId)) { throw new Error("invalid tenantId: must be non-empty and contain no '/'"); } - const nonce = toHex(crypto.getRandomValues(new Uint8Array(16))); + if ( + idempotencyKey !== undefined && + !SESSION_IDEMPOTENCY_KEY_PATTERN.test(idempotencyKey) + ) { + throw new Error("invalid idempotency key: must be a UUID"); + } + const nonce = + idempotencyKey !== undefined + ? toHex( + new Uint8Array( + await crypto.subtle.digest( + "SHA-256", + new TextEncoder().encode(`${tenantId}\0${idempotencyKey.toLowerCase()}`), + ), + ), + ).slice(0, 32) + : toHex(crypto.getRandomValues(new Uint8Array(16))); const payloadBytes = new TextEncoder().encode(JSON.stringify({ t: tenantId, n: nonce })); const key = await importHmacKey(env.AUTH_HMAC_SECRET); diff --git a/apps/backend/src/index.ts b/apps/backend/src/index.ts index 87f4be3..d005d72 100644 --- a/apps/backend/src/index.ts +++ b/apps/backend/src/index.ts @@ -13,7 +13,13 @@ import { Hono } from "hono"; import { getAgentByName, routeAgentRequest } from "agents"; import { safeParseCommand } from "@understudy/protocol"; -import { authenticate, mintSessionId, scopeSession, verifyExtensionToken } from "./auth"; +import { + authenticate, + mintSessionId, + scopeSession, + SESSION_IDEMPOTENCY_KEY_PATTERN, + verifyExtensionToken, +} from "./auth"; import type { DispatchOutcome, Env } from "./types"; import type { SessionAgent } from "./session"; @@ -31,7 +37,14 @@ app.post("/v1/sessions", async (c) => { const actor = await authenticate(c.req.raw, c.env); if (!actor) return c.json({ error: "unauthorized" }, 401); - const sessionId = await mintSessionId(actor.tenantId, c.env); + const idempotencyKey = c.req.header("idempotency-key")?.trim(); + if ( + idempotencyKey !== undefined && + !SESSION_IDEMPOTENCY_KEY_PATTERN.test(idempotencyKey) + ) { + return c.json({ error: "idempotency-key must be a UUID" }, 400); + } + const sessionId = await mintSessionId(actor.tenantId, c.env, idempotencyKey); return c.json({ sessionId }); }); diff --git a/apps/backend/test/auth.test.ts b/apps/backend/test/auth.test.ts index 6721b8a..5e4923a 100644 --- a/apps/backend/test/auth.test.ts +++ b/apps/backend/test/auth.test.ts @@ -110,6 +110,27 @@ describe("mintSessionId / scopeSession", () => { expect(await scopeSession(second, "tenantA", env)).toBe("ok"); }); + it("mints one stable session per tenant and idempotency key", async () => { + const env = makeEnv(); + const key = "d9428888-122b-4c26-a044-7096d6e845c5"; + + const first = await mintSessionId("tenantA", env, key); + const replay = await mintSessionId("tenantA", env, key); + const otherTenant = await mintSessionId("tenantB", env, key); + + expect(replay).toBe(first); + expect(otherTenant).not.toBe(first); + }); + + it.each(["", " ", "not-a-uuid"])( + "rejects malformed session idempotency key %j", + async (idempotencyKey) => { + await expect(mintSessionId("tenantA", makeEnv(), idempotencyKey)).rejects.toThrow( + /invalid idempotency key/, + ); + }, + ); + it.each(["acme/eu", "", "/", "a/b"])( "refuses to mint a sessionId for an unsafe tenantId %j (empty or slash-bearing would straddle the vault namespace)", async (badTenant) => { diff --git a/apps/backend/test/service.test.ts b/apps/backend/test/service.test.ts index d82939f..646e3ac 100644 --- a/apps/backend/test/service.test.ts +++ b/apps/backend/test/service.test.ts @@ -63,6 +63,15 @@ async function openSession(callerToken: string): Promise { return body.sessionId; } +async function openIdempotentSession(callerToken: string, idempotencyKey: string): Promise { + return exports.default.fetch( + authedRequest("/v1/sessions", callerToken, { + method: "POST", + headers: { "Idempotency-Key": idempotencyKey }, + }), + ); +} + /** Opens the fake-extension WS at the real onConnect-authed route (DL-006 critical fact). */ async function connectFakeExtension(sessionId: string, token = EXTENSION_TOKEN_A): Promise { const res = await exports.default.fetch( @@ -154,6 +163,29 @@ describe("GET /health", () => { }); }); +describe("POST /v1/sessions idempotency", () => { + it("replays the same tenant-scoped session for the same key", async () => { + const key = "d9428888-122b-4c26-a044-7096d6e845c5"; + + const first = await openIdempotentSession(CALLER_TOKEN_A, key); + const replay = await openIdempotentSession(CALLER_TOKEN_A, key); + + expect(first.status).toBe(200); + expect(replay.status).toBe(200); + expect(await replay.json()).toEqual(await first.json()); + }); + + it.each(["", " ", "not-a-uuid"])( + "rejects malformed idempotency key %j", + async (idempotencyKey) => { + const response = await openIdempotentSession(CALLER_TOKEN_A, idempotencyKey); + + expect(response.status).toBe(400); + expect(await response.json()).toEqual({ error: "idempotency-key must be a UUID" }); + }, + ); +}); + describe("auth and tenant scoping", () => { it("rejects a commands POST with no Authorization header as 401", async () => { // #given an open session @@ -294,7 +326,9 @@ describe("fill_secret", () => { // messages alike) so the no-leak check below can assert the plaintext // appears on the wire exactly once - the one hop where it must travel. const rawFrames: string[] = []; - socket.addEventListener("message", (event: MessageEvent) => rawFrames.push(event.data as string)); + socket.addEventListener("message", (event: MessageEvent) => { + rawFrames.push(event.data as string); + }); const logSpies = [ vi.spyOn(console, "log").mockImplementation(() => {}), @@ -1249,7 +1283,9 @@ describe("two-tenant vault isolation (cross-tenant secretRef scoping, server-sid const socket = await connectFakeExtension(sessionId, EXTENSION_TOKEN_B); const received = collectCommands(socket); const rawFrames: string[] = []; - socket.addEventListener("message", (event: MessageEvent) => rawFrames.push(event.data as string)); + socket.addEventListener("message", (event: MessageEvent) => { + rawFrames.push(event.data as string); + }); const vaultGetSpy = vi.spyOn(env.VAULT, "get"); try { diff --git a/docs/technical-plan.md b/docs/technical-plan.md index 4ef834e..9b9736f 100644 --- a/docs/technical-plan.md +++ b/docs/technical-plan.md @@ -447,14 +447,15 @@ auth, local `fill_secret` shim) and carries stale pre-Topology-1 prose; prefer t reference breakwater connector (`@understudy/connector`, mirroring the smart-compliance example). A real consumer (metamind / smart-compliance) drives understudy end-to-end with a Mastra agent + flowsafe approvals. *Cross-repo; understudy's deliverable is the published contract + reference connector, not - the agent.* **Status (2026-07-16): `packages/connector` BUILT** — observe (snapshot/get_tabs/wait) / + the agent.* **Status (2026-07-17): PACKAGES PUBLISHED** — observe (snapshot/get_tabs/wait) / act (click/type/navigate/key/scroll/switch_tab, grant-gated) / fill_credential (vaulted), egress-pinned - `runtime.fetch`, caller bearer auth, 15 tests against the real breakwater wrapper (fail-closed grant, - idempotent replay, per-hop egress denial, dry-run). Both packages are publish-ready (MIT, - `files`-scoped tarballs, `publishConfig.access: public`) and the changesets + GitHub Actions release - flow is wired (`.github/workflows/release.yml`, single-branch master — first push publishes 0.3.0 / - 0.1.0 with no changeset needed); publishing waits on the npm `understudy` org + `NPM_TOKEN` secret. - The consumer-side Mastra+flowsafe e2e remains open. + `runtime.fetch`, caller bearer auth, and tests against the real breakwater wrapper (fail-closed grant, + idempotent replay, per-hop egress denial, dry-run). `@understudy/protocol@0.5.0` and + `@understudy/connector@0.3.0` are published on npm with MIT-licensed, `files`-scoped tarballs and + npm provenance. The changesets + GitHub Actions release flow is wired in + `.github/workflows/release.yml`. Metamind contains the consumer-side Mastra workflow, flowsafe + approval, breakwater connector wiring, automated coverage, and attended runbook. The remaining + proof is an attended run with its Chromium extension connected. - **M5 — Substrate hardening. LARGELY LANDED (2026-07-17, the deferred-items sweep):** pre-accept WS/HTTP auth at the Worker edge (`onBeforeConnect`/`onBeforeRequest` — unauthorized upgrades are 401/404 before the DO accepts, in-DO gate kept as defense in depth); credential