Skip to content

feat: interactive server-side investigations + recall panel - #137

Merged
coccyx merged 16 commits into
masterfrom
interactive-server-investigations
Aug 15, 2026
Merged

feat: interactive server-side investigations + recall panel#137
coccyx merged 16 commits into
masterfrom
interactive-server-investigations

Conversation

@coccyx

@coccyx coccyx commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

When serverInvestigations is on, all investigations now run on the cell (server-side, persistent, and interactive), and the Investigator window gains a left-hand recall panel to find past ones. Off by default — flag off keeps the classic in-browser client Investigator unchanged.

Design: docs/research/server-investigations/interactive-and-recall.md. Built as three commits:

  • PR A — cell (feat(cell): interactive investigations + recall search endpoints): POST /investigations (create interactive from a free-form prompt), POST /investigations/:id/messages (follow-up turn, resume the loop), a non-terminal idle status, and GET /investigations?q=&limit=&before= (search + keyset pagination, returns title + mode). Requires a cell redeploy (see below).
  • PR B — app (feat(app): … transport, session hook, routing): transport (createInvestigation, sendInvestigationMessage, listInvestigations), a useInvestigationSession hook (streams the transcript via the shared applyLoopEvent, seeds the opening question, exposes sendMessage), and routing so the Investigate button / a new-investigation composer create server investigations. The interactive view reuses the framework's exported InvestigatorTranscript — no framework change.
  • PR C — app (feat(app): recall/search panel): the sidebar — search, "Load older", 15s status refresh, status chips, active-row highlight, "+ New"; collapses under 720px for mobile.

Key design points

  • idle status: interactive investigations answer a turn then park at idle (resumable), instead of concluding. A follow-up flips them back to running.
  • Cell is the record store (your call): the recall panel reads GET /investigations; drill-in streams the transcript.
  • No framework SHA bump.

Deploy / validate

  1. Merge this PR.
  2. Redeploy the cell (it has the new endpoints) — Step 3 of cell/infra/DEPLOY.md: celld deploy cell --bucket s3://cribl-apm-cell-test then systemctl restart celld. Schema changes are additive (guarded ALTER TABLE), safe on existing data.
  3. In the app Settings, turn Server-side investigations on.
  4. Validate on staging: click Investigate on an alert → it runs server-side; open the Investigator → the recall panel lists it, search works, click reopens it; ask a follow-up → the conversation resumes.

Known limitations (v1)

  • Reopening a past multi-turn interactive conversation shows the opening question + all assistant/tool events, but not intermediate user follow-up prompts (the cell keeps those in agent history, not as replayable events).
  • A resumed interactive investigation re-enters running without re-acquiring a global concurrency slot (human-paced, low volume; tracked for hardening).

Tests

  • Cell: titleFromPrompt unit tests; cell/scripts/smoke.mjs §10 exercises create → idle → follow-up → resume → idle + recall search against a live cell.
  • App: npx tsc --noEmit, npm run lint, 407 unit tests, npm run build all green. Deployed to staging as apm-0.13.30.tgz (flag off — no behavior change until the cell is redeployed and the flag is flipped).

🤖 Generated with Claude Code

coccyx and others added 16 commits August 13, 2026 16:06
@coccyx @claude
docs: design for interactive server investigations + recall panel
cbb868e 
Captures the two user decisions (cell-backed record store; full
interactive server chat) and the resulting architecture: a
non-terminal `idle` status for interactive investigations, cell
resume-from-history on each user message, an app-side interactive
view composed from the already-exported InvestigatorTranscript +
applyLoopEvent (no framework change), and a left-hand recall/search
panel backed by an extended GET /investigations. PR A (cell) / PR B
(transport + interactive view) / PR C (panel) sequence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(cell): interactive investigations + recall search endpoints (PR A)
16a59ba 
Adds the cell surface for UI-started, resumable investigations and the
recall panel, per docs/research/server-investigations/interactive-and-recall.md.

Protocol:
- InvestigationStatus gains a non-terminal `idle` (loop answered the
  current user turn, awaiting the next message).
- InvestigationSummaryRow gains `title` + `mode`; add CreateInvestigationBody
  and titleFromPrompt().

Coordinator DO:
- `investigations` gains `mode` + `title` (idempotent ALTER).
- POST /internal/create — queue a UI-initiated interactive investigation
  from {prompt, context?, title?} with a synthetic event_id.
- /internal/complete accepts `idle` (park, free the slot) and `resumed`.
- /internal/list gains ?q= (title/incident search), ?limit=, ?before=
  (keyset pagination), and returns title + mode. pump() dispatches by
  mode (/start vs /create).

Investigation DO:
- `investigation` gains `mode`, `title`, `turn_budget` (per-message cap).
- POST …/create (interactive seed from the user's prompt via the same
  buildSeedPrompt preamble) and POST …/messages (append a user turn,
  reset the budget, resume the loop, notify coordinator `resumed`).
- runTurn branches on mode: interactive `done`/error/cap PARK at idle
  (resumable, no lifecycle commit) instead of concluding/failing.
- /status reports mode + title.

Router: POST /investigations, POST /investigations/:id/messages, and
q/limit/before passthrough on GET /investigations.

Tests: titleFromPrompt unit tests; smoke.mjs section 10 exercises
create → idle → follow-up → resume → idle and the recall search against
a live cell.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(app): interactive server investigations — transport, session hoo…
ff8b0d0 
…k, routing (PR B)

Routes the Investigate flow through the cell when serverInvestigations
is on, so investigations run server-side, persist, and stay
interactive.

Transport (investigationTransport.ts):
- InvestigationStatus gains `idle`; add InvestigationMode + the
  InvestigationSummary row type; status response carries mode/title/seed.
- createInvestigation({prompt,context?,title?}), sendInvestigationMessage,
  listInvestigations({q,limit,before}); postJson helper.
- subscribeInvestigation gains stopOnIdle so an interactive session
  pauses polling at idle and a follow-up re-subscribes.

useInvestigationSession — generalizes the read-only replay hook: streams
the transcript via applyLoopEvent, seeds the opening user bubble (from
the passed prompt or the cell's stored seed), exposes sendMessage +
canSend/mode. Replaces useInvestigationReplay (removed).

InvestigatePage:
- Flag on + Investigate-button seed → create a server investigation and
  redirect to its interactive view. Flag on + no seed → a new-
  investigation composer. Flag off → the classic client InvestigatorChat.
- ServerInvestigationView renders the shared InvestigatorTranscript plus
  a follow-up composer when the investigation is interactive and open;
  autonomous/concluded ones render read-only. No framework change.

Known v1 limitation: reopening a past multi-turn interactive
conversation shows the opening question + all assistant/tool events but
not intermediate user prompts (the cell keeps those in agent history,
not as replayable events).

Version bump 0.13.29 → 0.13.30.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(app): recall/search panel for the Investigator window (PR C)
c9976d9 
Adds a left-hand sidebar (server mode only) that lists recent
investigations from the cell, searches them by title/incident, and
paginates older ones with a keyset cursor. Clicking a row opens that
investigation — interactive if still open, read-only if concluded.

- InvestigationsSidebar: debounced search, "Load older" via the
  `before` cursor, a 15s silent refresh to keep statuses current, a
  status chip per row (Running/Open/Done/Failed), active-row highlight,
  and a "+ New" button. Collapses under 720px so mobile keeps the full
  transcript width.
- InvestigatePage wraps the active view (interactive / creating / new
  composer) in a two-column layout with the sidebar when
  serverInvestigations is on; flag-off keeps the bare client chat.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(settings): allow manual cell-token entry, not just generate
5af0fd2 
The cell-token field was read-only (generate-only). You often already
have a token on the cell (e.g. `openssl rand -hex 32` set as UI_BEARER)
and just need the app side to match it. Make the field editable with a
"Save token" action; keep "Generate" for a fresh random one. Both write
the raw value to KV `cellToken` so the proxy injects it verbatim.

Version bump 0.13.30 → 0.13.31.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
fix(provisioning): stop alert_notify churn + deterministic banner + c…
2484880 
…ollapsible panel

Three fixes for the "searches not provisioned banner flaps randomly" report,
plus a UX ask:

- provision.ts: when SERVER_INVESTIGATIONS is unset, no longer default the
  flag OFF (which DELETED criblapm__alert_notify out from under a UI that has
  server-investigations ON — every routine `npm run deploy` fought the user's
  setting). Now infer the flag from the server's current state (alert_notify
  presence) so an unset deploy neither creates nor deletes it. The webhook
  target is only ensured on an explicit SERVER_INVESTIGATIONS=true, so a
  routine deploy no longer hard-exits on a missing CELL_URL.

- ProvisioningBanners: add serverInvestigations to the re-check deps. The flag
  gates alert_notify in the plan and loads async from KV; without it as a dep
  the banner checked once with the default value and never re-checked, so it
  flickered in/out on reload-vs-navigation timing. Now deterministic.

- InvestigationsSidebar: collapsible via a persisted (localStorage) toggle —
  collapses to a slim rail so the transcript gets full width.

Note (no code change): the Alerts page shows no investigation links because the
badges are data-driven from investigation lifecycle events in the dataset,
which the cell only writes for autonomous (alert-triggered) runs. That pipeline
isn't live yet — alert_notify isn't provisioned — so there are no events and no
links. Provisioning alert_notify (with the cell webhook) lights it up.

Version bump 0.13.31 → 0.13.32.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
docs(env): activate CELL_URL / CELL_WEBHOOK_BEARER in .env.example
6b35b24 
They were documented but fully commented. Make CELL_URL and
CELL_WEBHOOK_BEARER active fillable keys (empty by default, safe — my
provisioner change only ensures the webhook target on an explicit
SERVER_INVESTIGATIONS=true). Keep SERVER_INVESTIGATIONS commented since
it's a per-run toggle, and document that unset now means "leave the
trigger as-is."

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
fix(provision): correct webhook-target existence check
6b67a3d 
The by-id notification-targets GET returns 200 {items:[],count:0} when
absent (not 404), so the old 'GET didn't throw ⇒ exists' logic PATCHed a
nonexistent target and 404'd. Check the payload count/items instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
fix(provision): bind alert-notify via the notifications resource (sha…
6971699 
…red module)

The autonomous trigger never actually fired: the framework writes a
saved search's notifications inline in the search body, but Cribl
IGNORES that on write (silently keeps `{}`). Notifications are a separate
resource under `/m/<group>/notifications`, JOINed into the search on read.
So alert_notify ran every 5 min and posted nowhere → no investigations,
no Alerts links.

- New src/api/cellProvisioning.ts (browser-safe, shared): ensureCellWebhookTarget,
  ensureAlertNotification (POST/PUT to the notifications resource — the
  correct write path), removeAlertNotification. This is the module the
  Settings UI will call too, so CLI and UI provisioning become identical
  (the divergence — UI created the search but not the target/binding —
  was the bug you flagged).
- provision.ts: delegate to the shared module via wireCellTrigger, run
  AFTER the reconcile so alert_notify exists before its notification binds.
  Also fixes the earlier existence-check bug (by-id GET returns
  200 {items:[],count:0}, not 404).

Verified end-to-end: alert_notify now shows its notification bound to the
cell webhook; provision is idempotent (search reconciles to noop).

Follow-up (not in this commit): wire the Settings "Provision" flow to call
the shared module + add a cellWebhookBearer setting so the browser can
create the target — that completes UI == CLI.

Version bump 0.13.32 → 0.13.33.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(settings): UI provisioning ensures cell trigger too (UI == CLI)
b2d4dd5 
Makes the in-app "Provision" button do everything scripts/provision.ts
does, closing the divergence where the UI created the searches but not
the webhook target / notification binding.

- Bump framework SHA → 9881413 (ProvisioningPanel afterReconcile hook).
- SettingsPage passes afterReconcile that runs the SHARED cellProvisioning
  helpers: when server investigations is on it ensures the webhook target
  + binds alert_notify's notification; when off it removes the binding.
  Same code path as the CLI.
- New "Cell webhook bearer" Settings field (KV `cellWebhookBearer`) so the
  browser has the cell's WEBHOOK_BEARER to create the target — the secret
  the CLI reads from CELL_WEBHOOK_BEARER.

Version bump 0.13.33 → 0.13.34.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
chore: re-pin framework SHA to merged master (framework #26)
c11a6ac 
PR #26 (ProvisioningPanel afterReconcile hook) merged to framework
master as 1c37201. Re-pin from the branch commit (9881413) to the
master commit so app master no longer depends on an unmerged/deletable
branch commit. Framework code is byte-identical — no pack change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
feat(investigations): replay the user's messages in a saved session
a5b4dcb 
Reopening a past interactive investigation showed only the assistant's
side — the user's follow-up messages weren't recorded anywhere the
replay could read (they lived in the agent's raw history, not the
transcript). Record each follow-up as a `userMessage` transcript event.

- cell: protocol WireLoopEvent gains `userMessage`; the /messages handler
  appends one (ordered before the assistant's reply) so it replays.
- app: subscribeInvestigation surfaces userMessage frames via onUserMessage;
  useInvestigationSession renders them as user bubbles, deduped against the
  local optimistic append during a live send. Opening question still comes
  from the stored seed.
- smoke: assert the follow-up is recorded as a userMessage frame.

Requires a cell redeploy for the transcript-event side; app side deploys now.
Version bump 0.13.34 → 0.13.35.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
fix(provision): full notification shape in the alert_notify plan
f9f3d53 
Match the known-good Cribl notification structure (items[].id +
disabled, conf.savedQueryId, targetConfigs[].id) so the plan's
schedule.notifications deep-subset-matches what the server returns for
the notification (created via the notifications resource in
cellProvisioning). Without this the reconcile's isSameAsPlan sees a diff
and churns the search every run. (This edit was made during the
notifications investigation but not committed with the rest.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
docs: capture 2026-08-14 server-investigations learnings
9d09671 
Skill doc (docs/cribl-app-skill/skill.md) — durable Cribl-platform
gotchas that cost real debugging time this session:
- proxies.yml local/-override trap (stale grant survives redeploy/reinstall)
- notification-target by-id GET returns 200 {items:[],count:0}, not 404
- saved-search notifications are a SEPARATE resource (inline writes dropped)
- webhook-target token vs receiver bearer (401 vs 403), rotates on redeploy
- deploy no-ops without a version bump
- keep UI and CLI provisioners identical (shared module + afterReconcile)
- infer a flag-gated search from server state when the CLI flag is unreadable

Session doc (docs/sessions/2026-08-14-server-investigations-e2e.md) —
the interactive UI + recall panel + saved-session messages, and the
seven-bug cascade from "trigger silently never fires" to 202 accepted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
ci: read .framework-sha in workflows instead of hardcoding it
6380664 
The release-build action asserts its `framework-sha` input equals the
.framework-sha file, but ci.yml/release.yml hardcoded the SHA in three
places. They drifted (ci=ec58704, release=4ee0c5d, file=1c37201), so the
first framework bump this session failed CI with "action input does not
match .framework-sha". Read the file at runtime and pass it, so a bump
only touches .framework-sha. Bump the action refs to 1c37201.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx @claude
test(resilience-routes): make Investigate route mode-agnostic
973c3c0 
The interactive-investigations feature changes the Investigator page
title based on the serverInvestigations flag ("Copilot Investigation"
client vs "Cribl APM Copilot" server-side composer). The resilience test
hardcoded the client title, so it failed on the validation workspace
where the flag is on. Match either — this test only asserts the route
renders content (not the error boundary), not which mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0158jez4WfWj6ttmaxh3HEbT
@coccyx
coccyx merged commit 27f9053 into master Aug 15, 2026
4 checks passed
@coccyx
coccyx deleted the interactive-server-investigations branch August 15, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@coccyx