Skip to content

Add AI Observer Panel: per-handler test scaffolding - #261

Open
droarty wants to merge 1 commit into
mainfrom
issue-260-ai-observer-panel-test-scaffolding
Open

Add AI Observer Panel: per-handler test scaffolding#261
droarty wants to merge 1 commit into
mainfrom
issue-260-ai-observer-panel-test-scaffolding

Conversation

@droarty

@droarty droarty commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds a new defaults action type (server DatabasePersistor.ts via $mergeObjects aggregation update, client documentModelStore.ts, shared ActionItem type) — an object-level upsert-if-absent that adds missing keys without touching existing ones.
  • workflow-builder.json's "AI Requests" tab now upserts a state.tests object (one [] array per currently ai-routed handler) whenever opened, then renders the new aiObserverPanel component bound to @state.tests, replacing the previous read-only logTreePanel.
  • New aiObserverPanel component (apps/web/src/components/layout/AiObserverPanel.tsx, adapted from LogTreePanel): left side is a tree of handlers → their tests; selecting any node shows an "Add Test" button.
  • New get-handler-test-skeleton query: scans a handler's transform/condition/query fields (deliberately excluding its AI prompt text, to avoid false-positive matches) for state.*/message.* references and builds a matching skeleton test object.
  • "Add Test" appends the scaffolded skeleton via the existing append action with a dynamically-computed $state.tests.<handlerName> path, and the new test auto-selects once it appears.
  • New JsonEditor component shows the selected test's data, freely editable locally — no save/persist wiring yet, per the intended scope of this PR (follow-up work).

Test plan

  • Verified live via WebdriverIO against the real running app:
    • The defaults upsert adds [] for new ai-routed handlers without clobbering an existing handler's test array (checked directly against MongoDB)
    • Only ai-routed handlers get tests entries; non-ai handlers do not
    • Clicking a handler selects it and shows "Add Test"
    • "Add Test" scaffolds a skeleton correctly capturing state.*/message.* paths referenced by the handler's transform, while excluding decoy references embedded in its AI system prompt
    • The new test is auto-selected and its skeleton renders correctly in the JSON editor
    • The JSON editor is freely editable locally with no crash
  • npx nx build web and npx nx build api both pass
  • All temporary test scaffolding (spec files, test users/artifacts/channels/logs) were removed afterward, scoped by exact ID

🤖 Generated with Claude Code

Extends the "AI Requests" tab into a testing tool. Adds a `defaults`
action type (server + client) that upserts missing keys onto an
object without clobbering existing ones, used to seed a `tests: {}`
array per ai-routed handler on the artifact's own state whenever the
tab is opened.

New aiObserverPanel component (duplicated/adapted from LogTreePanel)
shows a tree of tests[handlerName] with a right-side detail pane.
"Add Test" scaffolds a new test object via a new get-handler-test-skeleton
query that scans a handler's transform/condition/query fields (not its
AI prompt text, to avoid false positives) for state.*/message.*
references and builds a matching skeleton, appended via the existing
`append` action with a dynamically-computed $state.tests.<handlerName>
path. The new test auto-selects and is editable in a new JsonEditor
component (local edits only for now -- persisting them is separate
follow-up work, per the stated scope of this PR).

Closes #260

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant