Design the Ultra verification and observability matrix

[iam-brain]

Question

What evidence must an implementation produce to prove that Ultra behaves correctly across catalog, prompt, request, retry, child-task, and failure paths?

Design a decision-complete matrix covering:

• catalog parsing and provider variants for Sol and Terra, including ordering, descriptions, eligibility metadata, stale fallback, and custom aliases;
• logical ultra versus wire max in normal Responses requests and every auxiliary path the plugin owns;
• explicit max remaining non-Ultra;
• prompt composition for every supported agent/profile and idempotent instruction injection;
• missing/disabled delegation, spawn failure, cancellation, recursion prevention, and concurrency warnings;
• account rotation, catalog-scope changes, session affinity, compaction, and child inheritance;
• model/client-version cold starts and cache transitions;
• request snapshots, shareable debug output, toasts, and redaction expectations;
• focused unit/integration tests, fixture updates, in-vivo/manual smoke checks, and the final npm run verify gate;
• acceptance criteria that distinguish product correctness from unverified closed-desktop parity.

The resolution should specify exact scenarios, expected observable outcomes, likely test homes, and minimum release evidence without writing production tests.