Persist clabby's own crashes & errors locally (observability)

[Zlyzart]

Problem

Clabby does not persist anything about its own failures. When the process panics or an internal error propagates, the output goes to stderr only and is lost as soon as the terminal closes or a headless/cron run ends. There is no on-disk record of clabby's crashes or errors.

Concretely today:

• crates/cli/src/main.rs initializes tracing_subscriber::fmt() writing to stderr, default filter warn — no file sink, no rotation.
• There is no panic hook; a panic prints the default message to stderr and unwinds, leaving no trace.
• main returns Result, so propagated errors print via anyhow to stderr and then vanish.

Note: an agent's own stdout/stderr is already persisted (runner streaming → db.insert_log). The gap is specifically clabby's own panics and internal errors.

Why it matters

This bit us directly: issue #89 set out to add a regression test for a real crash in run_managed, but the crash could not be reconstructed because nothing captured it — no panic message, location, backtrace, or surrounding log. Without persisted diagnostics, post-mortem debugging and accurate regression tests are impossible, and the cron/headless paths are effectively blind.

Goal

Provide durable, local persistence of clabby's own errors and crashes so a failure can be diagnosed (and turned into a regression test) after the fact — without violating Constitution §7 (core stays UI-free; a process-global hook is a driver concern, not core's).

Status

Intentionally not designed yet — this is a research/design issue. Open questions to resolve during refinement:

• Capture scope: panics only, or panics + propagated error!/warn! traces?
• Sink: a rotating log file (robust even mid-DB-failure) vs. the DB vs. both; where the file lives (next to the DB / under the config root / an OS state dir).
• Where installation belongs across drivers (CLI main, cron, future Tauri) so behavior can't diverge by entry point.
• Retention/rotation and any redaction of sensitive command lines/env.

Out of scope (for this issue)

• Implementation — design first.
• Agent session-output logging (already handled via db.insert_log).
• Remote/network telemetry or third-party crash reporting (core must stay network-free per §7/§11).

Related

• Blocks a faithful regression test in Unit-test session.rs streaming/lifecycle (log capture, exit-status mapping) #89 (whose crash we could not reproduce).

[github-actions]

The issue is well-motivated but intentionally left undesigned — four architectural decisions must be resolved before a bounded implementation can be written:

1. Capture scope — panics only, or panics plus propagated error!/warn! tracing events? Each option has meaningfully different implementation surface and noise characteristics.
2. Sink choice — rotating log file vs. the DB vs. both? Where does the file live (alongside the DB path, under the config root, or an OS state dir such as dirs::state_dir)? A file sink is more robust during a DB failure; the DB is already queryable — which wins, or are both needed?
3. Installation site across drivers — the CLI main, cron, and future Tauri all need the same hook; where does shared installation logic live so behavior can't diverge by entry point? Does it belong in a thin clabby-diagnostics crate, or does each driver install independently?
4. Retention, rotation, and redaction — maximum log-file size/age before rotation; whether command-line arguments or env values that appear in tracing fields need scrubbing before they hit disk.

Once a human answers these, a concrete implementation scope and complexity label can be assigned and status:ready applied.