Audit fail-closed granularity: block on auth/budget failure, allow on observability failure

[Destynova2]

Context

The most dangerous LiteLLM criticism (their own P0 "Stability Sprint", PR #34429) is fail-open budgets: on a Redis failure, requests could proceed and blow the budget even with fail_closed_budget_enforcement requested. For a regulated client, a budget is an authorization to consume, not just reporting.

grob is already fail-closed for budget: src/server/budget.rs:164/176 returns Err(BudgetExceeded) before the provider call, spend committed synchronously, and tests/…mod.rs asserts "dispatch must block on the policy budget cap". Good.

The delta

The analysis makes a sharper distinction we should prove we honour per subsystem:

• Authorization / budget / policy backend unavailable → the request MUST be blocked (no successful reservation ⇒ no provider call).
• Observability (audit sink, metrics, OTel) unavailable → the request MAY continue (availability preserved), with the degradation recorded.

Failing the wrong way in either direction is a bug: blocking on a metrics outage kills availability; continuing on a policy-engine outage silently grants access.

Ask

Audit each hot-path dependency and classify it explicitly:

Dependency	On failure
budget reservation (spend store / policy cap)	block
policy engine / HIT authorization	block
deterministic DLP	block
secret backend (OpenBao/GrobStore) at request time	block
audit log write	continue + flag
metrics / OTel export	continue
pricing/token estimate	continue (degrade)

Then add tests that inject each failure and assert the classified behaviour (property test style, alongside tests/enterprise/property_budget_test.rs). Document the table in an ADR so the contract is explicit.

Priority

Medium-high. The code is likely already correct for most rows; the value is a proven, documented contract rather than incidental behaviour.

[Destynova2]

ADR-0030 (#487) documents the audited fail-closed contract — the code is already correct on every authorizing dependency. This issue now scopes down to the remaining gap: fault-injection tests for the dependency-unavailable variants (poisoned spend mutex, wedged policy backend), which the current tests don't cover (they assert decision outcomes, not dependency failures).