feat(evaluation): post-generation citation and support audit (ADR-0016 increment 6)

[brunovicco]

Summary

• Implements the post-generation citation auditor defined in ADR-0016: claim segmentation, deterministic checks (citation existence, document version, and presence in the retrieved context), semantic verification via an LLM—only for claims whose citations pass all deterministic checks—at most one rewrite followed by a full re-audit, and explicit abstention when no claims survive.
• The verifier and rewriter use OpenAI, independently of the Gemini model used by the generator. This follows the same rationale as ADR-0018: reducing correlated errors between the component that generates the answer and the one that audits it.
• The mechanism is fully implemented but disabled by default (audit.enabled: false). The ADR itself explicitly lists the additional cost and latency as a negative consequence.
• Also fixes an issue found along the way: extract_citations silently filtered out malformed bracketed citations, making well_formed=False unreachable in the auditor. Citation extraction was split into extract_citation_candidates, which does not filter candidates and is used by the auditor.

Details

• New evaluation/audit_ports.py: introduces AnswerClaim, DeterministicCitationCheck, SemanticSupportResult, ClaimAudit, AuditResult, the AuditOutcome, SupportVerdict, and TemporalStatus enums, and the SemanticSupportVerifier and AnswerRewriter protocols. Reuses ModelIdentity from judge_ports.py (Increment 5).
• New evaluation/citation_audit.py: implements deterministic sentence segmentation using a regex that correctly avoids splitting references such as "Art. 1º", per-citation checks, and the complete audit orchestration. temporal_status is always UNKNOWN for now because no reliable temporal corpus is available yet.
• New adapters: generation/openai_semantic_verifier.py and generation/openai_answer_rewriter.py. They reuse ragas.llms.InstructorLLM only as a structured-completion wrapper—not the RAGAS metrics—and follow the same caching and rate-limiting pattern used by the project’s other adapters.
• New generation/auditing_answer_generator.py: a decorator that implements the existing AnswerGenerator port. The calling logic in answer_harness.py and run.py remains unchanged; only the generator construction in run.py changes conditionally.
• New evaluation/audit_metrics.py: adds malformed-citation, nonexistent-citation, out-of-context citation, uncited-claim, and unsupported-claim rates, as well as rewrite rate, rewrite success rate, and abstention rate—all computed deterministically from AuditResult.
• run.py: adds a new audit configuration section, disabled by default. build_run_record always declares audit_enabled, audit_provider, and audit_model.
• Extracts generation/citation_parsing.py, shared by both the generator and the auditor.

Out of scope (deferred and documented in the code)

• Calibrating audit decisions against human judgments and publishing original-versus-audited answers across the full split. This requires genuine human curation, the same constraint that applies to every previous increment.
• false_abstention_rate, which requires human-labeled data.
• LLM-assisted claim segmentation. The ADR marks this as optional; the current implementation is fully deterministic.
• Enabling auditing by default in run.py. It remains configuration-driven and disabled by default.

Test plan

• uv run pytest — 405 passed, 20 integration tests deselected
• uv run mypy src tests — no issues found in 170 source files
• uv run python scripts/quality_gate.py — all checks pass: linting, formatting, architecture, MCP, governance, typing, tests, security/Bandit, and dependencies/pip-audit
• grep for _extract_citations and metrics/audit across src/ and tests/ — no orphaned references