From bc7d67e8a2fd454fb36486ce0206707a8b5962a1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 19 Jul 2026 15:24:01 +0000 Subject: [PATCH] docs: restore truth-gate snapshot/base-head invariants dropped by doc rewrite The `test` CI job is red across the entire open-PR queue on a single assertion in `test_agent_completion_gate.py::CompletionGateDocumentationTests:: test_operator_doc_defines_schemas_and_enforcement` (7424 pass, 1 fail). Commit 0b65565 ("fix(ci): bind truth-gate snapshots and evidence") introduced label-event-only snapshot creation, GITHUB_TOKEN markers that do not recursively trigger `issue_comment`, and resolve/collection/ publication-time base+head SHA equality (stale_base / stale_head), and updated both the workflow and the operator doc together. A later docs-only commit (21d94b5) rewrote the file and reverted those two paragraphs to their pre-0b65565 wording, dropping four phrases the test still asserts: - "Snapshot creation is label-event-only" - "does not recursively trigger `issue_comment`" - "Resolve-time, collection-time, and publication-time PR base and head" - "immutable resolved base/head commit comparison" The behaviours are still implemented in .github/workflows/pr-checks.yml (snapshot-agent-task-intent job, RESOLVED_BASE_SHA/RESOLVED_HEAD_SHA, stale_base/stale_head), so this restores the accurate description rather than weakening the test. Documentation-only change. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_0183kbEEPheuejZkhiLtrRTD --- docs/agent-completion-truth-gate.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/agent-completion-truth-gate.md b/docs/agent-completion-truth-gate.md index b860001e0..3345ab9e8 100644 --- a/docs/agent-completion-truth-gate.md +++ b/docs/agent-completion-truth-gate.md @@ -12,7 +12,7 @@ The trusted publisher must bind report data to PR number, full head SHA, deliver Before delegation, create the task with the Agent task issue form. Agent login, run ID, objective, acceptance criteria, exact file scope, allowed extras, and focused test paths are the intent contract. Unrestricted scope is intentionally unavailable in the form until #874 provisions the protected `scope-unrestricted-approved` label and its authorization policy; any hand-authored unrestricted request without that label fails closed. -When a complete agent task is opened or first labeled by an actor with a standard `triage`-or-higher role (or a custom role that GitHub reports at legacy `write` level), the default-branch workflow writes a github-actions[bot] comment containing the SHA-256 digest of the normalized issue body and the unrestricted-scope approval state. The same live permission lookup applies to both event paths; issue author association alone is not trusted. GitHub's collaborator response cannot distinguish a custom triage-derived role from a custom read-derived role, so custom roles reported at legacy `read` fail closed and a standard triage-or-higher user must relabel. The collector requires that snapshot and requires it to predate the PR. Any later issue edit or label transition appends a bot-owned invalidation marker. Treating every queued edit/label event as invalidating makes GitHub concurrency coalescing lossless: an event that replaces a pending invalidation is itself an invalidation. The trusted marker comment dispatches immediate reevaluation, and the scheduled scanner also blocks permanently even if the original body or label state is restored. Existing tasks must be labeled again by a trusted user to create their one-time snapshot before dispatch. Create a new task for any post-snapshot edit; do not broaden a dispatched task in place. +When a complete agent task receives its initial `agent-task` or `mcp/agent` label from an actor with a standard `triage`-or-higher role (or a custom role that GitHub reports at legacy `write` level), the default-branch workflow writes a github-actions[bot] comment containing the SHA-256 digest of the normalized issue body and the unrestricted-scope approval state. Snapshot creation is label-event-only because GitHub emits separate `opened` and `labeled` workflow runs for an issue form that applies a label. The snapshot records the creating workflow run ID so re-running that same event is idempotent. Issue author association alone is not trusted. GitHub's collaborator response cannot distinguish a custom triage-derived role from a custom read-derived role, so custom roles reported at legacy `read` fail closed and a standard triage-or-higher user must relabel. The collector requires that snapshot and requires it to predate the PR. Any later issue edit or label transition appends a bot-owned invalidation marker. Treating every queued edit/label event as invalidating makes GitHub concurrency coalescing lossless: an event that replaces a pending invalidation is itself an invalidation. A trusted originating issue event dispatches immediate reevaluation; an untrusted or unverifiable editor falls back to the scheduled scanner because a marker written with `GITHUB_TOKEN` does not recursively trigger `issue_comment`. The scanner blocks permanently even if the original body or label state is restored. Existing tasks must be relabeled by a trusted user to create their one-time snapshot before dispatch. Create a new task for any post-snapshot edit; do not broaden a dispatched task in place. Agent pull requests link exactly one task with a closing keyword and include the agent-lock-manifest comment shown in the PR template. GitHub's authoritative closingIssuesReferences, the textual link, and the manifest must agree. The manifest login and run ID must exactly match the snapshotted issue. The declared agent publishes structured result evidence containing that run ID and the current PR head SHA; legacy unstructured readiness is never sufficient by itself. @@ -27,7 +27,7 @@ The workflow publishes all of the following: Even in the normal trust model—agents cannot write default-branch workflows or forge repository statuses—the custom status emitted here remains advisory. Follow-up #874 must bind evaluation to an independently head-bound required workflow or check before branch protection or a repository ruleset treats the result as merge enforcement. That ruleset must also require the repository's Copilot review, at least one approving review, and conversation resolution. The gate itself requires the maintainer-applied `copilot-rabbit` label, a non-dismissed Copilot review bound to the current head, every AI review thread resolved (including outdated threads), and committed focused unit tests. It binds to the exact-head trusted CI run, requires its `test` job to succeed, and requires that job's verbose pytest log to report at least one passing test for every declared path; an absent, deselected, or all-skipped path blocks. Human approval alone cannot satisfy those signals. Native review/conversation rules close the window between a new review comment and the scheduled refresh. -Each serialized run first posts a pending status tied to its Actions run. It uploads evidence and updates the PR comment before publishing a terminal status. A compare-and-swap check rejects superseded publication; an always-running finalizer turns publication failures into a failure status only when the run proves it owns the pending lease (or supersedes a predecessor). Unknown ownership fails the workflow without touching the shared status, and the scheduled scanner supplies eventual fail-closed recovery. A recovered infrastructure/publication failure remains failed until a new evidence event or a maintainer manually dispatches the workflow; the scanner does not churn unchanged evidence merely to retry it. GitHub permits at most 1,000 statuses for one SHA and context, so the workflow refuses to acquire a new lease at 998 existing per-PR gate statuses. The scanner may use a remaining slot to fail a proven stuck pending lease or invalidate a successful terminal result after evidence changes, but it never starts another full evaluation at that threshold; the operator must then push a new head or complete #874's Checks/App migration. If pending-status creation fails on a reused head, a previous successful status can remain visible until recovery; overwriting it without a proven lease could instead clobber a successor. This transient stale-green window is why this status is advisory until #874 supplies independently head-bound enforcement. Actions are pinned to full commit SHAs. Only refresh-dispatch jobs receive actions: write. +Each serialized run first posts a pending status tied to its Actions run. It uploads evidence and updates the PR comment before publishing a terminal status. Resolve-time, collection-time, and publication-time PR base and head commits must each remain the same 40-character SHA; a mismatch publishes `stale_base` or `stale_head` instead of reusing evidence across revisions. Changed-file evidence comes from the immutable resolved base/head commit comparison rather than the mutable live PR file list. A compare-and-swap check rejects superseded publication; an always-running finalizer turns publication failures into a failure status only when the run proves it owns the pending lease (or supersedes a predecessor). Unknown ownership fails the workflow without touching the shared status, and the scheduled scanner supplies eventual fail-closed recovery. A recovered infrastructure/publication failure remains failed until a new evidence event or a maintainer manually dispatches the workflow; the scanner does not churn unchanged evidence merely to retry it. GitHub permits at most 1,000 statuses for one SHA and context, so the workflow refuses to acquire a new lease at 998 existing per-PR gate statuses. The scanner may use a remaining slot to fail a proven stuck pending lease or invalidate a successful terminal result after evidence changes, but it never starts another full evaluation at that threshold; the operator must then push a new head or complete #874's Checks/App migration. If pending-status creation fails on a reused head, a previous successful status can remain visible until recovery; overwriting it without a proven lease could instead clobber a successor. This transient stale-green window is why this status is advisory until #874 supplies independently head-bound enforcement. Actions are pinned to full commit SHAs. Only refresh-dispatch jobs receive actions: write. Per-path focused-test proof currently parses terminal pytest outcomes from the exact-head trusted CI `test` job log. It handles both same-line verbose outcomes and node IDs followed by live-log output and a standalone terminal outcome. Exact path prefixes, whole-line terminal matching, test-node resets, and session-boundary resets prevent unrelated output from inheriting a prior test. A same-line candidate is deferred across blank lines so a following live-log header can disambiguate a parameter ID that resembles an outcome. Code executed by pytest can still deliberately forge console lines, so the required independent review remains part of the trust boundary. Follow-up #874 must move per-path evidence to a machine-readable report emitted and published by the protected CI workflow before this signal is treated as hostile-agent-resistant.