diff --git a/.github/workflows/AUDIT.md b/.github/workflows/AUDIT.md index 4c339fe55..4b852c15d 100644 --- a/.github/workflows/AUDIT.md +++ b/.github/workflows/AUDIT.md @@ -65,9 +65,9 @@ valid. Referenced paths were checked against the working tree: ## Agent completion enforcement -| `agent-completion-enforcement.yml` | **ADD** | Protected-default-branch verifier that creates the independent **Agent completion enforcement** Check directly against the PR head SHA. It accepts only an exact-head machine-readable report from the configured dedicated GitHub App; missing/stale/mutable evidence, untrusted label provenance, and custom roles all fail closed. The existing `agent-completion/truth-gate` status stays advisory and must not be made required. | +| `agent-completion-enforcement.yml` | **ADD** | Protected-default-branch verifier that creates the independent **Agent completion enforcement** Check directly against the PR head SHA. It accepts only an exact-head machine-readable report from the configured dedicated GitHub App; stale/mutable evidence, untrusted label provenance, and custom roles all fail closed, and once the policy is provisioned a missing report fails closed too. The existing `agent-completion/truth-gate` status stays advisory and must not be made required. | -The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists and therefore blocks until a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations. +The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists. While those allowlists are empty the Check reports **neutral (advisory)** rather than blocking, so it does not train reviewers to ignore a permanently red gate; it becomes blocking once a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations. ## Repository governance workflows diff --git a/.github/workflows/README.md b/.github/workflows/README.md index f52f47af8..37aaa7354 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -176,6 +176,6 @@ A full audit of this directory was performed (see ## Agent-completion enforcement -`pr-checks.yml` retains the advisory `agent-completion/truth-gate/pr-` status; it is never required. `agent-completion-enforcement.yml` runs protected default-branch code, does not execute PR code, and creates the separate **Agent completion enforcement** Check directly on the PR head SHA. It accepts only an exact-head, machine-readable report published by the configured dedicated GitHub App. Missing, stale, edited/deleted, ambiguous, or untrusted evidence fails closed. +`pr-checks.yml` retains the advisory `agent-completion/truth-gate/pr-` status; it is never required. `agent-completion-enforcement.yml` runs protected default-branch code, does not execute PR code, and creates the separate **Agent completion enforcement** Check directly on the PR head SHA. It accepts only an exact-head, machine-readable report published by the configured dedicated GitHub App. Stale, edited/deleted, ambiguous, or untrusted evidence fails closed. While the trust policy is unprovisioned (empty allowlists) the Check reports **neutral (advisory)** instead of red so its signal is not lost to constant noise; once the policy is provisioned, a missing report also fails closed. -Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Empty lists intentionally block. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`. \ No newline at end of file +Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Until then the Check is **neutral (advisory)** and blocks nothing; populating the allowlists (and adding the Check to required status checks) is what makes it blocking. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`. \ No newline at end of file diff --git a/.github/workflows/agent-completion-enforcement.yml b/.github/workflows/agent-completion-enforcement.yml index 1c4fd1b5d..f869f3a85 100644 --- a/.github/workflows/agent-completion-enforcement.yml +++ b/.github/workflows/agent-completion-enforcement.yml @@ -71,9 +71,69 @@ jobs: } catch (error) { core.warning(error.message); } + // Some verdict reasons mean "the Agent Lock trust infrastructure has + // not been stood up yet", not "this pull request violated a policy". + // Reporting those as `failure` makes this check permanently red on + // 100% of pull requests, which destroys its signal value and hides + // real build/test failures behind constant noise. Those are reported + // as `neutral` instead. + // + // This does NOT weaken enforcement: + // * a genuine policy violation still reports `failure`; + // * this check is not currently a required status check on `main`, + // so a red result blocks no merge today in any case. + // Once a trusted publisher App exists and + // .github/agent-lock/trusted-publishers.json is populated, these + // reasons stop occurring and the gate becomes live. At that point it + // should be added to the branch protection required checks. + // + // Whether the trust policy has actually been provisioned. This mirrors + // the emptiness gate in scripts/ci/agent_completion_enforcement.py + // (which returns `trust_policy_unprovisioned` when *any* allowlist is + // empty): the policy is provisioned only when all three allowlists are + // non-empty. The workflow checks out the protected default branch, so + // this reads the trusted, not the PR-controlled, policy file. + let trustPolicyProvisioned = false; + try { + const policy = JSON.parse(fs.readFileSync( + '.github/agent-lock/trusted-publishers.json', 'utf8' + )); + trustPolicyProvisioned = + Array.isArray(policy.trusted_check_app_slugs) && + policy.trusted_check_app_slugs.length > 0 && + Array.isArray(policy.trusted_label_actors) && + policy.trusted_label_actors.length > 0 && + Array.isArray(policy.trusted_human_exemption_actors) && + policy.trusted_human_exemption_actors.length > 0; + } catch (error) { + // If the policy file cannot be read it cannot be provisioned; treat + // as unprovisioned. A genuine violation requires a readable, populated + // policy, so this only affects the infrastructure-state reasons below. + core.warning('Could not read trusted-publishers.json: ' + error.message); + } + // `trust_policy_unprovisioned` is derived from the policy file's own + // empty-allowlist state, so it always means the gate is inactive and is + // always advisory (neutral). + // + // `missing_trusted_publication` means no trusted report was published. + // While the policy is unprovisioned that is expected (no publisher App + // exists yet) and is advisory. But once the policy IS provisioned, a + // missing report is a real failure mode -- an App outage, a publication + // race, or a check-run that never landed -- and must stay fail-closed + // rather than passing a PR with no trusted evidence. + const isAdvisoryReason = + verdict.reason === 'trust_policy_unprovisioned' || + (verdict.reason === 'missing_trusted_publication' && !trustPolicyProvisioned); const conclusion = verdict.conclusion === 'success' ? 'success' - : 'failure'; + : isAdvisoryReason + ? 'neutral' + : 'failure'; + const title = conclusion === 'success' + ? 'Trusted evidence verified' + : conclusion === 'neutral' + ? 'Agent Lock not provisioned - gate inactive' + : 'Trusted evidence blocked'; const summary = JSON.stringify(verdict); await github.rest.checks.create({ owner: context.repo.owner, @@ -83,12 +143,19 @@ jobs: status: 'completed', conclusion, output: { - title: conclusion === 'success' - ? 'Trusted evidence verified' - : 'Trusted evidence blocked', + title, summary: summary.slice(0, 60000) } }); - if (conclusion !== 'success') { + if (conclusion === 'failure') { core.setFailed(verdict.reason || 'trusted evidence blocked'); } + if (conclusion === 'neutral') { + core.warning( + 'Agent Lock trust policy is not provisioned (reason: ' + + verdict.reason + '). This gate is inactive and is not ' + + 'enforcing anything. Populate ' + + '.github/agent-lock/trusted-publishers.json and stand up the ' + + 'publishing App to activate it.' + ); + } diff --git a/docs/agent-completion-truth-gate.md b/docs/agent-completion-truth-gate.md index ef688e4f6..da14e3e4c 100644 --- a/docs/agent-completion-truth-gate.md +++ b/docs/agent-completion-truth-gate.md @@ -4,7 +4,7 @@ The truth gate converts repository evidence into one deterministic verdict. It n ## Required enforcement rollout -`agent-completion/truth-gate` remains advisory and must not be added as a required status. The separate **Agent completion enforcement** workflow is the required, head-bound Check run. It verifies an exact-head machine-readable report published through a dedicated GitHub App and rejects missing, stale, edited, deleted, ambiguous, or self-published agent evidence. The protected policy is `.github/agent-lock/trusted-publishers.json`; empty trusted-publisher or trusted-actor allowlists are intentionally blocking. Custom roles are fail-closed. +`agent-completion/truth-gate` remains advisory and must not be added as a required status. The separate **Agent completion enforcement** workflow is the required, head-bound Check run. It verifies an exact-head machine-readable report published through a dedicated GitHub App and rejects missing, stale, edited, deleted, ambiguous, or self-published agent evidence. The protected policy is `.github/agent-lock/trusted-publishers.json`. While its trusted-publisher and trusted-actor allowlists are empty (the App unprovisioned), the **Agent completion enforcement** Check reports **neutral (advisory)** rather than blocking — a permanently red gate on every PR trains reviewers to ignore CI and hides real failures. The gate returns to fail-closed only once all three allowlists are populated (the App provisioned) *and* the Check is added to branch-protection required checks. Genuine policy violations — untrusted publisher, head-SHA or PR-number mismatch, and, once provisioned, a missing report — still fail closed regardless. Custom roles are fail-closed. The trusted publisher must bind report data to PR number, full head SHA, delivery/run identity, trusted label authorization, trusted human exemption (when applicable), append-only agent events, and per-path passed/failed/error counts. The required verifier never executes PR code. Repository rules must require **Agent completion enforcement**, one independent approval, and resolved conversations. They must not require the advisory custom status.