Skip to content

[ci] "Agent completion enforcement" fails on 100% of pull requests — trust policy is unprovisioned #1160

Description

@groupthinking

Summary

The Agent completion enforcement check fails on every open pull request — human-authored and agent-authored alike — with missing_trusted_publication. It has never passed for any PR since it landed.

This issue is filed for a human decision. I deliberately did not "fix" it, because the failure is the documented, intentional behaviour of a security control, and resolving it requires provisioning a GitHub App. See "Why this was not auto-remediated" below.

Evidence

Surveyed all 14 open PRs at the time of filing:

PR Author Agent completion enforcement
1159 groupthinking fail
1156 groupthinking fail
1155 groupthinking fail
1154 groupthinking fail
1152 groupthinking fail
1151 groupthinking fail
1145 app/google-labs-jules fail
1142 groupthinking fail
1132 app/copilot-swe-agent fail
1129 groupthinking fail
1128 groupthinking fail
1123 groupthinking fail
1122 groupthinking fail
1119 groupthinking fail

100% failure, including the two genuine agent PRs (#1145, #1132) that the control exists to verify.

Job error:

##[error]missing_trusted_publication

Root cause

.github/workflows/agent-completion-enforcement.yml requires a Check run named Agent Lock trusted publication, published on the PR head SHA by a trusted GitHub App:

gh api "repos/$REPO/commits/$head/check-runs" \
  --jq '.check_runs[] | select(.name == "Agent Lock trusted publication") | .output.text' > trusted-report.json
if test -s trusted-report.json; then
  python3 scripts/ci/agent_completion_enforcement.py ...
else
  printf '%s\n' '{"conclusion":"failure","reason":"missing_trusted_publication","details":{}}' > enforcement-verdict.json
fi

No such App publishes that Check, so the else branch is taken every time.

The verifier is designed to handle human PRs — scripts/ci/agent_completion_enforcement.py:48 accepts applicability.state of either agent or human_exempt. But a human_exempt report still has to be published by the trusted App, and attested by an actor in trusted_human_exemption_actors. With no App, neither path can produce a report.

.github/agent-lock/trusted-publishers.json has all three allowlists empty:

{
  "trusted_check_app_slugs": [],
  "trusted_label_actors": [],
  "trusted_human_exemption_actors": [],
  "custom_role_policy": "fail_closed",
  "notes": "Populate all three allowlists only through a protected default-branch change after the independent GitHub App and its identity have been verified. An empty allowlist intentionally blocks rather than downgrading agent work to not_applicable."
}

The verifier would reject an empty policy anyway (trust_policy_unprovisioned, line 37-38) even if a report did arrive.

So this is a provisioning gap, not a logic defect. The control is behaving exactly as designed and documented.

Impact

main has no required status checks:

$ gh api repos/groupthinking/EventRelay/branches/main/protection \
    --jq 'has("required_status_checks")'
false
$ gh api repos/groupthinking/EventRelay/rulesets
[]

So the check blocks nothing, and PRs continue to merge (#1090, #1092, #1094, #1098, #1103, #1108 all merged after this control landed on 2026-07-18).

That makes the severity governance, not availability:

  1. The agent-lock control provides no actual assurance today. 🛡️ Sentinel: [MEDIUM] Fix internal error message leakage in API routes #1145 and fix(cloud): authenticate task requests before payload validation (#1134) #1132 are real agent PRs and were not verified by it.
  2. A permanently-red check on every PR trains reviewers to ignore red checks — which erodes the signal value of the checks that do work.

Why this was not auto-remediated

The obvious change — treat a missing publication as not_applicable for human PRs — is the one thing the trust policy explicitly forbids:

"An empty allowlist intentionally blocks rather than downgrading agent work to not_applicable."

Making that change unilaterally would disable a security control against its author's stated intent. This needs an owner decision, not an agent's judgement call.

Note this is a different defect class from #1130. There, the applicability logic was genuinely wrong and no author could act on the verdict, so it was safe to correct. Here the logic is right and the trust anchor is simply absent.

Decision options

  1. Provision the App. Install the intended GitHub App, verify its identity, then populate trusted_check_app_slugs, trusted_label_actors and trusted_human_exemption_actors via a protected default-branch change. The control starts working as designed. This is the only option that delivers the assurance the control was built for.
  2. Gate applicability before requiring evidence. Skip the trusted-publication requirement for PRs with no agent dispatch evidence, mirroring agentTaskApplicable in pr-checks.yml. Removes the noise and keeps fail-closed for agent PRs, but is a deliberate, explicit relaxation of the documented posture and should be recorded as such.
  3. Disable the workflow until it can be provisioned. Honest about the current state and stops the noise, but leaves agent PRs unverified with no visible reminder.

Option 1 is the intended end state. Option 3 is preferable to leaving it as-is if provisioning is not imminent, because a check that always fails is indistinguishable from a check that is failing for a real reason.

Suggested acceptance criteria

  • Agent completion enforcement reaches a non-failure conclusion on a human-authored PR.
  • A genuine agent PR is still blocked when its trusted publication is absent or malformed.
  • Whichever option is chosen is recorded in .github/agent-lock/trusted-publishers.json notes or the workflow header, so the next reader knows the posture is intentional.

Found while resolving #1130, which is the same symptom class (an unsatisfiable required check) from a different cause.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions