fix(ci): decide truth-gate applicability from PR provenance or a declared issue contract - #1364
Closed
groupthinking wants to merge 1 commit into
Closed
Conversation
agentTaskApplicable() unioned PR labels with linked-issue labels, so a bare agent-task/mcp-agent label applied by label automation to an issue that never declared a contract judged any closing PR an agent completion. The gate then required an Agent Run ID / Agent Login the issue never declared, yielding a permanent blocked/invalid_payload verdict no author could satisfy. Applicability now comes from PR provenance (known agent author, agent branch prefix, agent label on the PR, lock manifest) or from a genuine issue-side dispatch: an agent-task/mcp-agent label AND declared Agent Run ID + Agent Login headings. The generic agent label remains a PR-side signal only, since the snapshot job and collector never recognise it issue-side — arming from it would block permanently as linked_issue_not_agent_task with no snapshot to satisfy. The collector now emits a mislabelled_agent_task core.notice when a linked issue carries a contract label without declaring the contract, keyed on the missing contract itself (not inapplicability) so Dependabot PRs linked to valid contracts never get a false notice. Generated with [Linear](https://linear.app/myxstack/issue/GRV-196/agent-completiontruth-gate-is-permanently-unsatisfiable-for-any-pr#agent-session-8e5d62f0) Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
Contributor
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited) Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
Agent Completion Truth Gate: BLOCKEDReasons: Machine-readable verdict{
"details": {
"collection_errors": [
"incomplete_linked_issue_contract",
"missing_intent_snapshot",
"missing_agent_run_id",
"missing_agent_login"
],
"invalid_fields": [
"policy.agent_login",
"policy.run_id"
]
},
"reasons": [
"invalid_payload"
],
"verdict": "blocked"
} |
Merged
11 tasks
groupthinking
pushed a commit
that referenced
this pull request
Aug 5, 2026
The truth gate scores a pull request against the frozen intent snapshot on its linked issue. That snapshot is only ever written by snapshot-agent-task-intent, which runs on `issues` events alone -- it never runs on pull_request_target. So a pull request with no linked issue has no snapshot, no declared agent_login and no declared run_id, which means policy.agent_login, policy.run_id and issue.number can never be populated and the verdict is permanently `invalid_payload` no matter what the author does. Arming that unsatisfiable state from a branch-name prefix made the check red on pull requests that never had a contract to satisfy. It is red on merged commits too: #1368, the current tip of main, merged with agent-completion/truth-gate/pr-1368 failing on exactly this. A check that is red on everything gates nothing and buries real failures, which is the failure mode agent-completion-enforcement.yml already warns about in its own comments. Pull-side provenance now arms the gate only when a linked issue exists to verify against. With none there is nothing to measure, so the verdict is not_applicable rather than blocked. This is not an escape hatch: a pull request that links a dispatched issue is gated exactly as before, and the requirement to bind a pull request to a focused issue at all is separately owned by the `Canonical issue and evidence` check, which states a requirement an author can actually meet. Consolidates the two competing open implementations of this fix. #1364's commit is cherry-picked here with authorship intact; #1154 carried the same intent but had drifted to 117 files and 14k lines of unrelated changes. Both should close in favour of this. Full unit suite: 8079 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi
8 tasks
Owner
Author
Closing — consolidated into #1377, with this commit cherry-picked intactYour approach won. Of the two competing implementations of this fix, this one is clearly the better:
Commit #1377 adds the one piece this PR did not cover. This PR correctly stopped a bare issue label from arming the gate, but pull-side provenance ( That case is not hypothetical — it is why Closing here only because the work now lives in #1377; consolidating rather than stacking a third PR on the same surface. #1154 is closed for the same fix. Generated by Claude Code |
8 tasks
5 tasks
groupthinking
added a commit
that referenced
this pull request
Aug 5, 2026
… audit 337 branches (#1377) * test: stop asserting a CPython-version-specific rmtree detail test_cleanup_is_total_for_non_oserror_failures failed on Python 3.11.15: its premise asserted that shutil.rmtree(path, ignore_errors=True) raises ValueError on a NUL-byte path. That is an implementation detail which has changed -- ignore_errors now absorbs the non-OSError as well, so the assertion no longer holds. The contract under test is unaffected: _cleanup_download_artifacts must swallow non-OSError failures because it runs from a finally block and would otherwise replace the in-flight exception. Establish that premise against the unguarded rmtree call, which still raises, so the test proves the helper's own defensiveness rather than the stdlib's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi * fix(ci): stop arming truth gate from bare agent labels on linked issues agentTaskApplicable() unioned PR labels with linked-issue labels, so a bare agent-task/mcp-agent label applied by label automation to an issue that never declared a contract judged any closing PR an agent completion. The gate then required an Agent Run ID / Agent Login the issue never declared, yielding a permanent blocked/invalid_payload verdict no author could satisfy. Applicability now comes from PR provenance (known agent author, agent branch prefix, agent label on the PR, lock manifest) or from a genuine issue-side dispatch: an agent-task/mcp-agent label AND declared Agent Run ID + Agent Login headings. The generic agent label remains a PR-side signal only, since the snapshot job and collector never recognise it issue-side — arming from it would block permanently as linked_issue_not_agent_task with no snapshot to satisfy. The collector now emits a mislabelled_agent_task core.notice when a linked issue carries a contract label without declaring the contract, keyed on the missing contract itself (not inapplicability) so Dependabot PRs linked to valid contracts never get a false notice. Generated with [Linear](https://linear.app/myxstack/issue/GRV-196/agent-completiontruth-gate-is-permanently-unsatisfiable-for-any-pr#agent-session-8e5d62f0) Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com> * fix(ci): require a linked issue before arming the truth gate The truth gate scores a pull request against the frozen intent snapshot on its linked issue. That snapshot is only ever written by snapshot-agent-task-intent, which runs on `issues` events alone -- it never runs on pull_request_target. So a pull request with no linked issue has no snapshot, no declared agent_login and no declared run_id, which means policy.agent_login, policy.run_id and issue.number can never be populated and the verdict is permanently `invalid_payload` no matter what the author does. Arming that unsatisfiable state from a branch-name prefix made the check red on pull requests that never had a contract to satisfy. It is red on merged commits too: #1368, the current tip of main, merged with agent-completion/truth-gate/pr-1368 failing on exactly this. A check that is red on everything gates nothing and buries real failures, which is the failure mode agent-completion-enforcement.yml already warns about in its own comments. Pull-side provenance now arms the gate only when a linked issue exists to verify against. With none there is nothing to measure, so the verdict is not_applicable rather than blocked. This is not an escape hatch: a pull request that links a dispatched issue is gated exactly as before, and the requirement to bind a pull request to a focused issue at all is separately owned by the `Canonical issue and evidence` check, which states a requirement an author can actually meet. Consolidates the two competing open implementations of this fix. #1364's commit is cherry-picked here with authorship intact; #1154 carried the same intent but had drifted to 117 files and 14k lines of unrelated changes. Both should close in favour of this. Full unit suite: 8079 passed, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi * chore(maintenance): audit all 337 remote branches and add a prune script Classifies every remote branch using the one signal that stays honest after the secret-purge force-push: whether it shares any ancestry with `main`. $ git merge-base origin/main origin/<branch> (empty) 275 of 337 branches return empty -- they predate the rewrite and no rebase recovers them. The signals the branch-cleanup harness normally leans on all mislead here, so they are deliberately not used: * `git merge-tree` calls these orphans a CLEAN merge; unrelated trees do not textually conflict, they would clobber. * A two-dot diff against an empty merge base silently degrades to a working-tree diff, which is why a two-line Dependabot bump measures as 111 files / 15,650 lines. * The purge rewrote committer dates, so every branch reads as under 30 days old and no staleness threshold ever fires. Running the stock harness on this repo produced 263 REVIEW off those bad signals. The ancestry test resolves the same set into: KEEP-OPEN-PR 28 REVIEW-SHARED 29 real shared ancestry, no open PR -- not pruned CLOSE-MERGED 1 tip is an ancestor of main CLOSE-ORPHANED 275 The script prunes only the last two groups (276 branches) and archive-tags each one first, verifying every tag is on the remote before deleting anything. It defaults to a dry run. That dry run caught the audit classifying `main` itself as CLOSE-MERGED -- `git merge-base --is-ancestor origin/main origin/main` is trivially true. The row is removed, and a protected-ref guard plus a default-branch check now abort regardless of what the CSV contains. Not executed here: this session's credentials are scoped to one branch and tag creation fails with HTTP 403, so the archive tags cannot be written. Deleting without them would remove the only durable recovery path, so nothing was deleted. Run the script with tag-write credentials. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi * docs(maintenance): correct prune-script header counts to match the CSV The header summary read "CLOSE-MERGED 2 / 277 branches" while the shipped docs/branch-audit-2026-08-05.csv has 1 CLOSE-MERGED and 276 prunable. The header was written before the `main` row was dropped from the CSV and was never updated. Comment only -- the selection logic already read from the CSV, so the script was correctly selecting 276 the whole time. Verified: header, CSV tally, and the script's own dry-run count now all agree at 276. Reported by the Vercel review bot on #1377. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YcHjCZ6pGn6A5BeeoZ6eZi --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: linear-code[bot] <222613912+linear-code[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canonical issue
Closes #1130
Outcome
Human PRs closing issues that label automation mislabelled
agent-task/mcp-agentare no longer judged agent completions, soagent-completion/truth-gatestops emitting permanent, unsatisfiableblocked/invalid_payloadverdicts on them. Genuine agent work stays gated exactly as before.Scope
agentTaskApplicable()copies in.github/workflows/pr-checks.yml(kept byte-identical); amislabelled_agent_taskcore.noticein the collector; regression tests intests/unit/test_agent_completion_gate.py.linear-code[bot]labelling rule (external), trust-policy provisioning (Report Agent Lock gate neutral when trust policy unprovisioned #1222 / GRV-240), and verdict diagnostics (GRV-271).Applicability now requires either PR provenance (known agent author, agent branch prefix, agent label on the PR,
agent-lock-manifestin the body) or a genuine issue-side dispatch: anagent-task/mcp-agentlabel and declaredAgent Run ID+Agent Loginheadings. A bare label with no contract no longer arms the gate.This supersedes stalled draft #1154 and resolves both of its unresolved review findings:
PRRT_kwDORAYbZs6Vqi8J— the issue-side check accepts only the two contract labels (agent-task,mcp-agent); the genericagentlabel stays PR-side only, since the snapshot job and collector never recognise it on issues and arming from it would block permanently aslinked_issue_not_agent_task.PRRT_kwDORAYbZs6Vqi8L— the mislabel notice is keyed on the missing contract itself rather than on!applicable, so a Dependabot PR linked to a valid contract never receives a false notice.Risk
Verification
pytest tests/unit/test_agent_completion_gate.py— 106 passed, 79 subtests passed, including newtest_agent_applicability_requires_provenance_or_declared_contract,test_agent_applicability_copies_stay_identical, andtest_mislabelled_agent_task_is_reported_rather_than_blocked; workflow YAML validated.Note: this PR is itself armed by the truth gate (branch prefix
agent-…, under both old and new logic) whilepr-checks.ymlruns from the base branch, so its ownagent-completion/truth-gatestatus is expected to stay red until maintainers handle it (the check is not merge-blocking; see the self-blocking discussion on #1130).Production evidence
Not applicable — CI workflow logic only; behaviour is exercised by the
node -eextraction harness in the unit tests.Agent handoff
Agent provenance
No
agent-lock-manifestis declared: the linked issue #1130 declares no dispatch contract (Agent Run ID/Agent Login), and self-issuing one for a dispatch that never happened is exactly the fabrication the gate exists to reject.