Skip to content

fix(ci): arm the truth gate only on a real dispatch contract #1409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
groupthinking wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix(ci): arm the truth gate only on a real dispatch contract #1409

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
aaca52f
fix(ci): arm the truth gate only on a real dispatch contract
claude Aug 7, 2026
384be14
fix(ci): require the complete dispatch contract to arm the truth gate
claude Aug 7, 2026
f14ea74
docs: name the dispatch labels exactly in the truth-gate applicabilit…
claude Aug 7, 2026
0be4d0a
fix(ci): align the snapshot placeholder test with the gate's
claude Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
242 changes: 143 additions & 99 deletions .github/workflows/pr-checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,18 @@ jobs:
}
return output.join('\n').trim();
}
// Strips outer backticks before the placeholder test, matching
// `declared` in `agentTaskApplicable`. Issue forms render an
// unfilled field as `_No response_`, and authors routinely wrap
// values in backticks, so a field left unfilled inside a code span
// arrives as '`_No response_`'. Without the strip that reads as a
// real answer here while the gate's predicate rejects it -- the
// snapshot would be written but the gate would never arm, letting
// a malformed dispatch skip the check entirely. Both predicates
// must agree on every input.
function hasResponse(value) {
const response = String(value || '').trim();
const response = String(value || '')
.replace(/^\x60|\x60$/g, '').trim();
return Boolean(response && response !== '_No response_');
}
function checkboxChecked(value) {
Expand Down Expand Up @@ -646,39 +656,51 @@ jobs:
}
return output.join('\n').trim();
}
const body = String((issue && issue.body) || '');
function declared(headings) {
const value = section(
String((issue && issue.body) || ''), headings
).replace(/^\x60|\x60$/g, '').trim();
const value = section(body, headings)
.replace(/^\x60|\x60$/g, '').trim();
return Boolean(value) && value !== '_No response_';
}
function checked(headings) {
const value = section(body, headings).trim();
return value.split(/\r?\n/).some(line =>
/^\s*[-*]\s*\[[xX]\]/.test(line)
) || /^(?:yes|true)$/i.test(value);
}
// This must mirror the `complete` predicate in
// `snapshot-agent-task-intent`. That job refuses to write a
// snapshot (`incomplete_agent_task_contract`) unless every
// one of these holds, and a gate armed with no snapshot is
// permanently `invalid_payload`. Checking only run id and
// login armed the gate on issues the snapshot job rejects --
// the same unsatisfiable shape this rule exists to prevent,
// one level down.
const unrestrictedRequested =
checked(['unrestricted scope', 'scope unrestricted']);
const labelSource = issue && issue.labels
? Array.isArray(issue.labels)
? issue.labels
: issue.labels.nodes || []
: [];
if (unrestrictedRequested &&
!carriesLabel(labelSource,
['scopeunrestrictedapproved'])) {
return false;
}
return declared(['agent run id', 'run id']) &&
declared(['agent login']);
declared(['agent login']) &&
declared(['objective', 'description']) &&
declared(['acceptance criteria', 'acceptance tests']) &&
(declared(['declared file scope', 'file scope', 'scope']) ||
unrestrictedRequested) &&
/-\s*\[[xX]\]/.test(
section(body, ['pre-dispatch confirmation'])
);
}
const login = String(
pull && pull.user && pull.user.login || ''
);
const knownAgents = new Set([
'google-labs-jules[bot]',
'github-copilot[bot]',
'copilot-swe-agent[bot]',
'openai-codex[bot]',
'chatgpt-codex-connector[bot]'
]);
const agentBranch =
/^(?:agent|claude|codex|copilot|jules)[/-]/i.test(
String(pull && pull.head && pull.head.ref || '')
);
const manifestPresent =
/<!--\s*agent-lock-manifest\s*[\s\S]*?-->/i.test(
String(pull && pull.body || '')
);
// Provenance asserted by the pull request itself. Each of these
// is a claim by the producing side that this is agent work.
const pullProvenance = knownAgents.has(login) || agentBranch ||
manifestPresent ||
carriesLabel((pull && pull.labels) || [],
['agent', 'agenttask', 'mcpagent']);
// Issue-side dispatch. Label automation also applies agent task
// labels as topic tags to issues that never declared a contract,
// so the bare label is not evidence of a dispatch: it only
Expand All @@ -698,31 +720,36 @@ jobs:
const issueDispatch =
carriesLabel(issueLabelSource, ['agenttask', 'mcpagent']) &&
declaresAgentContract(selectedIssue);
// Pull-side provenance says who produced the branch. It is not
// evidence that a dispatch contract exists to measure that
// branch against. The gate scores a pull request against the
// frozen intent snapshot on its linked issue, and that snapshot
// is only ever written by `snapshot-agent-task-intent`, which
// runs on `issues` events alone. With no linked issue there is
// no snapshot, no declared run id and no declared login, so
// `policy.agent_login`, `policy.run_id` and `issue.number` are
// all unsatisfiable and the verdict is permanently
// `invalid_payload` regardless of what the author does. A
// branch named `claude/...` is a naming convention, not a
// dispatch. Arming on it alone is what made this check red on
// pull requests that never had a contract to satisfy -- including
// #1368, which merged with this status failing.
// Only an issue-side dispatch arms the gate. Pull-side
// provenance says who produced the branch; it is not evidence
// that a dispatch contract exists to measure that branch
// against. The gate scores a pull request against the frozen
// intent snapshot on its linked issue, and that snapshot is only
// ever written by `snapshot-agent-task-intent`, which runs on
// `issues` events alone, and only for issues labelled
// `agent-task`/`mcp-agent` that already declare a run id and
// login. Without that snapshot `policy.agent_login` and
// `policy.run_id` are unsatisfiable and the verdict is
// permanently `invalid_payload` regardless of what the author
// does. A branch named `claude/...` is a naming convention, not
// a dispatch.
//
// Arming on `pullProvenance && selectedIssue` -- provenance plus
// *any* linked issue -- put this check in direct contradiction
// with `PR Governance`, which requires exactly one
// `Closes #<issue>` reference. Satisfying one guaranteed failing
// the other: every well-formed agent pull request was armed
// against a contract that had never been written, so the gate
// was red on ~100% of pull requests, including merged ones
// (#1368, #1408). Requiring a real dispatch instead restores the
// #1130 reasoning to the arming rule that overrode it.
//
// So provenance arms the gate only once a linked issue exists to
// verify against; with none, there is nothing to measure and the
// verdict is `not_applicable`. This does not create an escape
// hatch: a pull request that links a dispatched issue is still
// fully gated, and requiring a pull request to bind to a focused
// issue at all is separately owned by `Canonical issue and
// evidence`, which states a requirement an author can actually
// meet.
return login !== 'dependabot[bot]' &&
(issueDispatch || (pullProvenance && Boolean(selectedIssue)));
// This is not an escape hatch: a pull request that links a
// genuinely dispatched issue is still fully gated, and requiring
// a pull request to bind to a focused issue at all is separately
// owned by `Canonical issue and evidence`, which states a
// requirement an author can actually meet.
return login !== 'dependabot[bot]' && issueDispatch;
}
function intentContractErrors(issue, comments, pullCreatedAt) {
const errors = [];
Expand Down Expand Up @@ -1972,39 +1999,51 @@ jobs:
}
return output.join('\n').trim();
}
const body = String((issue && issue.body) || '');
function declared(headings) {
const value = section(
String((issue && issue.body) || ''), headings
).replace(/^\x60|\x60$/g, '').trim();
const value = section(body, headings)
.replace(/^\x60|\x60$/g, '').trim();
return Boolean(value) && value !== '_No response_';
}
function checked(headings) {
const value = section(body, headings).trim();
return value.split(/\r?\n/).some(line =>
/^\s*[-*]\s*\[[xX]\]/.test(line)
) || /^(?:yes|true)$/i.test(value);
}
// This must mirror the `complete` predicate in
// `snapshot-agent-task-intent`. That job refuses to write a
// snapshot (`incomplete_agent_task_contract`) unless every
// one of these holds, and a gate armed with no snapshot is
// permanently `invalid_payload`. Checking only run id and
// login armed the gate on issues the snapshot job rejects --
// the same unsatisfiable shape this rule exists to prevent,
// one level down.
const unrestrictedRequested =
checked(['unrestricted scope', 'scope unrestricted']);
const labelSource = issue && issue.labels
? Array.isArray(issue.labels)
? issue.labels
: issue.labels.nodes || []
: [];
if (unrestrictedRequested &&
!carriesLabel(labelSource,
['scopeunrestrictedapproved'])) {
return false;
}
return declared(['agent run id', 'run id']) &&
declared(['agent login']);
declared(['agent login']) &&
declared(['objective', 'description']) &&
declared(['acceptance criteria', 'acceptance tests']) &&
(declared(['declared file scope', 'file scope', 'scope']) ||
unrestrictedRequested) &&
/-\s*\[[xX]\]/.test(
section(body, ['pre-dispatch confirmation'])
);
}
const login = String(
pull && pull.user && pull.user.login || ''
);
const knownAgents = new Set([
'google-labs-jules[bot]',
'github-copilot[bot]',
'copilot-swe-agent[bot]',
'openai-codex[bot]',
'chatgpt-codex-connector[bot]'
]);
const agentBranch =
/^(?:agent|claude|codex|copilot|jules)[/-]/i.test(
String(pull && pull.head && pull.head.ref || '')
);
const manifestPresent =
/<!--\s*agent-lock-manifest\s*[\s\S]*?-->/i.test(
String(pull && pull.body || '')
);
// Provenance asserted by the pull request itself. Each of these
// is a claim by the producing side that this is agent work.
const pullProvenance = knownAgents.has(login) || agentBranch ||
manifestPresent ||
carriesLabel((pull && pull.labels) || [],
['agent', 'agenttask', 'mcpagent']);
// Issue-side dispatch. Label automation also applies agent task
// labels as topic tags to issues that never declared a contract,
// so the bare label is not evidence of a dispatch: it only
Expand All @@ -2024,31 +2063,36 @@ jobs:
const issueDispatch =
carriesLabel(issueLabelSource, ['agenttask', 'mcpagent']) &&
declaresAgentContract(selectedIssue);
// Pull-side provenance says who produced the branch. It is not
// evidence that a dispatch contract exists to measure that
// branch against. The gate scores a pull request against the
// frozen intent snapshot on its linked issue, and that snapshot
// is only ever written by `snapshot-agent-task-intent`, which
// runs on `issues` events alone. With no linked issue there is
// no snapshot, no declared run id and no declared login, so
// `policy.agent_login`, `policy.run_id` and `issue.number` are
// all unsatisfiable and the verdict is permanently
// `invalid_payload` regardless of what the author does. A
// branch named `claude/...` is a naming convention, not a
// dispatch. Arming on it alone is what made this check red on
// pull requests that never had a contract to satisfy -- including
// #1368, which merged with this status failing.
// Only an issue-side dispatch arms the gate. Pull-side
// provenance says who produced the branch; it is not evidence
// that a dispatch contract exists to measure that branch
// against. The gate scores a pull request against the frozen
// intent snapshot on its linked issue, and that snapshot is only
// ever written by `snapshot-agent-task-intent`, which runs on
// `issues` events alone, and only for issues labelled
// `agent-task`/`mcp-agent` that already declare a run id and
// login. Without that snapshot `policy.agent_login` and
// `policy.run_id` are unsatisfiable and the verdict is
// permanently `invalid_payload` regardless of what the author
// does. A branch named `claude/...` is a naming convention, not
// a dispatch.
//
// Arming on `pullProvenance && selectedIssue` -- provenance plus
// *any* linked issue -- put this check in direct contradiction
// with `PR Governance`, which requires exactly one
// `Closes #<issue>` reference. Satisfying one guaranteed failing
// the other: every well-formed agent pull request was armed
// against a contract that had never been written, so the gate
// was red on ~100% of pull requests, including merged ones
// (#1368, #1408). Requiring a real dispatch instead restores the
// #1130 reasoning to the arming rule that overrode it.
//
// So provenance arms the gate only once a linked issue exists to
// verify against; with none, there is nothing to measure and the
// verdict is `not_applicable`. This does not create an escape
// hatch: a pull request that links a dispatched issue is still
// fully gated, and requiring a pull request to bind to a focused
// issue at all is separately owned by `Canonical issue and
// evidence`, which states a requirement an author can actually
// meet.
return login !== 'dependabot[bot]' &&
(issueDispatch || (pullProvenance && Boolean(selectedIssue)));
// This is not an escape hatch: a pull request that links a
// genuinely dispatched issue is still fully gated, and requiring
// a pull request to bind to a focused issue at all is separately
// owned by `Canonical issue and evidence`, which states a
// requirement an author can actually meet.
return login !== 'dependabot[bot]' && issueDispatch;
Comment on lines 2063 to +2095

@coderabbitai coderabbitai Bot Aug 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

The arming predicate is a strict subset of the snapshot contract, so a minimal issue still blocks forever. declaresAgentContract checks a run id and a login. snapshot-agent-task-intent additionally requires an objective, acceptance criteria, declared scope or approved unrestricted scope, and pre-dispatch confirmation. An issue that satisfies the first set but not the second arms the gate, gets no snapshot, and blocks permanently on missing_intent_snapshot. Define one shared contract predicate and reuse it everywhere.

  • .github/workflows/pr-checks.yml#L1987-L2019: replace the local declaresAgentContract with the shared full-contract predicate used by snapshot-agent-task-intent.
  • .github/workflows/pr-checks.yml#L677-L709: apply the identical replacement so the two agentTaskApplicable copies stay byte-identical.
  • docs/agent-completion-truth-gate.md#L38-L43: list the full contract that the arming rule requires, so the prose matches the corrected predicate.
📍 Affects 2 files
  • .github/workflows/pr-checks.yml#L1987-L2019 (this comment)
  • .github/workflows/pr-checks.yml#L677-L709
  • docs/agent-completion-truth-gate.md#L38-L43
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/pr-checks.yml around lines 1987 - 2019, Replace the local
declaresAgentContract arming check with one shared full-contract predicate
matching snapshot-agent-task-intent, including objective, acceptance criteria,
declared or approved unrestricted scope, pre-dispatch confirmation, run id, and
login. Apply the identical change to agentTaskApplicable at
.github/workflows/pr-checks.yml lines 677-709 and 1987-2019 so both copies
remain byte-identical; update docs/agent-completion-truth-gate.md lines 38-43 to
document the same full contract required by the arming rule.

}

const prNumber = Number(process.env.INPUT_PR_NUMBER || 0);
Expand Down
Loading
Loading