diff --git a/.github/agent-lock/trusted-publishers.json b/.github/agent-lock/trusted-publishers.json deleted file mode 100644 index 11e9b6dda..000000000 --- a/.github/agent-lock/trusted-publishers.json +++ /dev/null @@ -1,8 +0,0 @@ -{ - "schema_version": 1, - "trusted_check_app_slugs": [], - "trusted_label_actors": [], - "trusted_human_exemption_actors": [], - "custom_role_policy": "fail_closed", - "notes": "Populate all three allowlists only through a protected default-branch change after the independent GitHub App and its identity have been verified. An empty allowlist intentionally blocks rather than downgrading agent work to not_applicable." -} diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 920b1ad25..f4b1eb6a8 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -37,16 +37,3 @@ Provide the Vercel preview, production deployment, runtime evidence, or state wh - [ ] Required checks pass on the current head - [ ] Human decision is requested only for product, security, irreversible infrastructure, or production approval -## Agent provenance - -Human-authored pull requests may delete this section. Agent-authored pull requests must replace agent-lock-example with agent-lock-manifest and fill the values. Scope and test paths remain authoritative in the linked issue. - - - -The declared agent publishes a result comment on the linked issue or PR with the exact run ID and current 40-character head SHA. Replace `agent-lock-event-example` with `agent-lock-event` only when publishing real evidence. - - diff --git a/.github/workflows/AUDIT.md b/.github/workflows/AUDIT.md index 4b852c15d..1e470b474 100644 --- a/.github/workflows/AUDIT.md +++ b/.github/workflows/AUDIT.md @@ -28,7 +28,7 @@ concrete reason, verified against the actual repository tree. | `issue-triage.yml` | KEEP | Keyword auto-labeling + triage comment on new issues. | | `mcp-optimization.yml` | **DELETE** | Entire workflow targets `mcp-servers/mcp-profiling/` (requirements.txt, investigator_client.py, profiling_server.py) which does not exist — every run fails. | | `phase-goal-tracker.yml` | KEEP | Tracks markdown checklists on phase issues, keeps a single status comment updated, and auto-closes the issue when all checklist goals are complete. | -| `pr-checks.yml` | KEEP | Validates PR title/description; fork-safe comment handling. | +| `pr-checks.yml` | KEEP | Validates PR title/description. Truth-gate jobs removed; see `MERGE_POLICY.md`. | | `real-processing.yml` | KEEP | Manual single-video processing; well-formed. | | `secret-scan.yml` | KEEP | gitleaks on the working tree; action pinned to SHA, checksum-verified install. | | `security.yml` | KEEP | npm audit, safety, bandit, trivy; uploads SARIF. | @@ -65,9 +65,8 @@ valid. Referenced paths were checked against the working tree: ## Agent completion enforcement -| `agent-completion-enforcement.yml` | **ADD** | Protected-default-branch verifier that creates the independent **Agent completion enforcement** Check directly against the PR head SHA. It accepts only an exact-head machine-readable report from the configured dedicated GitHub App; stale/mutable evidence, untrusted label provenance, and custom roles all fail closed, and once the policy is provisioned a missing report fails closed too. The existing `agent-completion/truth-gate` status stays advisory and must not be made required. | -The protected policy at `.github/agent-lock/trusted-publishers.json` starts with empty allowlists. While those allowlists are empty the Check reports **neutral (advisory)** rather than blocking, so it does not train reviewers to ignore a permanently red gate; it becomes blocking once a repository administrator provisions the dedicated App and trusted actor identities through protected review. The repository ruleset must then require **Agent completion enforcement**, one independent approval, and resolved conversations. +The agent-lock trust policy and both agent-completion Checks were removed as unsatisfiable; `pr-governance.yml` is now the sole binding gate. See `MERGE_POLICY.md`. ## Repository governance workflows diff --git a/.github/workflows/README.md b/.github/workflows/README.md index 37aaa7354..cd82c2b60 100644 --- a/.github/workflows/README.md +++ b/.github/workflows/README.md @@ -170,12 +170,18 @@ A full audit of this directory was performed (see - [pytest-cov Documentation](https://pytest-cov.readthedocs.io/) -| Agent completion enforcement | `agent-completion-enforcement.yml` | `pull_request_target`; manual | Creates the independent, head-bound `Agent completion enforcement` Check from protected default-branch code. | | PR Governance | `pr-governance.yml` | `pull_request_target` (opened/edited/reopened/synchronize/ready_for_review) | Validates that every ready PR links exactly one real open canonical issue and contains non-empty delivery evidence sections; fails on competing PRs. | | Repository Reconciliation | `repository-reconciliation.yml` | daily (13:17 UTC); manual | Non-destructive daily report of ready PRs missing a canonical issue, issues with competing implementation PRs, and stale unattached branches. | ## Agent-completion enforcement -`pr-checks.yml` retains the advisory `agent-completion/truth-gate/pr-` status; it is never required. `agent-completion-enforcement.yml` runs protected default-branch code, does not execute PR code, and creates the separate **Agent completion enforcement** Check directly on the PR head SHA. It accepts only an exact-head, machine-readable report published by the configured dedicated GitHub App. Stale, edited/deleted, ambiguous, or untrusted evidence fails closed. While the trust policy is unprovisioned (empty allowlists) the Check reports **neutral (advisory)** instead of red so its signal is not lost to constant noise; once the policy is provisioned, a missing report also fails closed. +Removed. The `agent-completion/truth-gate` status and the `Agent completion +enforcement` Check were retired because they were unsatisfiable: the gate scored a +pull request against an intent snapshot written only on `issues` events, so any +pull request that satisfied `PR Governance` (which requires `Closes #`) +necessarily armed the gate and then failed it. It was red on ~100% of pull +requests, including merged ones such as #1368. -Before enabling the rule, provision `.github/agent-lock/trusted-publishers.json` through protected review with the trusted App and actor allowlists. Until then the Check is **neutral (advisory)** and blocks nothing; populating the allowlists (and adding the Check to required status checks) is what makes it blocking. Configure the repository ruleset to require **Agent completion enforcement**, one independent approval, and resolved conversations. Do not require `agent-completion/truth-gate`. \ No newline at end of file +Binding a pull request to one focused issue is now owned solely by +`pr-governance.yml`, which produces the `PR Governance` and `Canonical issue and +evidence` Checks. See `MERGE_POLICY.md` at the repository root. diff --git a/.github/workflows/agent-completion-enforcement.yml b/.github/workflows/agent-completion-enforcement.yml deleted file mode 100644 index f869f3a85..000000000 --- a/.github/workflows/agent-completion-enforcement.yml +++ /dev/null @@ -1,161 +0,0 @@ -name: Agent completion enforcement - -on: - pull_request_target: - types: [opened, reopened, synchronize, ready_for_review, labeled, unlabeled, edited] - workflow_dispatch: - inputs: - pull_request: - required: true - type: number - -permissions: {} - -jobs: - enforce: - name: Agent completion enforcement - runs-on: ubuntu-latest - timeout-minutes: 10 - permissions: - checks: write - contents: read - pull-requests: read - steps: - - name: Check out only the protected default-branch verifier - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - with: - ref: ${{ github.event.repository.default_branch }} - persist-credentials: false - - name: Require a trusted, append-only publication - id: verify - continue-on-error: true - env: - GH_TOKEN: ${{ github.token }} - PR: ${{ inputs.pull_request || github.event.pull_request.number }} - REPO: ${{ github.repository }} - run: | - set -euo pipefail - test -n "$PR" - head="$(gh api "repos/$REPO/pulls/$PR" --jq .head.sha)" - # The trusted GitHub App must publish this immutable report as a Check - # output/artifact. This workflow never executes PR-controlled code or - # accepts an artifact produced by a pull_request workflow. - gh api "repos/$REPO/commits/$head/check-runs" --jq '.check_runs[] | select(.name == "Agent Lock trusted publication") | .output.text' > trusted-report.json - if test -s trusted-report.json; then - python3 scripts/ci/agent_completion_enforcement.py trusted-report.json .github/agent-lock/trusted-publishers.json "$head" "$PR" > enforcement-verdict.json - else - printf '%s\n' '{"conclusion":"failure","reason":"missing_trusted_publication","details":{}}' > enforcement-verdict.json - fi - - name: Publish the required head-bound Check run - if: always() - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - env: - PR: ${{ inputs.pull_request || github.event.pull_request.number }} - with: - script: | - const fs = require('fs'); - const pull = await github.rest.pulls.get({ - owner: context.repo.owner, - repo: context.repo.repo, - pull_number: Number(process.env.PR) - }); - let verdict = { - conclusion: 'failure', - reason: 'verifier_did_not_publish', - details: {} - }; - try { - verdict = JSON.parse(fs.readFileSync( - 'enforcement-verdict.json', 'utf8' - )); - } catch (error) { - core.warning(error.message); - } - // Some verdict reasons mean "the Agent Lock trust infrastructure has - // not been stood up yet", not "this pull request violated a policy". - // Reporting those as `failure` makes this check permanently red on - // 100% of pull requests, which destroys its signal value and hides - // real build/test failures behind constant noise. Those are reported - // as `neutral` instead. - // - // This does NOT weaken enforcement: - // * a genuine policy violation still reports `failure`; - // * this check is not currently a required status check on `main`, - // so a red result blocks no merge today in any case. - // Once a trusted publisher App exists and - // .github/agent-lock/trusted-publishers.json is populated, these - // reasons stop occurring and the gate becomes live. At that point it - // should be added to the branch protection required checks. - // - // Whether the trust policy has actually been provisioned. This mirrors - // the emptiness gate in scripts/ci/agent_completion_enforcement.py - // (which returns `trust_policy_unprovisioned` when *any* allowlist is - // empty): the policy is provisioned only when all three allowlists are - // non-empty. The workflow checks out the protected default branch, so - // this reads the trusted, not the PR-controlled, policy file. - let trustPolicyProvisioned = false; - try { - const policy = JSON.parse(fs.readFileSync( - '.github/agent-lock/trusted-publishers.json', 'utf8' - )); - trustPolicyProvisioned = - Array.isArray(policy.trusted_check_app_slugs) && - policy.trusted_check_app_slugs.length > 0 && - Array.isArray(policy.trusted_label_actors) && - policy.trusted_label_actors.length > 0 && - Array.isArray(policy.trusted_human_exemption_actors) && - policy.trusted_human_exemption_actors.length > 0; - } catch (error) { - // If the policy file cannot be read it cannot be provisioned; treat - // as unprovisioned. A genuine violation requires a readable, populated - // policy, so this only affects the infrastructure-state reasons below. - core.warning('Could not read trusted-publishers.json: ' + error.message); - } - // `trust_policy_unprovisioned` is derived from the policy file's own - // empty-allowlist state, so it always means the gate is inactive and is - // always advisory (neutral). - // - // `missing_trusted_publication` means no trusted report was published. - // While the policy is unprovisioned that is expected (no publisher App - // exists yet) and is advisory. But once the policy IS provisioned, a - // missing report is a real failure mode -- an App outage, a publication - // race, or a check-run that never landed -- and must stay fail-closed - // rather than passing a PR with no trusted evidence. - const isAdvisoryReason = - verdict.reason === 'trust_policy_unprovisioned' || - (verdict.reason === 'missing_trusted_publication' && !trustPolicyProvisioned); - const conclusion = verdict.conclusion === 'success' - ? 'success' - : isAdvisoryReason - ? 'neutral' - : 'failure'; - const title = conclusion === 'success' - ? 'Trusted evidence verified' - : conclusion === 'neutral' - ? 'Agent Lock not provisioned - gate inactive' - : 'Trusted evidence blocked'; - const summary = JSON.stringify(verdict); - await github.rest.checks.create({ - owner: context.repo.owner, - repo: context.repo.repo, - name: 'Agent completion enforcement', - head_sha: pull.data.head.sha, - status: 'completed', - conclusion, - output: { - title, - summary: summary.slice(0, 60000) - } - }); - if (conclusion === 'failure') { - core.setFailed(verdict.reason || 'trusted evidence blocked'); - } - if (conclusion === 'neutral') { - core.warning( - 'Agent Lock trust policy is not provisioned (reason: ' + - verdict.reason + '). This gate is inactive and is not ' + - 'enforcing anything. Populate ' + - '.github/agent-lock/trusted-publishers.json and stand up the ' + - 'publishing App to activate it.' - ); - } diff --git a/.github/workflows/pr-checks.yml b/.github/workflows/pr-checks.yml index 8f5dec4fe..b9e6a5ec1 100644 --- a/.github/workflows/pr-checks.yml +++ b/.github/workflows/pr-checks.yml @@ -2,1717 +2,12 @@ name: PR Checks on: pull_request_target: - types: [opened, reopened, synchronize, edited, ready_for_review, converted_to_draft, labeled, unlabeled, closed] - issue_comment: - types: [created, edited, deleted] - issues: - types: [opened, edited, labeled, unlabeled, closed, reopened] - workflow_run: - workflows: ["CI"] - types: [completed] - schedule: - - cron: "*/15 * * * *" - workflow_dispatch: - inputs: - pull_request: - description: Pull request number to re-evaluate - required: true - type: number + types: [opened, reopened, synchronize, edited, ready_for_review] permissions: {} jobs: - snapshot-agent-task-intent: - if: >- - github.event_name == 'issues' && - (github.event.action == 'labeled' || - github.event.action == 'edited' || - github.event.action == 'unlabeled') - concurrency: - group: agent-intent-snapshot-${{ github.event.issue.number }} - cancel-in-progress: false - runs-on: ubuntu-latest - permissions: - issues: write - pull-requests: write - steps: - - name: Freeze the delegated issue body - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - with: - script: | - const crypto = require('crypto'); - const issue = context.payload.issue; - const workflowRunId = String(context.runId || ''); - if (!/^[1-9]\d*$/.test(workflowRunId)) { - core.setFailed('invalid_workflow_run_id'); - return; - } - const labels = new Set((issue.labels || []).map(label => - String(typeof label === 'string' ? label : label.name) - .toLowerCase().replace(/[^a-z0-9 ]/g, '').trim() - )); - const hasAgentTaskLabel = [...labels].some(label => - ['agenttask', 'mcpagent'].includes(label) - ); - const eventLabel = String( - context.payload.label && context.payload.label.name || '' - ).toLowerCase().replace(/[^a-z0-9 ]/g, '').trim(); - const snapshotMarker = - '/i - ); - if (match) { - const payload = JSON.parse(match[1].trim()); - snapshotRunId = String(payload.workflow_run_id || ''); - } - } catch (error) { - core.warning( - 'Could not read snapshot source run: ' + error.message - ); - } - } - const disposition = snapshotEventDisposition( - Boolean(existing), - context.payload.action, - eventLabel, - snapshotRunId, - workflowRunId - ); - if (disposition === 'same_snapshot_run') { - core.info('Snapshot already created by this workflow run'); - return; - } - if (disposition === 'invalidate') { - const invalidation = { - issue_number: issue.number, - event_action: context.payload.action, - actor: context.actor, - workflow_run_id: workflowRunId, - created_at: new Date().toISOString() - }; - await github.rest.issues.createComment({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: issue.number, - body: invalidationMarker + '\n' + - JSON.stringify(invalidation) + '\n-->' + - '\n\nFrozen agent-task intent was invalidated; create a new task.' - }); - return; - } - if (disposition !== 'snapshot' || !hasAgentTaskLabel) { - return; - } - function hasTrustedSnapshotPermission(permission, roleName) { - const trustedPermissions = new Set([ - 'admin', 'maintain', 'write', 'triage' - ]); - return trustedPermissions.has(permission) || - trustedPermissions.has(roleName); - } - let permission = ''; - let roleName = ''; - try { - const result = - await github.rest.repos.getCollaboratorPermissionLevel({ - owner: context.repo.owner, - repo: context.repo.repo, - username: context.actor - }); - permission = result.data.permission; - roleName = result.data.role_name; - } catch (error) { - core.warning( - 'Could not verify snapshot actor permission: ' + - error.message - ); - return; - } - if (!hasTrustedSnapshotPermission(permission, roleName)) { - core.info('Ignoring an untrusted agent-task snapshot'); - return; - } - function normaliseHeading(value) { - return value.toLowerCase().replace(/[^a-z0-9 ]/g, '').trim(); - } - function section(body, headings) { - const wanted = new Set(headings.map(normaliseHeading)); - const lines = String(body || '').split(/\r?\n/); - const output = []; - let collecting = false; - for (const line of lines) { - const heading = line.match(/^#{2,6}\s+(.+?)\s*$/); - if (heading) { - if (collecting) { - break; - } - collecting = wanted.has(normaliseHeading(heading[1])); - continue; - } - if (collecting) { - output.push(line); - } - } - return output.join('\n').trim(); - } - function hasResponse(value) { - const response = String(value || '').trim(); - return Boolean(response && response !== '_No response_'); - } - function checkboxChecked(value) { - const response = String(value || '').trim(); - return response.split(/\r?\n/).some(line => - /^\s*[-*]\s*\[[xX]\]/.test(line) - ) || /^(?:yes|true)$/i.test(response); - } - const issueBody = String(issue.body || ''); - const unrestricted = section( - issueBody, - ['unrestricted scope', 'scope unrestricted'] - ); - const unrestrictedRequested = checkboxChecked(unrestricted); - const declaredScope = section( - issueBody, - ['declared file scope', 'file scope', 'scope'] - ); - const confirmed = /-\s*\[[xX]\]/.test(section( - issueBody, - ['pre-dispatch confirmation'] - )); - const complete = [ - section(issueBody, ['agent login']), - section(issueBody, ['agent run id', 'run id']), - section(issueBody, ['objective', 'description']), - section(issueBody, ['acceptance criteria', 'acceptance tests']) - ].every(hasResponse) && - Boolean(hasResponse(declaredScope) || unrestrictedRequested) && - confirmed; - const unrestrictedApproved = - labels.has('scopeunrestrictedapproved'); - if (!complete || - (unrestrictedRequested && !unrestrictedApproved)) { - core.setFailed('incomplete_agent_task_contract'); - return; - } - const normalisedBody = issueBody - .replace(/\r\n/g, '\n').trim(); - const bodySha256 = crypto.createHash('sha256') - .update(normalisedBody, 'utf8').digest('hex'); - const snapshot = { - issue_number: issue.number, - body_sha256: bodySha256, - scope_unrestricted_approved: unrestrictedApproved, - workflow_run_id: workflowRunId, - created_at: new Date().toISOString() - }; - await github.rest.issues.createComment({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: issue.number, - body: snapshotMarker + '\n' + JSON.stringify(snapshot) + '\n-->' + - '\n\nAgent task intent frozen for completion-gate evaluation.' - }); - - refresh-open-pull-requests: - if: github.event_name == 'schedule' - concurrency: - group: agent-completion-scheduled-scan - cancel-in-progress: false - runs-on: ubuntu-latest - timeout-minutes: 14 - permissions: - actions: write - contents: read - issues: read - pull-requests: read - statuses: write - steps: - - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - with: - script: | - const crypto = require('crypto'); - const owner = context.repo.owner; - const repo = context.repo.repo; - const runTargetPrefix = context.serverUrl + '/' + owner + '/' + - repo + '/actions/runs/'; - function gateStatusContext(pullNumber) { - return 'agent-completion/truth-gate/pr-' + pullNumber; - } - function scheduledRefreshDecision(evidence) { - const statusCapacityFloor = 998; - if (!evidence.apiComplete) { - return 'retry'; - } - if (!evidence.latestState) { - return 'dispatch'; - } - const latestTime = Date.parse(evidence.latestCreatedAt); - const pendingTime = Date.parse(evidence.ownerPendingAt); - if (evidence.latestState === 'pending') { - if (!Number.isFinite(latestTime) || - !Number.isFinite(pendingTime) || - !evidence.pendingLeaseValid) { - return evidence.statusCount >= statusCapacityFloor - ? 'at_capacity' - : 'dispatch'; - } - if (['completed', 'missing'].includes( - evidence.ownerRunState - )) { - return evidence.statusCount < 1000 - ? 'finalize_pending' - : 'at_capacity'; - } - if (!['queued', 'in_progress'].includes( - evidence.ownerRunState - )) { - return 'retry'; - } - const now = Date.parse(evidence.now); - if (Number.isFinite(now) && - now - latestTime > 60 * 60 * 1000) { - return evidence.statusCount < 1000 - ? 'finalize_pending' - : 'at_capacity'; - } - return 'skip'; - } - if (!['success', 'failure', 'error'].includes( - evidence.latestState - ) || !Number.isFinite(latestTime) || - !Number.isFinite(pendingTime)) { - return evidence.statusCount >= statusCapacityFloor - ? 'at_capacity' - : 'dispatch'; - } - function isAfter(candidate, baseline) { - const candidateTime = Date.parse(candidate); - return Number.isFinite(candidateTime) && - candidateTime > baseline; - } - const evidenceChanged = - evidence.reviewProjectionChanged || - evidence.intentProjectionChanged || - evidence.policyProjectionChanged || - isAfter(evidence.commentUpdatedAt, pendingTime) || - isAfter(evidence.ciUpdatedAt, pendingTime); - if (evidenceChanged) { - if (evidence.statusCount < statusCapacityFloor) { - return 'dispatch'; - } - if (evidence.latestState !== 'success' || - evidence.terminalAlreadyInvalidated || - evidence.statusCount >= 1000) { - return 'at_capacity'; - } - return 'invalidate_terminal'; - } - if (evidence.statusCount >= statusCapacityFloor) { - return 'at_capacity'; - } - return 'skip'; - } - function latestIso(values) { - const times = values - .map(value => Date.parse(value)) - .filter(Number.isFinite); - return times.length > 0 - ? new Date(Math.max(...times)).toISOString() - : null; - } - function workflowRunId(targetUrl) { - const target = String(targetUrl || ''); - if (!target.startsWith(runTargetPrefix)) { - return null; - } - const suffix = target.slice(runTargetPrefix.length); - return /^\d+$/.test(suffix) ? suffix : null; - } - function terminalOwnerId(status) { - const match = String(status.description || '').match( - /^gate-owner:(\d+)(?:\s|$)/ - ); - return match ? match[1] : null; - } - function verdictProjection(comments) { - const marker = ''; - const comment = comments.find(candidate => - candidate.user && - candidate.user.login === 'github-actions[bot]' && - String(candidate.body || '').includes(marker) - ); - const encoded = String((comment && comment.body) || '').match( - /
([\s\S]*?)<\/pre>/
-              );
-              if (!encoded) {
-                return null;
-              }
-              try {
-                const verdict = JSON.parse(encoded[1]
-                  .replace(/</g, '<')
-                  .replace(/>/g, '>')
-                  .replace(/&/g, '&'));
-                if (!verdict || !Array.isArray(verdict.reasons) ||
-                    !verdict.details ||
-                    typeof verdict.details !== 'object') {
-                  return null;
-                }
-                const unknownProjectionReasons = new Set([
-                  'invalid_payload',
-                  'invalid_json',
-                  'input_read_failed',
-                  'verdict_artifact_missing',
-                  'evidence_collection_failed',
-                  'evidence_collection_step_failed',
-                  'gate_evaluation_step_failed',
-                  'evidence_artifact_failed',
-                  'gate_exit_code_mismatch'
-                ]);
-                const unusableProjectionReasons = new Set([
-                  'invalid_payload',
-                  'invalid_json',
-                  'input_read_failed',
-                  'verdict_artifact_missing',
-                  'evidence_collection_step_failed',
-                  'gate_evaluation_step_failed',
-                  'evidence_artifact_failed',
-                  'gate_exit_code_mismatch'
-                ]);
-                const collectionErrors = Array.isArray(
-                  verdict.details.collection_errors
-                )
-                  ? [...new Set(
-                    verdict.details.collection_errors
-                  )].sort()
-                  : [];
-                const mutablePolicyErrorNames = new Set([
-                  'draft_pr',
-                  'invalid_pr_title',
-                  'missing_copilot_rabbit_label',
-                  'invalid_agent_lock_manifest',
-                  'missing_agent_run_id',
-                  'agent_run_id_mismatch',
-                  'missing_agent_login',
-                  'agent_login_mismatch'
-                ]);
-                const rawIdentity = verdict.details.identity_projection;
-                const identityProjection = rawIdentity &&
-                  typeof rawIdentity === 'object'
-                    ? {
-                      issue_number:
-                        Number.isSafeInteger(rawIdentity.issue_number) &&
-                        rawIdentity.issue_number > 0
-                          ? rawIdentity.issue_number
-                          : null,
-                      agent_login:
-                        typeof rawIdentity.agent_login === 'string' &&
-                        rawIdentity.agent_login.trim()
-                          ? rawIdentity.agent_login.trim()
-                          : null,
-                      run_id:
-                        typeof rawIdentity.run_id === 'string' &&
-                        rawIdentity.run_id.trim()
-                          ? rawIdentity.run_id.trim()
-                          : null
-                    }
-                    : null;
-                return {
-                  notApplicable: verdict.verdict === 'not_applicable',
-                  projectionKnown:
-                    verdict.verdict === 'not_applicable' ||
-                    !verdict.reasons.some(reason =>
-                      unknownProjectionReasons.has(reason)
-                    ),
-                  projectionUsable:
-                    verdict.verdict === 'not_applicable' || (
-                      !verdict.reasons.some(reason =>
-                        unusableProjectionReasons.has(reason)
-                      ) && (
-                        !verdict.reasons.includes(
-                          'evidence_collection_failed'
-                        ) || Array.isArray(
-                          verdict.details.collection_errors
-                        )
-                      )
-                    ),
-                  copilotCurrentHeadReviewed: !verdict.reasons.includes(
-                    'missing_copilot_current_head_review'
-                  ),
-                  unresolved: Array.isArray(
-                    verdict.details.unresolved_reviews
-                  )
-                    ? [...new Set(
-                      verdict.details.unresolved_reviews
-                    )].sort()
-                    : [],
-                  collectionErrors,
-                  identityProjection,
-                  mutablePolicyErrors: [
-                    ...verdict.reasons,
-                    ...collectionErrors
-                  ].filter(error =>
-                    mutablePolicyErrorNames.has(error)
-                  ).filter((error, index, errors) =>
-                    errors.indexOf(error) === index
-                  ).sort()
-                };
-              } catch (error) {
-                return null;
-              }
-            }
-            function normaliseScheduledBotLogin(login) {
-              return String(login || '').toLowerCase()
-                .replace(/\[bot\]$/, '');
-            }
-            function scheduledCommentAffectsEvidence(
-              comment,
-              expectedAgents
-            ) {
-              const actor = comment && comment.user && comment.user.login;
-              if (!actor || !expectedAgents.has(actor)) {
-                return false;
-              }
-              const body = String(comment.body || '');
-              return //i
-              );
-              if (manifest) {
-                try {
-                  return String(
-                    JSON.parse(manifest[1].trim()).agent_login || ''
-                  ).trim();
-                } catch (error) {
-                  return '';
-                }
-              }
-              const lines = value.split(/\r?\n/);
-              const output = [];
-              let collecting = false;
-              for (const line of lines) {
-                const heading = line.match(/^#{2,6}\s+(.+?)\s*$/);
-                if (heading) {
-                  if (collecting) {
-                    break;
-                  }
-                  collecting = heading[1].toLowerCase()
-                    .replace(/[^a-z0-9 ]/g, '').trim() === 'agent login';
-                  continue;
-                }
-                if (collecting) {
-                  output.push(line);
-                }
-              }
-              return output.join('\n').trim()
-                .replace(/^\x60|\x60$/g, '');
-            }
-            function scheduledLinkProjection(pull, linkedIssues) {
-              const body = String(pull && pull.body || '');
-              let manifest = {};
-              const errors = [];
-              const manifestMatch = body.match(
-                //i
-              );
-              if (manifestMatch) {
-                try {
-                  manifest = JSON.parse(manifestMatch[1].trim());
-                } catch (error) {
-                  manifest = {};
-                  errors.push('invalid_agent_lock_manifest');
-                }
-              }
-              const manifestNumber =
-                Number.isSafeInteger(manifest.issue_number) &&
-                manifest.issue_number > 0
-                  ? manifest.issue_number
-                  : 0;
-              const referenceNumbers = [...new Set(
-                (linkedIssues || []).map(issue => issue.number)
-                  .filter(number =>
-                    Number.isSafeInteger(number) && number > 0
-                  )
-              )];
-              const authoritativeNumber = referenceNumbers.length === 1
-                ? referenceNumbers[0]
-                : 0;
-              const textualMatch = body.match(
-                /(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/i
-              );
-              const textualNumber = textualMatch
-                ? Number(textualMatch[1])
-                : 0;
-              if (referenceNumbers.length !== 1) {
-                errors.push('missing_closing_issue_reference');
-              }
-              if (referenceNumbers.length > 1) {
-                errors.push('multiple_closing_issues');
-              }
-              const contractValues = [
-                manifest.issue_number,
-                textualNumber,
-                authoritativeNumber
-              ];
-              if (!contractValues.every(value =>
-                Number.isSafeInteger(value) && value > 0
-              )) {
-                errors.push('incomplete_linked_issue_contract');
-              } else if (new Set(contractValues).size !== 1) {
-                errors.push('conflicting_linked_issue');
-              }
-              return {
-                selectedIssueNumber:
-                  manifestNumber || authoritativeNumber || textualNumber,
-                closingIssueCount: referenceNumbers.length,
-                errors: [...new Set(errors)].sort()
-              };
-            }
-            function agentTaskApplicable(pull, selectedIssue) {
-              function normaliseLabel(value) {
-                return String(value || '').toLowerCase()
-                  .replace(/[^a-z0-9 ]/g, '').trim();
-              }
-              function carriesLabel(source, accepted) {
-                return (source || []).map(label =>
-                  typeof label === 'string' ? label : label.name
-                ).map(normaliseLabel).some(label =>
-                  accepted.includes(label)
-                );
-              }
-              function declaresAgentContract(issue) {
-                function section(body, headings) {
-                  const wanted = new Set(headings.map(normaliseLabel));
-                  const lines = String(body || '').split(/\r?\n/);
-                  const output = [];
-                  let collecting = false;
-                  for (const line of lines) {
-                    const heading = line.match(/^#{2,6}\s+(.+?)\s*$/);
-                    if (heading) {
-                      if (collecting) {
-                        break;
-                      }
-                      collecting = wanted.has(normaliseLabel(heading[1]));
-                      continue;
-                    }
-                    if (collecting) {
-                      output.push(line);
-                    }
-                  }
-                  return output.join('\n').trim();
-                }
-                function declared(headings) {
-                  const value = section(
-                    String((issue && issue.body) || ''), headings
-                  ).replace(/^\x60|\x60$/g, '').trim();
-                  return Boolean(value) && value !== '_No response_';
-                }
-                return declared(['agent run id', 'run id']) &&
-                  declared(['agent login']);
-              }
-              const login = String(
-                pull && pull.user && pull.user.login || ''
-              );
-              const knownAgents = new Set([
-                'google-labs-jules[bot]',
-                'github-copilot[bot]',
-                'copilot-swe-agent[bot]',
-                'openai-codex[bot]',
-                'chatgpt-codex-connector[bot]'
-              ]);
-              const agentBranch =
-                /^(?:agent|claude|codex|copilot|jules)[/-]/i.test(
-                  String(pull && pull.head && pull.head.ref || '')
-                );
-              const manifestPresent =
-                //i.test(
-                  String(pull && pull.body || '')
-                );
-              // Provenance asserted by the pull request itself. Each of these
-              // is a claim by the producing side that this is agent work.
-              const pullProvenance = knownAgents.has(login) || agentBranch ||
-                manifestPresent ||
-                carriesLabel((pull && pull.labels) || [],
-                  ['agent', 'agenttask', 'mcpagent']);
-              // Issue-side dispatch. Label automation also applies agent task
-              // labels as topic tags to issues that never declared a contract,
-              // so the bare label is not evidence of a dispatch: it only
-              // counts when the issue actually declares the run id and login
-              // the gate goes on to require. Treating the bare label as
-              // sufficient made the gate permanently unsatisfiable for human
-              // pull requests closing such issues, because the contract they
-              // were measured against had never been written (#1130). Only the
-              // two contract labels count here -- the generic `agent` label is
-              // never recognised by the snapshot job or the collector, so it
-              // stays a pull-request-side provenance signal only.
-              const issueLabelSource = selectedIssue && selectedIssue.labels
-                ? Array.isArray(selectedIssue.labels)
-                  ? selectedIssue.labels
-                  : selectedIssue.labels.nodes || []
-                : [];
-              const issueDispatch =
-                carriesLabel(issueLabelSource, ['agenttask', 'mcpagent']) &&
-                declaresAgentContract(selectedIssue);
-              // Pull-side provenance says who produced the branch. It is not
-              // evidence that a dispatch contract exists to measure that
-              // branch against. The gate scores a pull request against the
-              // frozen intent snapshot on its linked issue, and that snapshot
-              // is only ever written by `snapshot-agent-task-intent`, which
-              // runs on `issues` events alone. With no linked issue there is
-              // no snapshot, no declared run id and no declared login, so
-              // `policy.agent_login`, `policy.run_id` and `issue.number` are
-              // all unsatisfiable and the verdict is permanently
-              // `invalid_payload` regardless of what the author does. A
-              // branch named `claude/...` is a naming convention, not a
-              // dispatch. Arming on it alone is what made this check red on
-              // pull requests that never had a contract to satisfy -- including
-              // #1368, which merged with this status failing.
-              //
-              // So provenance arms the gate only once a linked issue exists to
-              // verify against; with none, there is nothing to measure and the
-              // verdict is `not_applicable`. This does not create an escape
-              // hatch: a pull request that links a dispatched issue is still
-              // fully gated, and requiring a pull request to bind to a focused
-              // issue at all is separately owned by `Canonical issue and
-              // evidence`, which states a requirement an author can actually
-              // meet.
-              return login !== 'dependabot[bot]' &&
-                (issueDispatch || (pullProvenance && Boolean(selectedIssue)));
-            }
-            function intentContractErrors(issue, comments, pullCreatedAt) {
-              const errors = [];
-              const labels = new Set(
-                ((issue.labels && issue.labels.nodes) || []).map(label =>
-                  String(label.name || '').toLowerCase()
-                    .replace(/[^a-z0-9 ]/g, '').trim()
-                )
-              );
-              if (![...labels].some(label =>
-                ['agenttask', 'mcpagent'].includes(label)
-              )) {
-                errors.push('linked_issue_not_agent_task');
-              }
-              const snapshotComment = comments.find(comment =>
-                comment.user &&
-                comment.user.login === 'github-actions[bot]' &&
-                //i
-              );
-              try {
-                if (!snapshotMatch) {
-                  throw new Error('missing snapshot payload');
-                }
-                const snapshot = JSON.parse(snapshotMatch[1].trim());
-                const currentHash = crypto.createHash('sha256')
-                  .update(String(issue.body || '')
-                    .replace(/\r\n/g, '\n').trim(), 'utf8')
-                  .digest('hex');
-                if (snapshot.issue_number !== issue.number ||
-                    !/^[a-f0-9]{64}$/.test(String(
-                      snapshot.body_sha256 || ''
-                    ))) {
-                  errors.push('invalid_intent_snapshot');
-                } else if (snapshot.body_sha256 !== currentHash) {
-                  errors.push('intent_changed_after_dispatch');
-                }
-                const snapshotTime = Date.parse(snapshotComment.created_at);
-                const pullTime = Date.parse(pullCreatedAt);
-                if (!Number.isFinite(snapshotTime) ||
-                    !Number.isFinite(pullTime) ||
-                    snapshotTime >= pullTime) {
-                  errors.push('intent_snapshot_after_dispatch');
-                }
-                if (Boolean(snapshot.scope_unrestricted_approved) !==
-                    labels.has('scopeunrestrictedapproved')) {
-                  errors.push(
-                    'unrestricted_approval_changed_after_dispatch'
-                  );
-                }
-                if (intentInvalidated) {
-                  errors.push('intent_changed_after_dispatch');
-                }
-              } catch (error) {
-                errors.push('invalid_intent_snapshot');
-              }
-              return [...new Set(errors)].sort();
-            }
-            function scheduledIntentProjectionChanged(
-              previousProjection,
-              currentErrors,
-              closingIssueCount,
-              currentApplicable
-            ) {
-              if (!previousProjection) {
-                return false;
-              }
-              if (previousProjection.projectionUsable === false) {
-                return false;
-              }
-              if (previousProjection.notApplicable) {
-                return currentApplicable;
-              }
-              if (!currentApplicable) {
-                return true;
-              }
-              const relevantErrors = new Set([
-                'invalid_agent_lock_manifest',
-                'missing_closing_issue_reference',
-                'multiple_closing_issues',
-                'incomplete_linked_issue_contract',
-                'conflicting_linked_issue',
-                'linked_issue_unavailable',
-                'missing_linked_issue'
-              ]);
-              if (closingIssueCount === 1) {
-                for (const error of [
-                  'missing_intent_snapshot',
-                  'invalid_intent_snapshot',
-                  'intent_changed_after_dispatch',
-                  'intent_snapshot_after_dispatch',
-                  'unrestricted_approval_changed_after_dispatch',
-                  'linked_issue_not_agent_task'
-                ]) {
-                  relevantErrors.add(error);
-                }
-              }
-              const previousErrors = previousProjection.collectionErrors
-                .filter(error => relevantErrors.has(error)).sort();
-              const projectedCurrent = currentErrors
-                .filter(error => relevantErrors.has(error)).sort();
-              return JSON.stringify(projectedCurrent) !==
-                JSON.stringify(previousErrors);
-            }
-            function scheduledMutablePolicyErrors(
-              pull,
-              selectedIssue,
-              currentApplicable
-            ) {
-              if (!currentApplicable) {
-                return [];
-              }
-              function normaliseHeading(value) {
-                return String(value || '').toLowerCase()
-                  .replace(/[^a-z0-9 ]/g, '').trim();
-              }
-              function section(body, headings) {
-                const wanted = new Set(headings.map(normaliseHeading));
-                const lines = String(body || '').split(/\r?\n/);
-                const output = [];
-                let collecting = false;
-                for (const line of lines) {
-                  const heading = line.match(/^#{2,6}\s+(.+?)\s*$/);
-                  if (heading) {
-                    if (collecting) {
-                      break;
-                    }
-                    collecting = wanted.has(normaliseHeading(heading[1]));
-                    continue;
-                  }
-                  if (collecting) {
-                    output.push(line);
-                  }
-                }
-                return output.join('\n').trim();
-              }
-              const errors = [];
-              const title = String(pull && pull.title || '');
-              if (pull && pull.draft === true) {
-                errors.push('draft_pr');
-              }
-              if (title.length < 10 ||
-                  !/^(?:⚡\s*)?(?:feat|fix|docs|style|refactor|test|chore|perf|ci|build|revert)(?:\(.+\))?:/i.test(title)) {
-                errors.push('invalid_pr_title');
-              }
-              const labels = (pull && pull.labels) || [];
-              if (!labels.some(label =>
-                String(typeof label === 'string' ? label : label.name)
-                  .toLowerCase() === 'copilot-rabbit'
-              )) {
-                errors.push('missing_copilot_rabbit_label');
-              }
-              const body = String(pull && pull.body || '');
-              const manifestMatch = body.match(
-                //i
-              );
-              let manifest = {};
-              if (manifestMatch) {
-                try {
-                  manifest = JSON.parse(manifestMatch[1]);
-                } catch (error) {
-                  errors.push('invalid_agent_lock_manifest');
-                }
-              }
-              const issueBody = String(
-                selectedIssue && selectedIssue.body || ''
-              );
-              const expectedRunId = section(
-                issueBody,
-                ['agent run id', 'run id']
-              ).replace(/^\x60|\x60$/g, '').trim();
-              const expectedAgentLogin = section(
-                issueBody,
-                ['agent login']
-              ).replace(/^\x60|\x60$/g, '').trim();
-              if (!expectedRunId) {
-                errors.push('missing_agent_run_id');
-              } else if (String(manifest.run_id || '') !== expectedRunId) {
-                errors.push('agent_run_id_mismatch');
-              }
-              if (!expectedAgentLogin) {
-                errors.push('missing_agent_login');
-              } else if (String(manifest.agent_login || '') !==
-                  expectedAgentLogin) {
-                errors.push('agent_login_mismatch');
-              }
-              return [...new Set(errors)].sort();
-            }
-            function scheduledContractIdentity(
-              selectedIssueNumber,
-              selectedIssue,
-              currentApplicable
-            ) {
-              if (!currentApplicable) {
-                return null;
-              }
-              function normaliseHeading(value) {
-                return String(value || '').toLowerCase()
-                  .replace(/[^a-z0-9 ]/g, '').trim();
-              }
-              function section(body, headings) {
-                const wanted = new Set(headings.map(normaliseHeading));
-                const lines = String(body || '').split(/\r?\n/);
-                const output = [];
-                let collecting = false;
-                for (const line of lines) {
-                  const heading = line.match(/^#{2,6}\s+(.+?)\s*$/);
-                  if (heading) {
-                    if (collecting) {
-                      break;
-                    }
-                    collecting = wanted.has(normaliseHeading(heading[1]));
-                    continue;
-                  }
-                  if (collecting) {
-                    output.push(line);
-                  }
-                }
-                return output.join('\n').trim();
-              }
-              const issueBody = String(
-                selectedIssue && selectedIssue.body || ''
-              );
-              const agentLogin = section(
-                issueBody,
-                ['agent login']
-              ).replace(/^\x60|\x60$/g, '').trim();
-              const runId = section(
-                issueBody,
-                ['agent run id', 'run id']
-              ).replace(/^\x60|\x60$/g, '').trim();
-              return {
-                issue_number:
-                  Number.isSafeInteger(selectedIssueNumber) &&
-                  selectedIssueNumber > 0
-                    ? selectedIssueNumber
-                    : null,
-                agent_login: agentLogin || null,
-                run_id: runId || null
-              };
-            }
-            function scheduledIdentityProjectionChanged(
-              previousProjection,
-              currentIdentity,
-              currentApplicable
-            ) {
-              if (!previousProjection ||
-                  previousProjection.projectionUsable === false ||
-                  previousProjection.notApplicable ||
-                  !currentApplicable) {
-                return false;
-              }
-              return JSON.stringify(currentIdentity) !== JSON.stringify(
-                previousProjection.identityProjection
-              );
-            }
-            function scheduledMutablePolicyProjectionChanged(
-              previousProjection,
-              currentErrors,
-              currentApplicable
-            ) {
-              if (!previousProjection ||
-                  previousProjection.projectionUsable === false ||
-                  previousProjection.notApplicable ||
-                  !currentApplicable) {
-                return false;
-              }
-              return JSON.stringify(currentErrors) !== JSON.stringify(
-                previousProjection.mutablePolicyErrors || []
-              );
-            }
-            async function currentReviewProjection(pull) {
-              const submitted = await github.paginate(
-                github.rest.pulls.listReviews,
-                {owner, repo, pull_number: pull.number, per_page: 100}
-              );
-              const copilotReviewerLogins = new Set([
-                'copilot-pull-request-reviewer[bot]',
-                'copilot-pull-request-reviewer'
-              ]);
-              const copilotStates = new Set([
-                'APPROVED', 'COMMENTED', 'CHANGES_REQUESTED'
-              ]);
-              const copilotCurrentHeadReviewed = submitted.some(review =>
-                review && review.user &&
-                copilotReviewerLogins.has(review.user.login) &&
-                copilotStates.has(review.state) &&
-                review.commit_id === pull.head.sha
-              );
-              const latestByAuthor = new Map();
-              for (const review of submitted) {
-                if (!review.user ||
-                    !['APPROVED', 'CHANGES_REQUESTED', 'DISMISSED'].includes(
-                      review.state
-                    )) {
-                  continue;
-                }
-                const previous = latestByAuthor.get(review.user.login);
-                if (!previous ||
-                    Date.parse(review.submitted_at || 0) >=
-                      Date.parse(previous.submitted_at || 0)) {
-                  latestByAuthor.set(review.user.login, review);
-                }
-              }
-              const unresolved = [];
-              for (const review of latestByAuthor.values()) {
-                if (review.state === 'CHANGES_REQUESTED') {
-                  unresolved.push('review:' + review.id);
-                }
-              }
-              const reviewResult = await github.graphql(
-                'query($owner:String!,$repo:String!,$number:Int!){' +
-                'repository(owner:$owner,name:$repo){pullRequest(number:$number){' +
-                'reviewDecision reviewThreads(first:100){nodes{id isResolved ' +
-                'comments(first:100){nodes{body author{login}} ' +
-                'pageInfo{hasNextPage}}} pageInfo{hasNextPage}}}}}',
-                {owner, repo, number: pull.number}
-              );
-              const pullReview = reviewResult.repository.pullRequest;
-              if (pullReview.reviewDecision === 'CHANGES_REQUESTED') {
-                unresolved.push('reviewDecision');
-              }
-              const aiReviewers = new Set([
-                'google-labs-jules[bot]',
-                'github-copilot[bot]',
-                'copilot-swe-agent[bot]',
-                'openai-codex[bot]',
-                'chatgpt-codex-connector[bot]',
-                ...copilotReviewerLogins,
-                'coderabbitai[bot]',
-                'vercel[bot]'
-              ].map(normaliseScheduledBotLogin));
-              const threads = pullReview.reviewThreads;
-              if (threads.pageInfo.hasNextPage ||
-                  threads.nodes.some(thread =>
-                    thread.comments.pageInfo.hasNextPage
-                  )) {
-                throw new Error('review_projection_truncated');
-              }
-              for (const thread of threads.nodes) {
-                const blocking = thread.comments.nodes.some(comment =>
-                  (comment.author && aiReviewers.has(
-                    normaliseScheduledBotLogin(comment.author.login)
-                  )) ||
-                  /VADE-RECOMMENDATION:\s*FIX|\b(?:blocking|must fix|regression)\b/i.test(
-                    String(comment.body || '')
-                  )
-                );
-                if (!thread.isResolved && blocking) {
-                  unresolved.push(thread.id);
-                }
-              }
-              return {
-                notApplicable: false,
-                copilotCurrentHeadReviewed,
-                unresolved: [...new Set(unresolved)].sort()
-              };
-            }
-            const pulls = await github.paginate(
-              github.rest.pulls.list,
-              {
-                owner,
-                repo,
-                state: 'open',
-                per_page: 100
-              }
-            );
-            for (const pull of pulls) {
-              let decision = 'retry';
-              let pendingRecovery = null;
-              let terminalInvalidation = null;
-              let observedLatestId = null;
-              const pullGateContext = gateStatusContext(pull.number);
-              try {
-                const gateContext = pullGateContext;
-                const statuses = await github.paginate(
-                  github.rest.repos.listCommitStatusesForRef,
-                  {owner, repo, ref: pull.head.sha, per_page: 100}
-                );
-                const gateStatuses = statuses
-                  .filter(status => status.context === gateContext)
-                  .sort((left, right) =>
-                    Date.parse(right.created_at) -
-                      Date.parse(left.created_at) ||
-                    right.id - left.id
-                  );
-                const latest = gateStatuses[0] || null;
-                observedLatestId = latest ? String(latest.id) : null;
-                if (!latest) {
-                  decision = scheduledRefreshDecision({
-                    apiComplete: true,
-                    statusCount: gateStatuses.length,
-                    latestState: latest && latest.state
-                  });
-                } else if (latest.state === 'pending') {
-                  let ownerRunState = 'missing';
-                  const runId = workflowRunId(latest.target_url);
-                  let pendingLeaseValid = Boolean(runId);
-                  if (runId) {
-                    try {
-                      const run = await github.rest.actions.getWorkflowRun({
-                        owner, repo, run_id: runId
-                      });
-                      ownerRunState = run.data.status;
-                      pendingLeaseValid =
-                        run.data.name === 'PR Checks' &&
-                        run.data.path === '.github/workflows/pr-checks.yml' &&
-                        ['pull_request_target', 'workflow_dispatch'].includes(
-                          run.data.event
-                        );
-                    } catch (error) {
-                      if (error.status !== 404) {
-                        throw error;
-                      }
-                    }
-                  }
-                  decision = scheduledRefreshDecision({
-                    apiComplete: true,
-                    statusCount: gateStatuses.length,
-                    latestState: latest.state,
-                    latestCreatedAt: latest.created_at,
-                    ownerPendingAt: latest.created_at,
-                    ownerRunState,
-                    pendingLeaseValid,
-                    now: new Date().toISOString()
-                  });
-                  if (decision === 'finalize_pending') {
-                    pendingRecovery = {latest, gateContext, ownerRunState};
-                  }
-                } else {
-                  const ownerId = terminalOwnerId(latest);
-                  const ownerPending = ownerId && gateStatuses.find(status =>
-                    String(status.id) === ownerId &&
-                    status.state === 'pending'
-                  );
-                  const ownerRunId = ownerPending && workflowRunId(
-                    ownerPending.target_url
-                  );
-                  const terminalRunId = workflowRunId(latest.target_url);
-                  if (!ownerPending || !ownerRunId || !terminalRunId ||
-                      ownerRunId !== terminalRunId) {
-                    decision = gateStatuses.length >= 998
-                      ? 'at_capacity'
-                      : 'dispatch';
-                  } else {
-                    const comments = await github.paginate(
-                      github.rest.issues.listComments,
-                      {owner, repo, issue_number: pull.number, per_page: 100}
-                    );
-                    const previousProjection = verdictProjection(comments);
-                    let reviewProjectionChanged = Boolean(
-                      (!previousProjection ||
-                        !previousProjection.projectionKnown) &&
-                      latest.state === 'success'
-                    );
-                    if (previousProjection &&
-                        previousProjection.projectionKnown &&
-                        !previousProjection.notApplicable) {
-                      const currentProjection =
-                        await currentReviewProjection(pull);
-                      reviewProjectionChanged =
-                        currentProjection.copilotCurrentHeadReviewed !==
-                          previousProjection.copilotCurrentHeadReviewed ||
-                        JSON.stringify(currentProjection.unresolved) !==
-                        JSON.stringify(previousProjection.unresolved);
-                    }
-                    const closingResult = await github.graphql(
-                      'query($owner:String!,$repo:String!,$number:Int!){' +
-                      'repository(owner:$owner,name:$repo){pullRequest(number:$number){' +
-                      'closingIssuesReferences(first:20){nodes{number body ' +
-                      'repository{nameWithOwner} labels(first:100){nodes{name} ' +
-                      'pageInfo{hasNextPage}}} pageInfo{hasNextPage}}}}}',
-                      {owner, repo, number: pull.number}
-                    );
-                    const references = closingResult.repository.pullRequest
-                      .closingIssuesReferences;
-                    if (references.pageInfo.hasNextPage) {
-                      throw new Error('closing_issues_truncated');
-                    }
-                    const linkedIssues = references.nodes.filter(issue =>
-                      issue.repository.nameWithOwner === owner + '/' + repo
-                    );
-                    if (linkedIssues.some(issue =>
-                      issue.labels.pageInfo.hasNextPage
-                    )) {
-                      throw new Error('issue_labels_truncated');
-                    }
-                    const linkProjection = scheduledLinkProjection(
-                      pull,
-                      linkedIssues
-                    );
-                    const selectedIssueNumber =
-                      linkProjection.selectedIssueNumber;
-                    let selectedIssue = linkedIssues.find(issue =>
-                      issue.number === selectedIssueNumber
-                    ) || null;
-                    let selectedIssueUnavailable = false;
-                    const applicableWithoutSelectedIssue =
-                      agentTaskApplicable(pull, null);
-                    if (selectedIssueNumber && !selectedIssue) {
-                      try {
-                        const selectedResponse =
-                          await github.rest.issues.get({
-                            owner,
-                            repo,
-                            issue_number: selectedIssueNumber
-                          });
-                        selectedIssue = {
-                          ...selectedResponse.data,
-                          labels: {
-                            nodes: (selectedResponse.data.labels || []).map(
-                              label => typeof label === 'string'
-                                ? {name: label}
-                                : {name: label.name}
-                            )
-                          }
-                        };
-                      } catch (error) {
-                        if (error.status !== 404 &&
-                            !(applicableWithoutSelectedIssue &&
-                              linkProjection.errors.length > 0)) {
-                          throw error;
-                        }
-                        selectedIssueUnavailable = true;
-                      }
-                    }
-                    const issueComments = selectedIssue
-                      ? await github.paginate(
-                        github.rest.issues.listComments,
-                        {
-                          owner,
-                          repo,
-                          issue_number: selectedIssue.number,
-                          per_page: 100
-                        }
-                      )
-                      : [];
-                    const currentApplicable = agentTaskApplicable(
-                      pull,
-                      selectedIssue
-                    );
-                    const currentIntentErrors = [
-                      ...linkProjection.errors,
-                      ...(selectedIssueUnavailable
-                        ? ['linked_issue_unavailable']
-                        : []),
-                      ...(currentApplicable && !selectedIssue
-                        ? ['missing_linked_issue']
-                        : []),
-                      ...(linkProjection.closingIssueCount === 1 &&
-                          selectedIssue
-                        ? intentContractErrors(
-                          selectedIssue,
-                          issueComments,
-                          pull.created_at
-                        )
-                        : [])
-                    ].filter((value, index, values) =>
-                      values.indexOf(value) === index
-                    ).sort();
-                    const currentMutablePolicyErrors =
-                      scheduledMutablePolicyErrors(
-                        pull,
-                        selectedIssue,
-                        currentApplicable
-                      );
-                    const currentContractIdentity =
-                      scheduledContractIdentity(
-                        selectedIssueNumber,
-                        selectedIssue,
-                        currentApplicable
-                      );
-                    const policyProjectionChanged =
-                      scheduledMutablePolicyProjectionChanged(
-                        previousProjection,
-                        currentMutablePolicyErrors,
-                        currentApplicable
-                      ) || scheduledIdentityProjectionChanged(
-                        previousProjection,
-                        currentContractIdentity,
-                        currentApplicable
-                      );
-                    const intentProjectionChanged =
-                      scheduledIntentProjectionChanged(
-                        previousProjection,
-                        currentIntentErrors,
-                        linkProjection.closingIssueCount,
-                        currentApplicable
-                      );
-                    const expectedAgents = new Set([
-                      scheduledAgentLogin(pull.body),
-                      scheduledAgentLogin(
-                        selectedIssue && selectedIssue.body
-                      )
-                    ].filter(Boolean));
-                    const evidenceComments = [
-                      ...comments,
-                      ...issueComments
-                    ].filter(comment =>
-                      scheduledCommentAffectsEvidence(
-                        comment,
-                        expectedAgents
-                      )
-                    );
-                    const ciRuns = await github.paginate(
-                      github.rest.actions.listWorkflowRunsForRepo,
-                      {owner, repo, head_sha: pull.head.sha, per_page: 100}
-                    );
-                    const exactCiRuns = ciRuns.filter(run =>
-                      run.name === 'CI' &&
-                      run.path === '.github/workflows/ci.yml' &&
-                      run.event === 'pull_request' &&
-                      run.head_sha === pull.head.sha &&
-                      (run.pull_requests || []).some(
-                        attached => attached.number === pull.number
-                      )
-                    );
-                    decision = scheduledRefreshDecision({
-                      apiComplete: true,
-                      statusCount: gateStatuses.length,
-                      latestState: latest.state,
-                      latestCreatedAt: latest.created_at,
-                      ownerPendingAt: ownerPending.created_at,
-                      ownerRunState: 'completed',
-                      now: new Date().toISOString(),
-                      commentUpdatedAt: latestIso(evidenceComments.map(comment =>
-                        comment.updated_at || comment.created_at
-                      )),
-                      ciUpdatedAt: latestIso(exactCiRuns.map(run =>
-                        run.updated_at || run.created_at
-                      )),
-                      reviewProjectionChanged,
-                      intentProjectionChanged,
-                      policyProjectionChanged,
-                      terminalAlreadyInvalidated: String(
-                        latest.description || ''
-                      ).includes('scheduled-evidence-invalidated')
-                    });
-                    if (decision === 'invalidate_terminal') {
-                      terminalInvalidation = {
-                        latest,
-                        gateContext,
-                        ownerId
-                      };
-                    }
-                  }
-                }
-              } catch (error) {
-                core.warning(
-                  'Evidence scan for PR #' + pull.number +
-                  ' will retry: ' + error.message
-                );
-                decision = scheduledRefreshDecision({
-                  apiComplete: false,
-                  statusCount: 0,
-                  latestState: null
-                });
-              }
-              if (decision === 'at_capacity') {
-                core.warning(
-                  'PR #' + pull.number + ' reached the safe status limit; ' +
-                  'push a new head or complete #874'
-                );
-              }
-              if ([
-                'dispatch',
-                'finalize_pending',
-                'invalidate_terminal'
-              ].includes(decision)) {
-                try {
-                  const currentStatuses = await github.paginate(
-                    github.rest.repos.listCommitStatusesForRef,
-                    {owner, repo, ref: pull.head.sha, per_page: 100}
-                  );
-                  const currentLatest = currentStatuses
-                    .filter(status => status.context === pullGateContext)
-                    .sort((left, right) =>
-                      Date.parse(right.created_at) -
-                        Date.parse(left.created_at) ||
-                      right.id - left.id
-                    )[0] || null;
-                  const currentLatestId = currentLatest
-                    ? String(currentLatest.id)
-                    : null;
-                  if (currentLatestId !== observedLatestId) {
-                    core.info(
-                      'Skipping stale scheduled decision for PR #' +
-                      pull.number
-                    );
-                    continue;
-                  }
-                } catch (error) {
-                  core.warning(
-                    'Could not recheck PR #' + pull.number + ': ' +
-                    error.message
-                  );
-                  continue;
-                }
-              }
-              if (decision === 'finalize_pending' && pendingRecovery) {
-                const latest = pendingRecovery.latest;
-                await github.rest.repos.createCommitStatus({
-                  owner,
-                  repo,
-                  sha: pull.head.sha,
-                  state: 'failure',
-                  context: pendingRecovery.gateContext,
-                  description: (
-                    'gate-owner:' + latest.id +
-                    ' scheduled recovery: ' +
-                    pendingRecovery.ownerRunState
-                  ).slice(0, 140),
-                  target_url: latest.target_url
-                });
-              }
-              if (decision === 'invalidate_terminal' &&
-                  terminalInvalidation) {
-                const latest = terminalInvalidation.latest;
-                await github.rest.repos.createCommitStatus({
-                  owner,
-                  repo,
-                  sha: pull.head.sha,
-                  state: 'failure',
-                  context: terminalInvalidation.gateContext,
-                  description: (
-                    'gate-owner:' + terminalInvalidation.ownerId +
-                    ' scheduled-evidence-invalidated'
-                  ).slice(0, 140),
-                  target_url: latest.target_url
-                });
-              }
-              if (decision === 'dispatch') {
-                await github.rest.actions.createWorkflowDispatch({
-                  owner,
-                  repo,
-                  workflow_id: 'pr-checks.yml',
-                  ref: context.payload.repository.default_branch,
-                  inputs: {pull_request: String(pull.number)}
-                });
-              }
-            }
-
-  dispatch-evidence-refresh:
-    needs: snapshot-agent-task-intent
-    if: >-
-      always() &&
-      (github.event_name == 'issue_comment' ||
-       github.event_name == 'issues' ||
-       github.event_name == 'workflow_run')
-    runs-on: ubuntu-latest
-    permissions:
-      actions: write
-      contents: read
-      pull-requests: read
-    steps:
-      - name: Queue the affected pull requests
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        with:
-          script: |
-            const owner = context.repo.owner;
-            const repo = context.repo.repo;
-            const trustedCommentAssociations = new Set([
-              'OWNER', 'MEMBER', 'COLLABORATOR'
-            ]);
-            const knownAgentCommenters = new Set([
-              'google-labs-jules[bot]',
-              'github-copilot[bot]',
-              'copilot-swe-agent[bot]',
-              'openai-codex[bot]',
-              'chatgpt-codex-connector[bot]',
-              'github-actions[bot]'
-            ]);
-            const trustedIssuePermissions = new Set([
-              'admin', 'maintain', 'write', 'triage'
-            ]);
-            function mayDispatchEvidenceRefresh(
-              authorAssociation,
-              actor,
-              trustedAssociations,
-              knownAgents
-            ) {
-              return trustedAssociations.has(authorAssociation) ||
-                knownAgents.has(actor);
-            }
-            function mayDispatchIssueRefresh(
-              actor,
-              permission,
-              roleName,
-              trustedPermissions
-            ) {
-              return Boolean(actor) && (
-                trustedPermissions.has(permission) ||
-                trustedPermissions.has(roleName)
-              );
-            }
-            function issueCommentAffectsEvidence(
-              issue,
-              comment,
-              previousBody
-            ) {
-              function normaliseHeading(value) {
-                return String(value || '').toLowerCase()
-                  .replace(/[^a-z0-9 ]/g, '').trim();
-              }
-              function section(body, heading) {
-                const lines = String(body || '').split(/\r?\n/);
-                const output = [];
-                let collecting = false;
-                for (const line of lines) {
-                  const match = line.match(/^#{2,6}\s+(.+?)\s*$/);
-                  if (match) {
-                    if (collecting) {
-                      break;
-                    }
-                    collecting = normaliseHeading(match[1]) === heading;
-                    continue;
-                  }
-                  if (collecting) {
-                    output.push(line);
-                  }
-                }
-                return output.join('\n').trim()
-                  .replace(/^\x60|\x60$/g, '');
-              }
-              const actor = comment && comment.user && comment.user.login;
-              const invalidationPattern =
-                //i
-              );
-              if (manifest) {
-                try {
-                  expectedAgent = String(
-                    JSON.parse(manifest[1].trim()).agent_login || ''
-                  ).trim();
-                } catch (error) {
-                  return false;
-                }
-              } else {
-                expectedAgent = section(
-                  issue && issue.body,
-                  'agent login'
-                );
-              }
-              if (!expectedAgent || actor !== expectedAgent) {
-                return false;
-              }
-              function isAgentEventBody(body) {
-                const value = String(body || '');
-                return //i.test(
-                  String(pull && pull.body || '')
-                );
-              // Provenance asserted by the pull request itself. Each of these
-              // is a claim by the producing side that this is agent work.
-              const pullProvenance = knownAgents.has(login) || agentBranch ||
-                manifestPresent ||
-                carriesLabel((pull && pull.labels) || [],
-                  ['agent', 'agenttask', 'mcpagent']);
-              // Issue-side dispatch. Label automation also applies agent task
-              // labels as topic tags to issues that never declared a contract,
-              // so the bare label is not evidence of a dispatch: it only
-              // counts when the issue actually declares the run id and login
-              // the gate goes on to require. Treating the bare label as
-              // sufficient made the gate permanently unsatisfiable for human
-              // pull requests closing such issues, because the contract they
-              // were measured against had never been written (#1130). Only the
-              // two contract labels count here -- the generic `agent` label is
-              // never recognised by the snapshot job or the collector, so it
-              // stays a pull-request-side provenance signal only.
-              const issueLabelSource = selectedIssue && selectedIssue.labels
-                ? Array.isArray(selectedIssue.labels)
-                  ? selectedIssue.labels
-                  : selectedIssue.labels.nodes || []
-                : [];
-              const issueDispatch =
-                carriesLabel(issueLabelSource, ['agenttask', 'mcpagent']) &&
-                declaresAgentContract(selectedIssue);
-              // Pull-side provenance says who produced the branch. It is not
-              // evidence that a dispatch contract exists to measure that
-              // branch against. The gate scores a pull request against the
-              // frozen intent snapshot on its linked issue, and that snapshot
-              // is only ever written by `snapshot-agent-task-intent`, which
-              // runs on `issues` events alone. With no linked issue there is
-              // no snapshot, no declared run id and no declared login, so
-              // `policy.agent_login`, `policy.run_id` and `issue.number` are
-              // all unsatisfiable and the verdict is permanently
-              // `invalid_payload` regardless of what the author does. A
-              // branch named `claude/...` is a naming convention, not a
-              // dispatch. Arming on it alone is what made this check red on
-              // pull requests that never had a contract to satisfy -- including
-              // #1368, which merged with this status failing.
-              //
-              // So provenance arms the gate only once a linked issue exists to
-              // verify against; with none, there is nothing to measure and the
-              // verdict is `not_applicable`. This does not create an escape
-              // hatch: a pull request that links a dispatched issue is still
-              // fully gated, and requiring a pull request to bind to a focused
-              // issue at all is separately owned by `Canonical issue and
-              // evidence`, which states a requirement an author can actually
-              // meet.
-              return login !== 'dependabot[bot]' &&
-                (issueDispatch || (pullProvenance && Boolean(selectedIssue)));
-            }
-
-            const prNumber = Number(process.env.INPUT_PR_NUMBER || 0);
-            if (!prNumber) {
-              fs.writeFileSync('gate-input.json', JSON.stringify({policy: {applicable: false}}, null, 2));
-              core.setOutput('pr_number', '');
-              core.setOutput('head_sha', '');
-              return;
-            }
-
-            const pr = (await github.rest.pulls.get({owner, repo, pull_number: prNumber})).data;
-            const resolvedHead = String(
-              process.env.RESOLVED_HEAD_SHA || ''
-            ).toLowerCase();
-            const resolvedBase = String(
-              process.env.RESOLVED_BASE_SHA || ''
-            ).toLowerCase();
-            const collectedHead = String(pr.head.sha || '').toLowerCase();
-            const collectedBase = String(pr.base.sha || '').toLowerCase();
-            core.setOutput('pr_number', String(prNumber));
-            core.setOutput('head_sha', collectedHead);
-            core.setOutput('base_sha', collectedBase);
-            if (!/^[a-f0-9]{40}$/.test(resolvedHead) ||
-                collectedHead !== resolvedHead) {
-              collectionErrors.push('stale_head');
-            }
-            if (!/^[a-f0-9]{40}$/.test(resolvedBase) ||
-                collectedBase !== resolvedBase) {
-              collectionErrors.push('stale_base');
-            }
-            const comparison = (await
-              github.rest.repos.compareCommitsWithBasehead({
-                owner,
-                repo,
-                basehead: pr.base.sha + '...' + pr.head.sha,
-                per_page: 100
-              })
-            ).data;
-            const changed = Array.isArray(comparison.files)
-              ? comparison.files
-              : [];
-            if (changed.length !== pr.changed_files) {
-              collectionErrors.push('changed_files_truncated');
-            }
-            const changedFiles = [...new Set(changed.flatMap(file => [
-              file.filename,
-              file.previous_filename
-            ].filter(Boolean)))];
-            const presentChangedFiles = new Set(
-              changed
-                .filter(file => file.status !== 'removed')
-                .map(file => file.filename)
-            );
-            const prBody = String(pr.body || '');
-            const manifestMatch = prBody.match(//i);
-            let manifest = {};
-            if (manifestMatch) {
-              try {
-                manifest = JSON.parse(manifestMatch[1]);
-              } catch (error) {
-                collectionErrors.push('invalid_agent_lock_manifest');
-              }
-            }
-            let closingReferenceNumbers = [];
-            try {
-              const closingResult = await github.graphql(
-                'query($owner:String!,$repo:String!,$number:Int!){' +
-                'repository(owner:$owner,name:$repo){pullRequest(number:$number){' +
-                'closingIssuesReferences(first:20){nodes{number repository{' +
-                'nameWithOwner}} pageInfo{hasNextPage}}}}}',
-                {owner, repo, number: prNumber}
-              );
-              const references =
-                closingResult.repository.pullRequest.closingIssuesReferences;
-              closingReferenceNumbers = [...new Set(references.nodes
-                .filter(reference =>
-                  reference.repository.nameWithOwner === owner + '/' + repo
-                )
-                .map(reference => reference.number))];
-              if (references.pageInfo.hasNextPage) {
-                collectionErrors.push('closing_issues_truncated');
-              }
-              if (closingReferenceNumbers.length > 1) {
-                collectionErrors.push('multiple_closing_issues');
-              }
-            } catch (error) {
-              collectionErrors.push('closing_issues_unavailable');
-            }
-            const closing = prBody.match(
-              /(?:close[sd]?|fix(?:e[sd])?|resolve[sd]?)\s+#(\d+)/i
-            );
-            const closingIssueNumber = closing ? Number(closing[1]) : 0;
-            const referencedIssueNumber =
-              closingReferenceNumbers.length === 1
-                ? closingReferenceNumbers[0]
-                : 0;
-            function linkedIssueContract(
-              manifestNumber,
-              textualNumber,
-              authoritativeNumber
-            ) {
-              const values = [
-                manifestNumber,
-                textualNumber,
-                authoritativeNumber
-              ];
-              if (!values.every(value =>
-                Number.isSafeInteger(value) && value > 0
-              )) {
-                return 'missing';
-              }
-              return new Set(values).size === 1 ? 'ok' : 'conflicting';
-            }
-            const linkContract = linkedIssueContract(
-              manifest.issue_number,
-              closingIssueNumber,
-              referencedIssueNumber
-            );
-            if (linkContract === 'missing') {
-              collectionErrors.push('incomplete_linked_issue_contract');
-            } else if (linkContract === 'conflicting') {
-              collectionErrors.push('conflicting_linked_issue');
-            }
-            const manifestIssueNumber =
-              Number.isSafeInteger(manifest.issue_number) &&
-              manifest.issue_number > 0
-                ? manifest.issue_number
-                : 0;
-            const issueNumber = manifestIssueNumber ||
-              referencedIssueNumber || closingIssueNumber;
-            let issue = null;
-            let issueComments = [];
-            let prComments = [];
-            if (issueNumber) {
-              try {
-                issue = (await github.rest.issues.get({owner, repo, issue_number: issueNumber})).data;
-              } catch (error) {
-                collectionErrors.push('linked_issue_unavailable');
-              }
-              try {
-                issueComments = await github.paginate(
-                  github.rest.issues.listComments,
-                  {owner, repo, issue_number: issueNumber, per_page: 100}
-                );
-              } catch (error) {
-                collectionErrors.push('issue_comments_unavailable');
-              }
-            }
-            try {
-              prComments = await github.paginate(
-                github.rest.issues.listComments,
-                {owner, repo, issue_number: prNumber, per_page: 100}
-              );
-            } catch (error) {
-              collectionErrors.push('pr_comments_unavailable');
-            }
-
-            const prLabels = new Set((pr.labels || []).map(label =>
-              normaliseHeading(typeof label === 'string' ? label : label.name)
-            ));
-            const issueLabels = new Set(((issue && issue.labels) || []).map(label =>
-              normaliseHeading(typeof label === 'string' ? label : label.name)
-            ));
-            const applicable = agentTaskApplicable(pr, issue);
-            if (applicable && !issue) {
-              collectionErrors.push('missing_linked_issue');
-            }
-            if (applicable && closingReferenceNumbers.length !== 1) {
-              collectionErrors.push('missing_closing_issue_reference');
-            }
-            if (applicable && issue &&
-                !['agenttask', 'mcpagent'].some(label => issueLabels.has(label))) {
-              collectionErrors.push('linked_issue_not_agent_task');
-            }
-
-            const issueBody = String((issue && issue.body) || '');
-            if (applicable && issue) {
-              const snapshotComment = issueComments.find(comment =>
-                comment.user && comment.user.login === 'github-actions[bot]' &&
-                //i
-                );
-                try {
-                  const snapshot = JSON.parse(snapshotMatch[1].trim());
-                  const currentHash = crypto.createHash('sha256')
-                    .update(issueBody.replace(/\r\n/g, '\n').trim(), 'utf8')
-                    .digest('hex');
-                  if (snapshot.issue_number !== issueNumber ||
-                      !/^[a-f0-9]{64}$/.test(String(snapshot.body_sha256 || ''))) {
-                    collectionErrors.push('invalid_intent_snapshot');
-                  } else if (snapshot.body_sha256 !== currentHash) {
-                    collectionErrors.push('intent_changed_after_dispatch');
-                  }
-                  if (!snapshotPredatesPull(
-                    snapshotComment.created_at,
-                    pr.created_at
-                  )) {
-                    collectionErrors.push('intent_snapshot_after_dispatch');
-                  }
-                  if (Boolean(snapshot.scope_unrestricted_approved) !==
-                      issueLabels.has('scopeunrestrictedapproved')) {
-                    collectionErrors.push(
-                      'unrestricted_approval_changed_after_dispatch'
-                    );
-                  }
-                  if (intentInvalidated &&
-                      !collectionErrors.includes(
-                        'intent_changed_after_dispatch'
-                      )) {
-                    collectionErrors.push('intent_changed_after_dispatch');
-                  }
-                } catch (error) {
-                  collectionErrors.push('invalid_intent_snapshot');
-                }
-              }
-            }
-            const objectiveSection = section(issueBody, ['objective', 'description']);
-            const hasObjectiveHeading = /^#{2,6}\s+(objective|description)\s*$/im.test(issueBody);
-            const description = objectiveSection || (hasObjectiveHeading ? '' : issueBody.trim());
-            const acceptance = listItems(section(issueBody, ['acceptance criteria', 'acceptance tests']));
-            const declaredFiles = listItems(section(issueBody, ['declared file scope', 'file scope', 'scope']));
-            const allowedExtraFiles = listItems(section(issueBody, ['allowed extra files', 'scope allowlist']));
-            const unrestrictedSection = section(issueBody, ['unrestricted scope', 'scope unrestricted']);
-            const unrestrictedRequested = checkboxChecked(
-              unrestrictedSection
-            );
-            const scopeUnrestricted = unrestrictedRequested &&
-              issueLabels.has('scopeunrestrictedapproved');
-            if (applicable && unrestrictedRequested && !scopeUnrestricted) {
-              collectionErrors.push('unrestricted_scope_not_approved');
-            }
-            const expectedTests = listItems(section(issueBody, ['focused test paths', 'focused tests', 'test scope']));
-            const expectedRunId = section(issueBody, ['agent run id', 'run id'])
-              .replace(/^\x60|\x60$/g, '').trim();
-            const expectedAgentLogin = section(issueBody, ['agent login'])
-              .replace(/^\x60|\x60$/g, '').trim();
-            // A linked issue carrying an agent task label without declaring a
-            // run id and login was never dispatched to an agent -- the label
-            // is topic noise. Surface the mislabel so it is visible and
-            // correctable (#1130). Keyed on the missing contract itself, not
-            // on inapplicability, so pull requests that are inapplicable for
-            // other reasons (for example Dependabot's unconditional
-            // exclusion) never receive a notice falsely claiming a declared
-            // contract is missing.
-            const contractDeclared = [expectedRunId, expectedAgentLogin]
-              .every(value => value && value !== '_No response_');
-            if (issue && !contractDeclared &&
-                ['agenttask', 'mcpagent'].some(label =>
-                  issueLabels.has(label))) {
-              core.notice(
-                'mislabelled_agent_task: issue #' + issue.number +
-                ' carries an agent task label but declares no Agent Run ID' +
-                ' / Agent Login. The label asserts an agent dispatch' +
-                ' contract; remove it from the issue if it was applied in' +
-                ' error.'
-              );
-            }
-            if (applicable && !expectedRunId) {
-              collectionErrors.push('missing_agent_run_id');
-            } else if (applicable && String(manifest.run_id || '') !== expectedRunId) {
-              collectionErrors.push('agent_run_id_mismatch');
-            }
-            if (applicable && !expectedAgentLogin) {
-              collectionErrors.push('missing_agent_login');
-            } else if (applicable &&
-                       String(manifest.agent_login || '') !== expectedAgentLogin) {
-              collectionErrors.push('agent_login_mismatch');
-            }
-
-            const eventAuthors = new Set(
-              expectedAgentLogin ? [expectedAgentLogin] : []
-            );
-            const events = [];
-            function legacyRunId(value) {
-              const match = String(value || '').match(
-                /(?:\b(?:run|task)[ _-]?id\s*[:=\/]\s*|\btasks?\/)([A-Za-z0-9][A-Za-z0-9._:-]{0,127})/i
-              );
-              return match
-                ? match[1].replace(/[.,;:]+$/, '')
-                : null;
-            }
-            if (issueNumber) {
-              const eventComments = [...new Map(
-                [...issueComments, ...prComments].map(comment => [
-                  comment.id,
-                  comment
-                ])
-              ).values()];
-              for (const comment of eventComments) {
-                if (!comment.user || !eventAuthors.has(comment.user.login)) {
-                  continue;
-                }
-                const body = String(comment.body || '');
-                const structuredMatch = body.match(
-                  //i
-                );
-                if (structuredMatch) {
-                  try {
-                    const structuredEvent = JSON.parse(
-                      structuredMatch[1].trim()
-                    );
-                    if (!['artifact_ready', 'completed', 'error'].includes(
-                      structuredEvent.kind
-                    ) || typeof structuredEvent.run_id !== 'string' ||
-                        typeof structuredEvent.head_sha !== 'string' ||
-                        !/^[a-f0-9]{40}$/i.test(structuredEvent.head_sha)) {
-                      throw new Error('invalid structured event');
-                    }
-                    events.push({
-                      kind: structuredEvent.kind,
-                      sequence: Date.parse(comment.created_at) || comment.id,
-                      comment_id: comment.id,
-                      author: comment.user.login,
-                      run_id: structuredEvent.run_id,
-                      head_sha: structuredEvent.head_sha.toLowerCase()
-                    });
-                  } catch (error) {
-                    collectionErrors.push('invalid_agent_event');
-                  }
-                  continue;
-                }
-                let kind = '';
-                if (/unexpected error|unable to complete|wasn.t able to complete|run failed|failed to complete/i.test(body)) {
-                  kind = 'error';
-                } else if (/ready for (?:a )?review|pull request is ready|created (?:a )?pull request|\[PR\]\([^)]*\)\s+has been created/i.test(body)) {
-                  kind = 'artifact_ready';
-                } else if (/\b(?:task|work|implementation)\s+(?:is\s+)?complete(?:d)?\b/i.test(body)) {
-                  kind = 'completed';
-                }
-                if (kind) {
-                  const headMatch = body.match(/\b[a-f0-9]{40}\b/i);
-                  events.push({
-                    kind,
-                    sequence: Date.parse(comment.created_at) || comment.id,
-                    comment_id: comment.id,
-                    author: comment.user.login,
-                    run_id: legacyRunId(body),
-                    head_sha: headMatch ? headMatch[0].toLowerCase() : null
-                  });
-                }
-              }
-              events.sort((left, right) => left.sequence - right.sequence);
-            }
-
-            const reviews = [];
-            const submitted = await github.paginate(
-              github.rest.pulls.listReviews,
-              {owner, repo, pull_number: prNumber, per_page: 100}
-            );
-            const copilotReviewerLogins = new Set([
-              'copilot-pull-request-reviewer[bot]',
-              'copilot-pull-request-reviewer'
-            ]);
-            const submittedCopilotStates = new Set([
-              'APPROVED', 'COMMENTED', 'CHANGES_REQUESTED'
-            ]);
-            function normaliseBotLogin(login) {
-              return String(login || '').toLowerCase()
-                .replace(/\[bot\]$/, '');
-            }
-            function aiReviewerLoginSet() {
-              return new Set([
-                'google-labs-jules[bot]',
-                'github-copilot[bot]',
-                'copilot-swe-agent[bot]',
-                'openai-codex[bot]',
-                'chatgpt-codex-connector[bot]',
-                'copilot-pull-request-reviewer[bot]',
-                'copilot-pull-request-reviewer',
-                'coderabbitai[bot]',
-                'vercel[bot]'
-              ].map(login => String(login || '').toLowerCase()
-                .replace(/\[bot\]$/, '')));
-            }
-            const aiReviewerLogins = aiReviewerLoginSet();
-            function isCurrentCopilotReview(
-              review,
-              reviewerLogins,
-              submittedStates,
-              headSha
-            ) {
-              return Boolean(
-                review && review.user &&
-                reviewerLogins.has(review.user.login) &&
-                submittedStates.has(review.state) &&
-                review.commit_id === headSha
-              );
-            }
-            const copilotCurrentHeadReviewed = submitted.some(review =>
-              isCurrentCopilotReview(
-                review,
-                copilotReviewerLogins,
-                submittedCopilotStates,
-                pr.head.sha
-              )
-            );
-            const copilotRabbitLabel = (pr.labels || []).some(label =>
-              String(typeof label === 'string' ? label : label.name)
-                .toLowerCase() === 'copilot-rabbit'
-            );
-            const latestByAuthor = new Map();
-            for (const review of submitted) {
-              if (!review.user ||
-                  !['APPROVED', 'CHANGES_REQUESTED', 'DISMISSED'].includes(
-                    review.state
-                  )) {
-                continue;
-              }
-              const previous = latestByAuthor.get(review.user.login);
-              if (!previous || Date.parse(review.submitted_at || 0) >= Date.parse(previous.submitted_at || 0)) {
-                latestByAuthor.set(review.user.login, review);
-              }
-            }
-            for (const review of latestByAuthor.values()) {
-              if (review.state === 'CHANGES_REQUESTED') {
-                reviews.push({blocking: true, resolved: false, source: 'review:' + review.id});
-              }
-            }
-            try {
-              const result = await github.graphql(
-                'query($owner:String!,$repo:String!,$number:Int!){' +
-                'repository(owner:$owner,name:$repo){pullRequest(number:$number){' +
-                'reviewDecision reviewThreads(first:100){nodes{id isResolved comments(first:100){' +
-                'nodes{body author{login}} pageInfo{hasNextPage}}} pageInfo{hasNextPage}}}}}',
-                {owner, repo, number: prNumber}
-              );
-              if (result.repository.pullRequest.reviewDecision ===
-                  'CHANGES_REQUESTED') {
-                reviews.push({
-                  blocking: true,
-                  resolved: false,
-                  source: 'reviewDecision'
-                });
-              }
-              const threads = result.repository.pullRequest.reviewThreads;
-              for (const thread of threads.nodes) {
-                if (thread.comments.pageInfo.hasNextPage) {
-                  collectionErrors.push(
-                    'review_thread_comments_truncated'
-                  );
-                }
-                const blocking = thread.comments.nodes.some(comment =>
-                  (comment.author &&
-                    aiReviewerLogins.has(
-                      normaliseBotLogin(comment.author.login)
-                    )) ||
-                  /VADE-RECOMMENDATION:\s*FIX|\b(?:blocking|must fix|regression)\b/i.test(
-                    String(comment.body || '')
-                  )
-                );
-                if (!thread.isResolved && blocking) {
-                  reviews.push({blocking: true, resolved: false, source: thread.id});
-                }
-              }
-              if (threads.pageInfo.hasNextPage) {
-                collectionErrors.push('review_threads_truncated');
-              }
-            } catch (error) {
-              collectionErrors.push('review_threads_unavailable');
-            }
-
-            function focusedTestResultsFromLog(log, expectedPaths) {
-              const outcomes = {
-                PASSED: 'passed',
-                FAILED: 'failed',
-                ERROR: 'errors',
-                SKIPPED: 'skipped',
-                XFAIL: 'xfailed',
-                XPASS: 'xpassed'
-              };
-              const results = Object.fromEntries(expectedPaths.map(path => [
-                path,
-                {
-                  passed: 0,
-                  failed: 0,
-                  errors: 0,
-                  skipped: 0,
-                  xfailed: 0,
-                  xpassed: 0
-                }
-              ]));
-              const lines = String(log || '')
-                .replace(/\x1B\[[0-?]*[ -/]*[@-~]/g, '')
-                .split(/\r\n|\n|\r/);
-              let activePath = null;
-              let sawLiveLog = false;
-              let pendingCombined = null;
-              function record(path, outcome) {
-                results[path][outcomes[outcome]] += 1;
-              }
-              function commitPendingCombined() {
-                if (pendingCombined) {
-                  record(
-                    pendingCombined.path,
-                    pendingCombined.outcome
-                  );
-                  pendingCombined = null;
-                }
-              }
-              for (const rawLine of lines) {
-                const line = rawLine
-                  .replace(/^\uFEFF/, '')
-                  .replace(/^\d{4}-\d{2}-\d{2}T\S+\s+/, '')
-                  .trimEnd();
-                if (!line.trim()) {
-                  continue;
-                }
-                const liveLogHeader =
-                  /^-+\s+live log (?:setup|call|teardown)\s+-+$/i.test(
-                    line
-                  );
-                if (pendingCombined) {
-                  if (liveLogHeader) {
-                    activePath = pendingCombined.path;
-                    sawLiveLog = true;
-                    pendingCombined = null;
-                    continue;
-                  }
-                  commitPendingCombined();
-                }
-                const standaloneOutcome = line.match(
-                  /^(PASSED|FAILED|ERROR|SKIPPED|XFAIL|XPASS)(?:\s+\(.*\))?(?:\s+\[\s*\d+%\])?$/
-                );
-                if (standaloneOutcome) {
-                  if (activePath && sawLiveLog) {
-                    record(activePath, standaloneOutcome[1]);
-                  }
-                  activePath = null;
-                  sawLiveLog = false;
-                  continue;
-                }
-                if (/^=+/.test(line) || /^##\[/.test(line)) {
-                  activePath = null;
-                  sawLiveLog = false;
-                  continue;
-                }
-                const separator = line.indexOf('::');
-                const candidate = separator > 0
-                  ? line.slice(0, separator)
-                  : '';
-                const tail = separator > 0
-                  ? line.slice(separator + 2)
-                  : '';
-                const nodePath =
-                  candidate === candidate.trimStart() &&
-                  candidate.endsWith('.py') &&
-                  tail && !/^\s/.test(tail)
-                    ? candidate
-                    : null;
-                if (nodePath !== null) {
-                  activePath = expectedPaths.includes(nodePath)
-                    ? nodePath
-                    : null;
-                  sawLiveLog = false;
-                  if (!activePath) {
-                    continue;
-                  }
-                  const combinedOutcome = line.match(
-                    /\s(PASSED|FAILED|ERROR|SKIPPED|XFAIL|XPASS)(?:\s+\(.*\))?(?:\s+\[\s*\d+%\])?\s*$/
-                  );
-                  if (combinedOutcome) {
-                    pendingCombined = {
-                      path: activePath,
-                      outcome: combinedOutcome[1]
-                    };
-                    activePath = null;
-                  }
-                  continue;
-                }
-                if (liveLogHeader && activePath) {
-                  sawLiveLog = true;
-                }
-              }
-              commitPendingCombined();
-              return results;
-            }
-            async function ciRunFor(sha, expectedEvent) {
-              if (!sha) {
-                return null;
-              }
-              const workflowRuns = (await github.rest.actions.listWorkflowRunsForRepo({
-                owner, repo, head_sha: sha, per_page: 100
-              })).data.workflow_runs;
-              const ciRuns = workflowRuns
-                .filter(run =>
-                  run.name === 'CI' &&
-                  run.path === '.github/workflows/ci.yml' &&
-                  run.event === expectedEvent &&
-                  run.head_sha === sha &&
-                  (expectedEvent !== 'pull_request' ||
-                    (run.pull_requests || []).some(
-                      pull => pull.number === prNumber
-                    ))
-                )
-                .sort((left, right) =>
-                  Date.parse(right.created_at) - Date.parse(left.created_at)
-                );
-              return ciRuns[0] || null;
-            }
-            function ciRunPassed(run) {
-              return Boolean(run) && run.status === 'completed' &&
-                run.conclusion === 'success';
-            }
-            async function collectFocusedTestResults(run, expectedPaths) {
-              const emptyResults = focusedTestResultsFromLog(
-                '',
-                expectedPaths
-              );
-              if (expectedPaths.length === 0 || !ciRunPassed(run)) {
-                return emptyResults;
-              }
-              try {
-                const jobs = await github.paginate(
-                  github.rest.actions.listJobsForWorkflowRun,
-                  {
-                    owner,
-                    repo,
-                    run_id: run.id,
-                    filter: 'latest',
-                    per_page: 100
-                  }
-                );
-                const testJobs = jobs.filter(job => job.name === 'test');
-                if (testJobs.length !== 1 ||
-                    testJobs[0].status !== 'completed' ||
-                    testJobs[0].conclusion !== 'success') {
-                  collectionErrors.push(
-                    'focused_test_evidence_unavailable'
-                  );
-                  return emptyResults;
-                }
-                const logResponse =
-                  await github.rest.actions.downloadJobLogsForWorkflowRun({
-                    owner,
-                    repo,
-                    job_id: testJobs[0].id
-                  });
-                const logData = logResponse.data;
-                let logText = typeof logData === 'string'
-                  ? logData
-                  : logData && typeof logData.text === 'function'
-                    ? await logData.text()
-                    : Buffer.from(logData || []).toString('utf8');
-                const location = logResponse.headers &&
-                  logResponse.headers.location;
-                if (!logText && location) {
-                  const download = await fetch(location);
-                  if (!download.ok) {
-                    throw new Error('focused test log download failed');
-                  }
-                  logText = await download.text();
-                }
-                if (!logText) {
-                  collectionErrors.push(
-                    'focused_test_evidence_unavailable'
-                  );
-                  return emptyResults;
-                }
-                return focusedTestResultsFromLog(logText, expectedPaths);
-              } catch (error) {
-                collectionErrors.push('focused_test_evidence_unavailable');
-                return emptyResults;
-              }
-            }
-            const requiredCiRun = await ciRunFor(
-              pr.head.sha,
-              'pull_request'
-            );
-            const requiredChecksPassed = ciRunPassed(requiredCiRun);
-            const postMergeCiRun = Boolean(pr.merged)
-              ? await ciRunFor(pr.merge_commit_sha, 'push')
-              : null;
-            const postMergeChecksPassed = ciRunPassed(postMergeCiRun);
-            if (changedFiles.includes('.github/workflows/ci.yml')) {
-              collectionErrors.push('trusted_ci_workflow_changed');
-            }
-
-            const behaviorChangedFiles = changedFiles.filter(path => {
-              const explicitlyNonBehavioral =
-                /(?:^|\/)(?:test|tests|__tests__)(?:\/|_)/i.test(path) ||
-                /(?:\.spec\.|\.test\.)/i.test(path) ||
-                /^docs\//i.test(path) ||
-                /\.md$/i.test(path) ||
-                /^\.github\/ISSUE_TEMPLATE\//i.test(path) ||
-                /^\.github\/pull_request_template\.md$/i.test(path);
-              return !explicitlyNonBehavioral;
-            });
-            const ciCollectedTests = expectedTests.every(path =>
-              /^tests\/unit\/(?:test_.*|.*_test)\.py$/.test(path) &&
-              path !== 'tests/unit/test_transcript_action_workflow.py'
-            );
-            const allExpectedTestsChanged =
-              expectedTests.length > 0 &&
-              ciCollectedTests &&
-              expectedTests.every(path => presentChangedFiles.has(path));
-            const focusedTestFiles = allExpectedTestsChanged ? expectedTests : [];
-            const focusedTestResults = await collectFocusedTestResults(
-              requiredCiRun,
-              focusedTestFiles
-            );
-            const titleValid = pr.title.length >= 10 &&
-              /^(?:⚡\s*)?(?:feat|fix|docs|style|refactor|test|chore|perf|ci|build|revert)(?:\(.+\))?:/i.test(pr.title);
-
-            const finalPr = (await github.rest.pulls.get({
-              owner,
-              repo,
-              pull_number: prNumber
-            })).data;
-            if (String(finalPr.head.sha || '').toLowerCase() !==
-                collectedHead) {
-              collectionErrors.push('stale_head');
-            }
-            if (String(finalPr.base.sha || '').toLowerCase() !==
-                collectedBase) {
-              collectionErrors.push('stale_base');
-            }
-            const payload = {
-              policy: {
-                applicable,
-                agent_login: expectedAgentLogin || null,
-                run_id: expectedRunId || null,
-                head_sha: pr.head.sha
-              },
-              issue: {
-                number: issueNumber || null,
-                description,
-                acceptance_criteria: acceptance,
-                declared_files: declaredFiles,
-                allowed_extra_files: allowedExtraFiles,
-                scope_unrestricted: scopeUnrestricted
-              },
-              pull_request: {
-                number: prNumber,
-                changed_files: changedFiles,
-                present_changed_files: [...presentChangedFiles],
-                merged: Boolean(pr.merged),
-                draft: Boolean(pr.draft),
-                title_valid: titleValid,
-                required_checks_passed: requiredChecksPassed,
-                post_merge_checks_passed: postMergeChecksPassed
-              },
-              events,
-              reviews,
-              evidence: {
-                behavior_changed_files: behaviorChangedFiles,
-                focused_test_files: focusedTestFiles,
-                focused_test_results: focusedTestResults,
-                copilot_current_head_reviewed: copilotCurrentHeadReviewed,
-                copilot_rabbit_label: copilotRabbitLabel
-              },
-              collection_errors: collectionErrors
-            };
-            fs.writeFileSync('gate-input.json', JSON.stringify(payload, null, 2));
-
-      - name: Evaluate completion evidence
-        id: gate
-        shell: bash
-        run: |
-          set +e
-          python3 .trusted/scripts/ci/agent_completion_gate.py gate-input.json > gate-verdict.json
-          code=$?
-          set -e
-          echo "exit_code=$code" >> "$GITHUB_OUTPUT"
-
-      - name: Upload gate evidence
-        id: upload
-        if: always() && steps.resolve.outputs.pr_number != ''
-        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
-        with:
-          name: agent-completion-verdict-${{ steps.resolve.outputs.pr_number }}
-          path: |
-            gate-input.json
-            gate-verdict.json
-          if-no-files-found: error
-
-      - name: Publish stable status and comment
-        id: publish
-        if: always() && steps.resolve.outputs.pr_number != ''
-        uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
-        env:
-          PR_NUMBER: ${{ steps.resolve.outputs.pr_number }}
-          HEAD_SHA: ${{ steps.resolve.outputs.head_sha }}
-          BASE_SHA: ${{ steps.resolve.outputs.base_sha }}
-          COLLECTED_HEAD_SHA: ${{ steps.collect.outputs.head_sha }}
-          COLLECTED_BASE_SHA: ${{ steps.collect.outputs.base_sha }}
-          PENDING_STATUS_ID: ${{ steps.resolve.outputs.pending_status_id }}
-          COLLECT_OUTCOME: ${{ steps.collect.outcome }}
-          GATE_OUTCOME: ${{ steps.gate.outcome }}
-          GATE_EXIT_CODE: ${{ steps.gate.outputs.exit_code }}
-          ARTIFACT_OUTCOME: ${{ steps.upload.outcome }}
-        with:
-          script: |
-            const fs = require('fs');
-            const owner = context.repo.owner;
-            const repo = context.repo.repo;
-            const marker = '';
-            const runUrlPrefix = context.serverUrl + '/' + owner + '/' +
-              repo + '/actions/runs/';
-            const runUrl = runUrlPrefix + context.runId;
-            const gateContext = 'agent-completion/truth-gate/pr-' +
-              process.env.PR_NUMBER;
-            function gateStatusDisposition(
-              status,
-              expectedPendingId,
-              currentRunUrl,
-              targetPrefix
-            ) {
-              if (!/^\d+$/.test(String(expectedPendingId || '')) ||
-                  !status || !/^\d+$/.test(String(status.id || ''))) {
-                return 'fail_closed';
-              }
-              const target = String(
-                (status && status.target_url) || ''
-              );
-              const expectedId = BigInt(String(expectedPendingId));
-              const statusId = BigInt(String(status.id));
-              function validRunTarget(targetUrl) {
-                const value = String(targetUrl || '');
-                if (!value.startsWith(targetPrefix)) {
-                  return false;
-                }
-                const suffix = value.slice(targetPrefix.length);
-                return /^\d+$/.test(suffix);
-              }
-              function statusOwnerId(candidate) {
-                if (candidate.state === 'pending') {
-                  return BigInt(String(candidate.id));
-                }
-                const owner = String(candidate.description || '').match(
-                  /^gate-owner:(\d+)(?:\s|$)/
-                );
-                return owner ? BigInt(owner[1]) : null;
-              }
-              if (!validRunTarget(currentRunUrl) ||
-                  !validRunTarget(target)) {
-                return 'fail_closed';
-              }
-              const ownerId = statusOwnerId(status);
-              if (ownerId === null) {
-                return 'fail_closed';
-              }
-              if (ownerId === expectedId && target === currentRunUrl) {
-                if (statusId === expectedId &&
-                    status.state === 'pending') {
-                  return 'current_pending';
-                }
-                if (['failure', 'error'].includes(status.state)) {
-                  return 'already_failed';
-                }
-                if (status.state === 'success') {
-                  return 'already_succeeded';
-                }
-                return 'fail_closed';
-              }
-              if (target === currentRunUrl) {
-                return 'fail_closed';
-              }
-              if (ownerId > expectedId) {
-                return 'successor';
-              }
-              if (ownerId < expectedId) {
-                return 'predecessor';
-              }
-              return 'fail_closed';
-            }
-            function ownedDescription(message) {
-              return (
-                'gate-owner:' + process.env.PENDING_STATUS_ID + ' ' +
-                String(message || '')
-              ).slice(0, 140);
-            }
-            async function currentRunMayPublish(status) {
-              const disposition = gateStatusDisposition(
-                status,
-                process.env.PENDING_STATUS_ID,
-                runUrl,
-                runUrlPrefix
-              );
-              if (disposition === 'current_pending') {
-                return true;
-              }
-              if (disposition === 'predecessor') {
-                core.warning('Overriding a late predecessor publication');
-                return true;
-              }
-              if (disposition === 'successor') {
-                core.warning('Skipping superseded gate publication');
-                return false;
-              }
-              if (disposition === 'already_succeeded') {
-                core.warning('Current run already published success');
-                return false;
-              }
-              if (disposition === 'already_failed') {
-                core.setFailed('Current gate status is already failed');
-                return false;
-              }
-              core.setFailed(
-                'Current run has no proven gate-status lease'
-              );
-              return false;
-            }
-            function commitEvidenceDisposition(
-              resolvedCommit,
-              collectedCommit,
-              currentCommit
-            ) {
-              const commits = [
-                resolvedCommit,
-                collectedCommit,
-                currentCommit
-              ].map(
-                commit => String(commit || '').toLowerCase()
-              );
-              if (commits.some(
-                commit => !/^[a-f0-9]{40}$/.test(commit)
-              )) {
-                return 'stale_commit';
-              }
-              return commits.every(commit => commit === commits[0])
-                ? 'current_commit'
-                : 'stale_commit';
-            }
-            let verdict = {
-              verdict: 'blocked',
-              reasons: ['verdict_artifact_missing'],
-              details: {}
-            };
-            try {
-              const candidate = JSON.parse(
-                fs.readFileSync('gate-verdict.json', 'utf8')
-              );
-              const validVerdicts = new Set([
-                'blocked', 'ready', 'completed', 'not_applicable'
-              ]);
-              if (!candidate || typeof candidate !== 'object' ||
-                  !validVerdicts.has(candidate.verdict) ||
-                  !Array.isArray(candidate.reasons) ||
-                  candidate.reasons.some(reason => typeof reason !== 'string') ||
-                  !candidate.details || typeof candidate.details !== 'object' ||
-                  Array.isArray(candidate.details)) {
-                throw new Error('invalid verdict schema');
-              }
-              verdict = candidate;
-            } catch (error) {
-              core.warning('Could not validate gate verdict: ' + error.message);
-            }
-
-            const forcedReasons = [];
-            if (process.env.COLLECT_OUTCOME !== 'success') {
-              forcedReasons.push('evidence_collection_step_failed');
-            }
-            if (process.env.GATE_OUTCOME !== 'success') {
-              forcedReasons.push('gate_evaluation_step_failed');
-            }
-            if (process.env.ARTIFACT_OUTCOME !== 'success') {
-              forcedReasons.push('evidence_artifact_failed');
-            }
-            if (verdict.verdict !== 'blocked' &&
-                process.env.GATE_EXIT_CODE !== '0') {
-              forcedReasons.push('gate_exit_code_mismatch');
-            }
-            const current = (await github.rest.pulls.get({
-              owner,
-              repo,
-              pull_number: Number(process.env.PR_NUMBER)
-            })).data;
-            if (commitEvidenceDisposition(
-              process.env.HEAD_SHA,
-              process.env.COLLECTED_HEAD_SHA,
-              current.head.sha
-            ) !== 'current_commit') {
-              forcedReasons.push('stale_head');
-            }
-            if (commitEvidenceDisposition(
-              process.env.BASE_SHA,
-              process.env.COLLECTED_BASE_SHA,
-              current.base.sha
-            ) !== 'current_commit') {
-              forcedReasons.push('stale_base');
-            }
-            if (forcedReasons.length > 0) {
-              verdict = {
-                verdict: 'blocked',
-                reasons: [...new Set([...verdict.reasons, ...forcedReasons])],
-                details: verdict.details
-              };
-            }
-
-            async function latestGateStatus() {
-              const statuses = await github.paginate(
-                github.rest.repos.listCommitStatusesForRef,
-                {
-                  owner,
-                  repo,
-                  ref: process.env.HEAD_SHA,
-                  per_page: 100
-                }
-              );
-              return statuses
-                .filter(status =>
-                  status.context === gateContext
-                )
-                .sort((left, right) =>
-                  Date.parse(right.created_at) - Date.parse(left.created_at) ||
-                  right.id - left.id
-                )[0];
-            }
-            let latest = await latestGateStatus();
-            if (!await currentRunMayPublish(latest)) {
-              return;
-            }
-
-            const reasons = verdict.reasons;
-            const escaped = JSON.stringify(verdict, null, 2)
-              .replace(/&/g, '&')
-              .replace(//g, '>');
-            const body = marker + '\n## Agent Completion Truth Gate: ' +
-              verdict.verdict.toUpperCase() + '\n\n' +
-              (reasons.length > 0
-                ? '**Reasons:** ' + reasons
-                  .map(reason => '' + reason + '').join(', ')
-                : '**Evidence agrees.**') +
-              '\n\n
Machine-readable verdict
' +
-              escaped + '
\n\n[Workflow evidence](' + runUrl + ')'; - const comments = await github.paginate( - github.rest.issues.listComments, - { - owner, - repo, - issue_number: Number(process.env.PR_NUMBER), - per_page: 100 - } - ); - const existing = comments.find(comment => - comment.user && comment.user.login === 'github-actions[bot]' && - String(comment.body || '').includes(marker) - ); - if (existing) { - await github.rest.issues.updateComment({ - owner, - repo, - comment_id: existing.id, - body - }); - } else { - await github.rest.issues.createComment({ - owner, - repo, - issue_number: Number(process.env.PR_NUMBER), - body - }); - } - await core.summary - .addHeading('Agent Completion Truth Gate') - .addCodeBlock(JSON.stringify(verdict, null, 2), 'json') - .write(); - - latest = await latestGateStatus(); - if (!await currentRunMayPublish(latest)) { - return; - } - const passed = verdict.verdict !== 'blocked'; - const summary = reasons.length > 0 - ? reasons.join(', ') - : verdict.verdict + ': all rules passed'; - await github.rest.repos.createCommitStatus({ - owner, - repo, - sha: process.env.HEAD_SHA, - state: passed ? 'success' : 'failure', - context: gateContext, - description: ownedDescription(summary), - target_url: runUrl - }); - if (!passed) { - core.setFailed('Agent completion evidence is blocked'); - } - - - name: Finalize failed gate publication - if: >- - always() && - steps.resolve.outputs.pr_number != '' && - steps.publish.outcome != 'success' - uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 - env: - PR_NUMBER: ${{ steps.resolve.outputs.pr_number }} - HEAD_SHA: ${{ steps.resolve.outputs.head_sha }} - PENDING_STATUS_ID: ${{ steps.resolve.outputs.pending_status_id }} - with: - script: | - const owner = context.repo.owner; - const repo = context.repo.repo; - const runUrlPrefix = context.serverUrl + '/' + owner + '/' + - repo + '/actions/runs/'; - const runUrl = runUrlPrefix + context.runId; - const gateContext = 'agent-completion/truth-gate/pr-' + - process.env.PR_NUMBER; - function gateStatusDisposition( - status, - expectedPendingId, - currentRunUrl, - targetPrefix - ) { - if (!/^\d+$/.test(String(expectedPendingId || '')) || - !status || !/^\d+$/.test(String(status.id || ''))) { - return 'fail_closed'; - } - const target = String( - (status && status.target_url) || '' - ); - const expectedId = BigInt(String(expectedPendingId)); - const statusId = BigInt(String(status.id)); - function validRunTarget(targetUrl) { - const value = String(targetUrl || ''); - if (!value.startsWith(targetPrefix)) { - return false; - } - const suffix = value.slice(targetPrefix.length); - return /^\d+$/.test(suffix); - } - function statusOwnerId(candidate) { - if (candidate.state === 'pending') { - return BigInt(String(candidate.id)); - } - const owner = String(candidate.description || '').match( - /^gate-owner:(\d+)(?:\s|$)/ - ); - return owner ? BigInt(owner[1]) : null; - } - if (!validRunTarget(currentRunUrl) || - !validRunTarget(target)) { - return 'fail_closed'; - } - const ownerId = statusOwnerId(status); - if (ownerId === null) { - return 'fail_closed'; - } - if (ownerId === expectedId && target === currentRunUrl) { - if (statusId === expectedId && - status.state === 'pending') { - return 'current_pending'; - } - if (['failure', 'error'].includes(status.state)) { - return 'already_failed'; - } - if (status.state === 'success') { - return 'already_succeeded'; - } - return 'fail_closed'; - } - if (target === currentRunUrl) { - return 'fail_closed'; - } - if (ownerId > expectedId) { - return 'successor'; - } - if (ownerId < expectedId) { - return 'predecessor'; - } - return 'fail_closed'; - } - function mayFinalizeFailure(disposition) { - return new Set([ - 'current_pending', - 'predecessor', - 'already_succeeded' - ]).has(disposition); - } - const statuses = await github.paginate( - github.rest.repos.listCommitStatusesForRef, - { - owner, - repo, - ref: process.env.HEAD_SHA, - per_page: 100 - } - ); - const latest = statuses - .filter(status => - status.context === gateContext - ) - .sort((left, right) => - Date.parse(right.created_at) - Date.parse(left.created_at) || - right.id - left.id - )[0]; - const disposition = gateStatusDisposition( - latest, - process.env.PENDING_STATUS_ID, - runUrl, - runUrlPrefix - ); - if (disposition === 'successor') { - core.warning('A successor owns the gate status'); - return; - } - if (disposition === 'already_failed') { - return; - } - if (!mayFinalizeFailure(disposition)) { - core.warning('No proven lease for failure publication'); - return; - } - if (disposition === 'already_succeeded') { - core.warning( - 'Overriding success after failed gate publication' - ); - } - const description = ( - 'gate-owner:' + process.env.PENDING_STATUS_ID + ' ' + - 'gate publication failed' - ).slice(0, 140); - await github.rest.repos.createCommitStatus({ - owner, - repo, - sha: process.env.HEAD_SHA, - state: 'failure', - context: gateContext, - description, - target_url: runUrl - }); - - - name: Enforce verdict - if: >- - always() && - steps.resolve.outputs.pr_number != '' && - (steps.collect.outcome != 'success' || - steps.gate.outputs.exit_code != '0' || - steps.upload.outcome != 'success' || - steps.publish.outcome != 'success') - run: exit 1 diff --git a/docs/agent-completion-truth-gate.md b/docs/agent-completion-truth-gate.md deleted file mode 100644 index da14e3e4c..000000000 --- a/docs/agent-completion-truth-gate.md +++ /dev/null @@ -1,141 +0,0 @@ -# Agent Completion Truth Gate - -The truth gate converts repository evidence into one deterministic verdict. It never asks an LLM whether work is complete and it never approves, merges, closes, or reopens anything. - -## Required enforcement rollout - -`agent-completion/truth-gate` remains advisory and must not be added as a required status. The separate **Agent completion enforcement** workflow is the required, head-bound Check run. It verifies an exact-head machine-readable report published through a dedicated GitHub App and rejects missing, stale, edited, deleted, ambiguous, or self-published agent evidence. The protected policy is `.github/agent-lock/trusted-publishers.json`. While its trusted-publisher and trusted-actor allowlists are empty (the App unprovisioned), the **Agent completion enforcement** Check reports **neutral (advisory)** rather than blocking — a permanently red gate on every PR trains reviewers to ignore CI and hides real failures. The gate returns to fail-closed only once all three allowlists are populated (the App provisioned) *and* the Check is added to branch-protection required checks. Genuine policy violations — untrusted publisher, head-SHA or PR-number mismatch, and, once provisioned, a missing report — still fail closed regardless. Custom roles are fail-closed. - -The trusted publisher must bind report data to PR number, full head SHA, delivery/run identity, trusted label authorization, trusted human exemption (when applicable), append-only agent events, and per-path passed/failed/error counts. The required verifier never executes PR code. Repository rules must require **Agent completion enforcement**, one independent approval, and resolved conversations. They must not require the advisory custom status. - -## Enforcement lifecycle - -Before delegation, create the task with the Agent task issue form. Agent login, run ID, objective, acceptance criteria, exact file scope, allowed extras, and focused test paths are the intent contract. Unrestricted scope is intentionally unavailable in the form until #874 provisions the protected `scope-unrestricted-approved` label and its authorization policy; any hand-authored unrestricted request without that label fails closed. - -When a complete agent task is opened or first labeled by an actor with a standard `triage`-or-higher role (or a custom role that GitHub reports at legacy `write` level), the default-branch workflow writes a github-actions[bot] comment containing the SHA-256 digest of the normalized issue body and the unrestricted-scope approval state. The same live permission lookup applies to both event paths; issue author association alone is not trusted. GitHub's collaborator response cannot distinguish a custom triage-derived role from a custom read-derived role, so custom roles reported at legacy `read` fail closed and a standard triage-or-higher user must relabel. The collector requires that snapshot and requires it to predate the PR. Any later issue edit or label transition appends a bot-owned invalidation marker. Treating every queued edit/label event as invalidating makes GitHub concurrency coalescing lossless: an event that replaces a pending invalidation is itself an invalidation. The trusted marker comment dispatches immediate reevaluation, and the scheduled scanner also blocks permanently even if the original body or label state is restored. Existing tasks must be labeled again by a trusted user to create their one-time snapshot before dispatch. Create a new task for any post-snapshot edit; do not broaden a dispatched task in place. - -Agent pull requests link exactly one task with a closing keyword and include the agent-lock-manifest comment shown in the PR template. GitHub's authoritative closingIssuesReferences, the textual link, and the manifest must agree. The manifest login and run ID must exactly match the snapshotted issue. The declared agent publishes structured result evidence containing that run ID and the current PR head SHA; legacy unstructured readiness is never sufficient by itself. - -The trusted workflow runs on pull-request changes and serializes all evaluation for one PR. Pull-request and linked-issue contract changes, declared-agent result comments, and completed CI runs dispatch into that same queue; ordinary discussion comments do not. Every 15 minutes one serialized scanner compares the last owning pending lease with declared-agent result comments, exact-head CI, current applicability, the three-way issue link, frozen intent, mutable PR policy (draft/title/manifest identity and the `copilot-rabbit` label), and current review evidence. An active pending run is left alone, while a completed, missing, or hour-stale owning run is finalized directly as failure without acquiring another lease. Unchanged terminal results produce no new status. Scanner API uncertainty retries without dispatching. Issue events dispatch immediately only when the sender's current repository role is standard `triage` or higher, or GitHub reports its legacy permission at `write` or higher; known agent identities do not bypass that lookup, and unverifiable or custom-read-level events fall back to the scheduled scan. Review decisions, applicability and policy changes, and thread resolution therefore converge within 15 minutes. It checks out only the default-branch gate and never executes code from the PR head. - -The workflow publishes all of the following: - -- per-PR commit status `agent-completion/truth-gate/pr-`; -- one updatable PR comment with marker agent-completion-truth-gate:v1; -- an Actions summary; -- gate-input.json and gate-verdict.json artifacts. - -Even in the normal trust model—agents cannot write default-branch workflows or forge repository statuses—the custom status emitted here remains advisory. Follow-up #874 must bind evaluation to an independently head-bound required workflow or check before branch protection or a repository ruleset treats the result as merge enforcement. That ruleset must also require the repository's Copilot review, at least one approving review, and conversation resolution. The gate itself requires the maintainer-applied `copilot-rabbit` label, a non-dismissed Copilot review bound to the current head, every AI review thread resolved (including outdated threads), and committed focused unit tests. It binds to the exact-head trusted CI run, requires its `test` job to succeed, and requires that job's verbose pytest log to report at least one passing test for every declared path; an absent, deselected, or all-skipped path blocks. Human approval alone cannot satisfy those signals. Native review/conversation rules close the window between a new review comment and the scheduled refresh. - -Each serialized run first posts a pending status tied to its Actions run. It uploads evidence and updates the PR comment before publishing a terminal status. A compare-and-swap check rejects superseded publication; an always-running finalizer turns publication failures into a failure status only when the run proves it owns the pending lease (or supersedes a predecessor). Unknown ownership fails the workflow without touching the shared status, and the scheduled scanner supplies eventual fail-closed recovery. A recovered infrastructure/publication failure remains failed until a new evidence event or a maintainer manually dispatches the workflow; the scanner does not churn unchanged evidence merely to retry it. GitHub permits at most 1,000 statuses for one SHA and context, so the workflow refuses to acquire a new lease at 998 existing per-PR gate statuses. The scanner may use a remaining slot to fail a proven stuck pending lease or invalidate a successful terminal result after evidence changes, but it never starts another full evaluation at that threshold; the operator must then push a new head or complete #874's Checks/App migration. If pending-status creation fails on a reused head, a previous successful status can remain visible until recovery; overwriting it without a proven lease could instead clobber a successor. This transient stale-green window is why this status is advisory until #874 supplies independently head-bound enforcement. Actions are pinned to full commit SHAs. Only refresh-dispatch jobs receive actions: write. - -Per-path focused-test proof currently parses terminal pytest outcomes from the exact-head trusted CI `test` job log. It handles both same-line verbose outcomes and node IDs followed by live-log output and a standalone terminal outcome. Exact path prefixes, whole-line terminal matching, test-node resets, and session-boundary resets prevent unrelated output from inheriting a prior test. A same-line candidate is deferred across blank lines so a following live-log header can disambiguate a parameter ID that resembles an outcome. Code executed by pytest can still deliberately forge console lines, so the required independent review remains part of the trust boundary. Follow-up #874 must move per-path evidence to a machine-readable report emitted and published by the protected CI workflow before this signal is treated as hostile-agent-resistant. - -If an agent has repository-write credentials that can create Actions workflows or post statuses/comments, github-actions[bot] and a status-context string are not independent provenance. The collector also sees only the current comment bodies; an agent that can edit or delete its own result comment can erase contradictory self-reported history. In that threat model, keep this workflow advisory until snapshot, append-only result evidence, and check publication move to a dedicated GitHub App (or an organization ruleset-required trusted workflow) and bind the required check to that identity. - -## Applicability - -The gate applies when any of these signals identify agent work: - -- a known agent bot authored the PR; -- the branch starts with agent/, claude/, codex/, copilot/, or jules/; -- the PR or linked issue has agent, agent-task, or mcp/agent; -- the PR contains an agent-lock-manifest comment. - -Dependabot is exempt. Other human-authored PRs receive not_applicable. - -## Input schema - -The CLI accepts one JSON object: - - { - "policy": { - "applicable": true, - "agent_login": "google-labs-jules[bot]", - "run_id": "provider-run-id", - "head_sha": "1111111111111111111111111111111111111111" - }, - "issue": { - "number": 802, - "description": "Required objective", - "acceptance_criteria": ["Observable criterion"], - "declared_files": ["src/example.py", "tests/unit/test_example.py"], - "allowed_extra_files": ["docs/example.md"], - "scope_unrestricted": false - }, - "pull_request": { - "number": 813, - "changed_files": ["src/example.py", "tests/unit/test_example.py"], - "present_changed_files": ["src/example.py", "tests/unit/test_example.py"], - "merged": false, - "draft": false, - "title_valid": true, - "required_checks_passed": true, - "post_merge_checks_passed": false - }, - "events": [ - {"kind": "artifact_ready", "sequence": 1, "author": "google-labs-jules[bot]", "run_id": "provider-run-id", "head_sha": "1111111111111111111111111111111111111111"}, - {"kind": "error", "sequence": 2, "author": "google-labs-jules[bot]", "run_id": "provider-run-id", "head_sha": "1111111111111111111111111111111111111111"} - ], - "reviews": [ - {"blocking": true, "resolved": false, "source": "thread-id"} - ], - "evidence": { - "behavior_changed_files": ["src/example.py"], - "focused_test_files": ["tests/unit/test_example.py"], - "focused_test_results": { - "tests/unit/test_example.py": {"passed": 1, "failed": 0, "errors": 0, "skipped": 0, "xfailed": 0, "xpassed": 0} - }, - "copilot_current_head_reviewed": true, - "copilot_rabbit_label": true - }, - "collection_errors": [] - } - -Run it with: - - python3 scripts/ci/agent_completion_gate.py gate-input.json - -The CLI prints JSON to standard output. Exit status 1 means blocked; ready, completed, and not_applicable return 0. -An applicable policy must explicitly provide `applicable: true`, a nonblank `agent_login` and `run_id`, a 40-character hexadecimal `head_sha`, and a positive JSON integer `issue.number`. Object sections must be JSON objects; every event requires an `author` exactly equal to `policy.agent_login`, a supported `kind`, and an integer `sequence`, with any supplied run/head metadata strictly typed; and every review requires a nonblank source plus JSON booleans for `blocking` and `resolved`. `changed_files` contains every touched current and previous path, including removals, while `present_changed_files` contains only nonremoved current paths and must be a subset of `changed_files`. Every declared file is a required deliverable that must remain present at its exact path on the PR head, while allowed-extra files remain optional; for a rename, declare the destination and allowlist the source. Supplied criteria and path entries must contain non-whitespace text. Focused-test results must contain exactly one result object per declared focused-test path, with nonnegative integer counts for passed, failed, errors, skipped, xfailed, and xpassed; each path needs at least one pass and no failure or error. `copilot_current_head_reviewed` means a non-dismissed submitted Copilot review (`APPROVED`, `COMMENTED`, or `CHANGES_REQUESTED`) whose `commit_id` equals the current PR head; it is evidence of a current-head review, not native GitHub approval. Missing evidence sections, falsey containers such as `[]`, and stringified booleans are invalid payloads; they never inherit defaults. A policy containing only `applicable: false` remains a valid non-agent exemption. - -## Verdict schema - -Every result has the same shape: - - { - "verdict": "blocked", - "reasons": ["scope_drift"], - "details": { - "undeclared_files": ["package-lock.json"], - "identity_projection": { - "issue_number": 870, - "agent_login": "example-agent[bot]", - "run_id": "provider-run-id" - } - } - } - -Every evaluated applicable result carries the selected issue/agent/run identity so the scheduled fallback can detect a same-head contract switch even when both old and new contracts are otherwise valid. Hard-unknown infrastructure verdicts are not used as a comparison baseline. - -Verdict meanings: - -- blocked: one or more rules failed. -- ready: pre-merge evidence agrees, but the PR is not complete. -- completed: the PR is merged and required post-merge checks passed. -- not_applicable: policy explicitly exempted the PR. - -## Fail-closed rules - -The gate blocks a missing, late, or changed intent snapshot; agent/run/head identity mismatches; blank intent; missing acceptance criteria; missing scope; an empty PR diff; omitted, deleted, or renamed-away declared files; undeclared paths (including a rename's previous path); unapproved unrestricted scope; failed evidence collection; missing current-run output; current-run agent errors; contradictory readiness/completion and error events; a missing current-head Copilot review or `copilot-rabbit` label; unresolved AI or other blocking reviews; failed required checks; draft or invalidly titled PRs; missing, deleted, deselected, all-skipped, or failing focused Python unit-test evidence; and merged work without passing post-merge checks on the merge SHA. - -Artifact ready is not completion. A Ready for review comment followed by an error is agent_run_failed. Generic green CI never overrides an unresolved review. An unmerged PR can be ready, but it can never be completed. - -The deterministic evaluator blocks, and the workflow run fails, if checkout, evidence collection, evaluation, evidence upload, comment publication, or final status publication fails. Every terminal status carries the immutable pending-status ID [acquired lease]; status handoff compares those owner IDs so a newer lease overrides a late predecessor while an older lease yields to a successor. Unknown or malformed ownership fails the run without publishing from an unproven lease. Because an older success can remain visible during that recovery window, this custom status alone is not fail-closed merge enforcement. The snapshot assumes repository write access and the default branch are trusted; organizations that delegate repository-write credentials to agents should move snapshot creation behind a protected environment or an independently authenticated GitHub App. - -## Technical Constraints - -- **Snapshot creation is label-event-only**: Snapshot comments are generated exclusively during issue label actions to guarantee security boundaries and ensure metadata stability. -- **Recursion protection**: Status checks and gate evaluation does not recursively trigger `issue_comment` events to prevent infinite automated loop cycles. -- **Trace parameters**: Resolve-time, collection-time, and publication-time PR base and head SHAs are captured explicitly to prevent race conditions during concurrent runs. -- **Commit comparisons**: Every verdict includes an immutable resolved base/head commit comparison to guarantee that evaluations apply exactly to the proposed diff. diff --git a/scripts/ci/agent_completion_enforcement.py b/scripts/ci/agent_completion_enforcement.py deleted file mode 100644 index f746c6f20..000000000 --- a/scripts/ci/agent_completion_enforcement.py +++ /dev/null @@ -1,90 +0,0 @@ -"""Fail-closed verifier for the independently published agent-lock report. - -The verifier intentionally accepts only a report published by a configured GitHub -App. It is designed to run from a default-branch pull_request_target workflow -that creates the required Check run directly on the pull request head SHA. -""" - -from __future__ import annotations - -import json -import re -import sys -from pathlib import Path -from typing import Any - - -SHA = re.compile(r"^[0-9a-f]{40}$", re.IGNORECASE) - - -def verdict(reason: str, **details: Any) -> dict[str, Any]: - return {"conclusion": "failure", "reason": reason, "details": details} - - -def verify(payload: Any, policy: Any, head_sha: str, pull_number: int) -> dict[str, Any]: - if not isinstance(payload, dict) or not isinstance(policy, dict): - return verdict("invalid_payload") - if not SHA.fullmatch(head_sha) or type(pull_number) is not int or pull_number < 1: - return verdict("invalid_invocation") - if policy.get("custom_role_policy") != "fail_closed": - return verdict("invalid_custom_role_policy") - apps = policy.get("trusted_check_app_slugs") - labels = policy.get("trusted_label_actors") - exemptions = policy.get("trusted_human_exemption_actors") - if not all(isinstance(value, list) and all(isinstance(item, str) and item for item in value) - for value in (apps, labels, exemptions)): - return verdict("invalid_trust_policy") - if not apps or not labels or not exemptions: - return verdict("trust_policy_unprovisioned") - required = {"schema_version", "pull_number", "head_sha", "publisher", "applicability", "label_authorization", "focused_tests", "agent_events"} - if set(payload) != required or payload.get("schema_version") != 1: - return verdict("invalid_report_schema") - if payload.get("pull_number") != pull_number or str(payload.get("head_sha", "")).lower() != head_sha.lower(): - return verdict("report_identity_mismatch") - publisher = payload.get("publisher") - if not isinstance(publisher, dict) or publisher.get("app_slug") not in apps or not isinstance(publisher.get("delivery_id"), str) or not publisher["delivery_id"]: - return verdict("untrusted_or_non_append_only_publication") - applicability = payload.get("applicability") - if not isinstance(applicability, dict) or applicability.get("state") not in {"agent", "human_exempt"}: - return verdict("missing_trusted_applicability") - if applicability["state"] == "human_exempt" and applicability.get("attested_by") not in exemptions: - return verdict("untrusted_human_exemption") - label = payload.get("label_authorization") - if not isinstance(label, dict) or label.get("copilot_rabbit") is not True or label.get("applied_by") not in labels: - return verdict("untrusted_label_authorization") - focused = payload.get("focused_tests") - if not isinstance(focused, dict) or focused.get("producer") != publisher.get("app_slug"): - return verdict("untrusted_focused_test_report") - paths = focused.get("paths") - if not isinstance(paths, dict) or not paths: - return verdict("missing_focused_test_report") - for path, result in paths.items(): - if not isinstance(path, str) or not path or not isinstance(result, dict): - return verdict("invalid_focused_test_report") - if type(result.get("passed")) is not int or result["passed"] < 1: - return verdict("focused_test_failed", path=path) - if any(type(result.get(key)) is not int or result[key] != 0 for key in ("failed", "errors")): - return verdict("focused_test_failed", path=path) - events = payload.get("agent_events") - if not isinstance(events, list) or not events: - return verdict("missing_append_only_agent_events") - if any(not isinstance(event, dict) or event.get("channel") != publisher.get("app_slug") for event in events): - return verdict("untrusted_or_mutable_agent_events") - if any(event.get("kind") == "error" for event in events): - return verdict("agent_run_failed") - if not any(event.get("kind") in {"artifact_ready", "completed"} for event in events): - return verdict("missing_agent_success_event") - return {"conclusion": "success", "reason": "verified", "details": {"head_sha": head_sha.lower(), "pull_number": pull_number}} - - -def main() -> int: - if len(sys.argv) != 5: - raise SystemExit("usage: verifier REPORT POLICY HEAD_SHA PULL_NUMBER") - report, policy, head, pull = sys.argv[1:] - result = verify(json.loads(Path(report).read_text()), json.loads(Path(policy).read_text()), head, int(pull)) - print(json.dumps(result, sort_keys=True)) - return 0 if result["conclusion"] == "success" else 1 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/scripts/ci/agent_completion_gate.py b/scripts/ci/agent_completion_gate.py deleted file mode 100644 index 80c3ee41b..000000000 --- a/scripts/ci/agent_completion_gate.py +++ /dev/null @@ -1,443 +0,0 @@ -"""Evidence-based completion verdicts for autonomous agent work.""" - -import argparse -import json -import re -import sys -from pathlib import Path -from typing import Any, Dict, List - - -def _collection_errors(payload: Any) -> List[str]: - """Return the non-empty collection errors recorded by evidence collection.""" - - if not isinstance(payload, dict): - return [] - raw = payload.get("collection_errors") - if not isinstance(raw, list): - return [] - return [str(error) for error in raw if str(error).strip()] - - -def _invalid_payload(payload: Any, invalid_fields: List[str]) -> Dict[str, Any]: - """Build a fail-closed ``invalid_payload`` verdict with collector diagnostics. - - Every ``invalid_payload`` return routes through here so the collector's own - ``collection_errors`` are surfaced consistently — not only on the late - field-validation path. ``verdict`` and ``reasons`` stay byte-identical for - every input; the helper only enriches ``details``, keeping the gate - fail-closed while telling the author what to fix. - """ - - details: Dict[str, Any] = {} - if invalid_fields: - details["invalid_fields"] = sorted(set(invalid_fields)) - surfaced_errors = _collection_errors(payload) - if surfaced_errors: - details["collection_errors"] = surfaced_errors - return { - "verdict": "blocked", - "reasons": ["invalid_payload"], - "details": details, - } - - -def evaluate(payload: Any) -> Dict[str, Any]: - """Evaluate agent execution evidence and return a fail-closed verdict.""" - - if not isinstance(payload, dict): - return _invalid_payload(payload, []) - - policy = payload.get("policy") - if not isinstance(policy, dict): - return _invalid_payload(payload, ["policy"]) - if "applicable" not in policy or type(policy["applicable"]) is not bool: - return _invalid_payload(payload, ["policy.applicable"]) - if policy.get("applicable") is False: - return {"verdict": "not_applicable", "reasons": [], "details": {}} - - issue = payload.get("issue") - pull_request = payload.get("pull_request") - evidence = payload.get("evidence") - events = payload.get("events") - reviews = payload.get("reviews") - collection_errors = payload.get("collection_errors") - invalid_fields = [] - - for field in ("agent_login", "run_id"): - value = policy.get(field) - if not isinstance(value, str) or not value.strip(): - invalid_fields.append("policy." + field) - head_sha = policy.get("head_sha") - if not isinstance(head_sha, str) or not re.fullmatch( - r"[a-fA-F0-9]{40}", head_sha - ): - invalid_fields.append("policy.head_sha") - - for field, value in ( - ("issue", issue), - ("pull_request", pull_request), - ("evidence", evidence), - ): - if not isinstance(value, dict): - invalid_fields.append(field) - if isinstance(issue, dict) and ( - type(issue.get("number")) is not int or issue.get("number") <= 0 - ): - invalid_fields.append("issue.number") - for field, value in ( - ("events", events), - ("reviews", reviews), - ("collection_errors", collection_errors), - ): - if not isinstance(value, list): - invalid_fields.append(field) - - if isinstance(events, list): - expected_agent_login = str(policy.get("agent_login") or "").strip() - for index, event in enumerate(events): - if not isinstance(event, dict): - invalid_fields.append("events") - continue - event_kind = event.get("kind") - if type(event_kind) is not str or event_kind not in { - "artifact_ready", - "completed", - "error", - }: - invalid_fields.append(f"events[{index}].kind") - if type(event.get("sequence")) is not int: - invalid_fields.append(f"events[{index}].sequence") - run_id = event.get("run_id") - if run_id is not None and ( - not isinstance(run_id, str) or not run_id.strip() - ): - invalid_fields.append(f"events[{index}].run_id") - event_head_sha = event.get("head_sha") - if event_head_sha is not None and ( - not isinstance(event_head_sha, str) - or not re.fullmatch(r"[a-fA-F0-9]{40}", event_head_sha) - ): - invalid_fields.append(f"events[{index}].head_sha") - if "comment_id" in event and type(event["comment_id"]) is not int: - invalid_fields.append(f"events[{index}].comment_id") - author = event.get("author") - if ( - not isinstance(author, str) - or not author.strip() - or author != expected_agent_login - ): - invalid_fields.append(f"events[{index}].author") - if "raw_body" in event and not isinstance(event["raw_body"], str): - invalid_fields.append(f"events[{index}].raw_body") - - if isinstance(reviews, list): - for index, review in enumerate(reviews): - if not isinstance(review, dict): - invalid_fields.append("reviews") - continue - for field in ("blocking", "resolved"): - if type(review.get(field)) is not bool: - invalid_fields.append(f"reviews[{index}].{field}") - source = review.get("source") - if not isinstance(source, str) or not source.strip(): - invalid_fields.append(f"reviews[{index}].source") - - if isinstance(collection_errors, list) and not all( - isinstance(error, str) and error.strip() - for error in collection_errors - ): - invalid_fields.append("collection_errors") - - if isinstance(issue, dict): - if issue.get("description") is not None and not isinstance( - issue.get("description"), str - ): - invalid_fields.append("issue.description") - for field in ("acceptance_criteria", "declared_files"): - value = issue.get(field) - if not isinstance(value, list): - invalid_fields.append("issue." + field) - elif isinstance(value, list) and not all( - isinstance(item, str) and bool(item.strip()) for item in value - ): - invalid_fields.append("issue." + field) - allowed_extra_files = issue.get("allowed_extra_files", []) - if not isinstance(allowed_extra_files, list) or ( - isinstance(allowed_extra_files, list) - and not all( - isinstance(item, str) and bool(item.strip()) - for item in allowed_extra_files - ) - ): - invalid_fields.append("issue.allowed_extra_files") - if type(issue.get("scope_unrestricted")) is not bool: - invalid_fields.append("issue.scope_unrestricted") - - if isinstance(pull_request, dict): - for field in ("changed_files", "present_changed_files"): - value = pull_request.get(field) - if not isinstance(value, list): - invalid_fields.append("pull_request." + field) - elif not all( - isinstance(item, str) and bool(item.strip()) for item in value - ): - invalid_fields.append("pull_request." + field) - changed_files = pull_request.get("changed_files") - present_changed_files = pull_request.get("present_changed_files") - if ( - isinstance(changed_files, list) - and isinstance(present_changed_files, list) - and all(isinstance(item, str) for item in changed_files) - and all(isinstance(item, str) for item in present_changed_files) - and not set(present_changed_files).issubset(set(changed_files)) - ): - invalid_fields.append("pull_request.present_changed_files") - for field in ( - "merged", - "draft", - "title_valid", - "required_checks_passed", - "post_merge_checks_passed", - ): - if type(pull_request.get(field)) is not bool: - invalid_fields.append("pull_request." + field) - - if isinstance(evidence, dict): - for field in ("behavior_changed_files", "focused_test_files"): - value = evidence.get(field) - if not isinstance(value, list): - invalid_fields.append("evidence." + field) - elif isinstance(value, list) and not all( - isinstance(item, str) and bool(item.strip()) for item in value - ): - invalid_fields.append("evidence." + field) - focused_test_files = evidence.get("focused_test_files") - focused_test_results = evidence.get("focused_test_results") - result_fields = { - "passed", - "failed", - "errors", - "skipped", - "xfailed", - "xpassed", - } - invalid_focused_results = not isinstance(focused_test_results, dict) - if isinstance(focused_test_results, dict): - focused_files_valid = isinstance(focused_test_files, list) and all( - isinstance(path, str) and bool(path.strip()) - for path in focused_test_files - ) - if not focused_files_valid or set(focused_test_results) != set( - focused_test_files - ): - invalid_focused_results = True - for path, counts in focused_test_results.items(): - if not isinstance(path, str) or not path.strip(): - invalid_focused_results = True - continue - if not isinstance(counts, dict) or set(counts) != result_fields: - invalid_focused_results = True - continue - if any( - type(counts[field]) is not int or counts[field] < 0 - for field in result_fields - ): - invalid_focused_results = True - if invalid_focused_results: - invalid_fields.append("evidence.focused_test_results") - present_changed_files = ( - pull_request.get("present_changed_files") - if isinstance(pull_request, dict) - else None - ) - if ( - isinstance(focused_test_files, list) - and isinstance(present_changed_files, list) - and all(isinstance(path, str) for path in focused_test_files) - and all(isinstance(path, str) for path in present_changed_files) - and not set(focused_test_files).issubset(set(present_changed_files)) - ): - invalid_fields.append("evidence.focused_test_files") - for field in ( - "copilot_current_head_reviewed", - "copilot_rabbit_label", - ): - if type(evidence.get(field)) is not bool: - invalid_fields.append("evidence." + field) - if invalid_fields: - return _invalid_payload(payload, invalid_fields) - - reasons = [] - identity_projection = { - "issue_number": issue.get("number"), - "agent_login": str(policy.get("agent_login") or "").strip() or None, - "run_id": str(policy.get("run_id") or "").strip() or None, - } - details = {"identity_projection": identity_projection} - collection_errors = _collection_errors(payload) - if collection_errors: - reasons.append("evidence_collection_failed") - details["collection_errors"] = collection_errors - if not str(issue.get("description") or "").strip(): - reasons.append("blank_issue_description") - if not issue.get("acceptance_criteria"): - reasons.append("missing_acceptance_criteria") - if not issue.get("declared_files") and not issue.get("scope_unrestricted", False): - reasons.append("missing_declared_scope") - - if not issue.get("scope_unrestricted", False): - declared = set(issue.get("declared_files") or []) - declared.update(issue.get("allowed_extra_files") or []) - changed = set(pull_request.get("changed_files") or []) - undeclared = sorted(changed - declared) - if undeclared: - reasons.append("scope_drift") - details["undeclared_files"] = undeclared - - missing_declared = sorted( - set(issue.get("declared_files") or []) - - set(pull_request.get("present_changed_files") or []) - ) - if missing_declared: - reasons.append("missing_declared_files") - details["missing_declared_files"] = missing_declared - - if not pull_request.get("changed_files"): - reasons.append("empty_pr_diff") - - events = payload.get("events") or [] - active_run_id = str(policy.get("run_id") or "").strip() - if active_run_id: - active_head_sha = str(policy.get("head_sha") or "").strip().lower() - scoped_events = [ - event - for event in events - if str(event.get("run_id") or "").strip() == active_run_id - ] - current_events = [ - event - for event in scoped_events - if str(event.get("kind") or "").lower() == "error" - or not active_head_sha - or str(event.get("head_sha") or "").strip().lower() - == active_head_sha - ] - unscoped_errors = [ - event - for event in events - if not str(event.get("run_id") or "").strip() - and str(event.get("kind") or "").lower() == "error" - ] - scoped_error_exists = any( - str(event.get("kind") or "").lower() == "error" - for event in scoped_events - ) - error_evidence_exists = scoped_error_exists or bool(unscoped_errors) - legacy_positive_evidence = [ - event - for event in events - if error_evidence_exists - and not str(event.get("run_id") or "").strip() - and str(event.get("kind") or "").lower() - in {"artifact_ready", "completed"} - ] - events = current_events + unscoped_errors + legacy_positive_evidence - - terminal_kinds = { - str(event.get("kind") or "").lower() - for event in events - } - if not terminal_kinds.intersection({"artifact_ready", "completed"}): - reasons.append("missing_agent_result") - if "error" in terminal_kinds: - reasons.append("agent_run_failed") - if ( - "error" in terminal_kinds - and terminal_kinds.intersection({"artifact_ready", "completed"}) - ): - reasons.append("contradictory_terminal_events") - - unresolved_reviews = [ - str(review.get("source") or "unknown") - for review in (payload.get("reviews") or []) - if review.get("blocking") is True and review.get("resolved") is not True - ] - if unresolved_reviews: - reasons.append("unresolved_review") - details["unresolved_reviews"] = unresolved_reviews - - if evidence.get("copilot_current_head_reviewed") is not True: - reasons.append("missing_copilot_current_head_review") - if evidence.get("copilot_rabbit_label") is not True: - reasons.append("missing_copilot_rabbit_label") - - if pull_request.get("required_checks_passed") is not True: - reasons.append("required_checks_failed") - if pull_request.get("draft") is True: - reasons.append("draft_pr") - if pull_request.get("title_valid") is not True: - reasons.append("invalid_pr_title") - - if evidence.get("behavior_changed_files"): - if not evidence.get("focused_test_files"): - reasons.append("missing_test_evidence") - else: - focused_test_results = evidence.get("focused_test_results") or {} - failing_focused_tests = sorted( - path - for path in evidence.get("focused_test_files") or [] - if focused_test_results[path]["passed"] < 1 - or focused_test_results[path]["failed"] > 0 - or focused_test_results[path]["errors"] > 0 - ) - if failing_focused_tests: - reasons.append("focused_tests_failed") - details["focused_test_failures"] = failing_focused_tests - - if ( - pull_request.get("merged") is True - and pull_request.get("post_merge_checks_passed") is not True - ): - reasons.append("post_merge_checks_failed") - - if reasons: - return {"verdict": "blocked", "reasons": reasons, "details": details} - - if pull_request.get("merged") is True: - return {"verdict": "completed", "reasons": [], "details": details} - - return {"verdict": "ready", "reasons": [], "details": details} - - -def main(argv: Any = None) -> int: - """Evaluate one JSON evidence file and emit a machine-readable verdict.""" - - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("input", help="JSON evidence file, or - for stdin") - args = parser.parse_args(argv) - - try: - if args.input == "-": - payload = json.load(sys.stdin) - else: - payload = json.loads(Path(args.input).read_text(encoding="utf-8")) - result = evaluate(payload) - except json.JSONDecodeError as exc: - result = { - "verdict": "blocked", - "reasons": ["invalid_json"], - "details": {"error": str(exc)}, - } - except (OSError, UnicodeError) as exc: - result = { - "verdict": "blocked", - "reasons": ["input_read_failed"], - "details": {"error": str(exc)}, - } - print(json.dumps(result, sort_keys=True)) - return 1 if result["verdict"] == "blocked" else 0 - - -if __name__ == "__main__": - raise SystemExit(main()) diff --git a/tests/unit/test_agent_completion_enforcement.py b/tests/unit/test_agent_completion_enforcement.py deleted file mode 100644 index fc25281e4..000000000 --- a/tests/unit/test_agent_completion_enforcement.py +++ /dev/null @@ -1,31 +0,0 @@ -import unittest - -from scripts.ci.agent_completion_enforcement import verify - -HEAD = "a" * 40 -POLICY = {"custom_role_policy": "fail_closed", "trusted_check_app_slugs": ["agent-lock-trusted"], "trusted_label_actors": ["maintainer"], "trusted_human_exemption_actors": ["maintainer"]} - - -def report(): - return {"schema_version": 1, "pull_number": 9, "head_sha": HEAD, "publisher": {"app_slug": "agent-lock-trusted", "delivery_id": "delivery-1"}, "applicability": {"state": "agent"}, "label_authorization": {"copilot_rabbit": True, "applied_by": "maintainer"}, "focused_tests": {"producer": "agent-lock-trusted", "paths": {"tests/unit/test_x.py": {"passed": 1, "failed": 0, "errors": 0}}}, "agent_events": [{"channel": "agent-lock-trusted", "kind": "artifact_ready"}]} - - -class EnforcementTests(unittest.TestCase): - def test_accepts_head_bound_trusted_report(self): - self.assertEqual(verify(report(), POLICY, HEAD, 9)["conclusion"], "success") - - def test_unprovisioned_policy_blocks(self): - policy = dict(POLICY, trusted_check_app_slugs=[]) - self.assertEqual(verify(report(), policy, HEAD, 9)["reason"], "trust_policy_unprovisioned") - - def test_stale_report_blocks(self): - body = report(); body["head_sha"] = "b" * 40 - self.assertEqual(verify(body, POLICY, HEAD, 9)["reason"], "report_identity_mismatch") - - def test_error_cannot_be_erased(self): - body = report(); body["agent_events"].append({"channel": "agent-lock-trusted", "kind": "error"}) - self.assertEqual(verify(body, POLICY, HEAD, 9)["reason"], "agent_run_failed") - - def test_untrusted_label_blocks(self): - body = report(); body["label_authorization"]["applied_by"] = "agent" - self.assertEqual(verify(body, POLICY, HEAD, 9)["reason"], "untrusted_label_authorization") diff --git a/tests/unit/test_agent_completion_gate.py b/tests/unit/test_agent_completion_gate.py deleted file mode 100644 index c58715293..000000000 --- a/tests/unit/test_agent_completion_gate.py +++ /dev/null @@ -1,3634 +0,0 @@ -import importlib -import io -import json -import subprocess -import tempfile -import unittest -from contextlib import redirect_stdout -from pathlib import Path - - -def _gate_module(): - try: - return importlib.import_module( - "scripts.ci.agent_completion_gate" - ) - except ModuleNotFoundError: - raise AssertionError("completion gate is not implemented") from None - - -def _evaluate(payload): - return _gate_module().evaluate(payload) - - -def _valid_payload(): - run_id = "agent-run-123" - head_sha = "a" * 40 - return { - "issue": { - "number": 813, - "description": "Add a deterministic completion gate.", - "acceptance_criteria": ["Reject incomplete agent output."], - "declared_files": [ - "src/youtube_extension/agent_lock/completion_gate.py", - "tests/unit/test_agent_completion_gate.py", - ], - "scope_unrestricted": False, - }, - "pull_request": { - "changed_files": [ - "src/youtube_extension/agent_lock/completion_gate.py", - "tests/unit/test_agent_completion_gate.py", - ], - "present_changed_files": [ - "src/youtube_extension/agent_lock/completion_gate.py", - "tests/unit/test_agent_completion_gate.py", - ], - "merged": False, - "draft": False, - "title_valid": True, - "required_checks_passed": True, - "post_merge_checks_passed": False, - }, - "events": [ - { - "kind": "completed", - "sequence": 1, - "author": "example-agent[bot]", - "run_id": run_id, - "head_sha": head_sha, - } - ], - "reviews": [], - "evidence": { - "behavior_changed_files": [ - "src/youtube_extension/agent_lock/completion_gate.py" - ], - "focused_test_files": ["tests/unit/test_agent_completion_gate.py"], - "focused_test_results": { - "tests/unit/test_agent_completion_gate.py": { - "passed": 1, - "failed": 0, - "errors": 0, - "skipped": 0, - "xfailed": 0, - "xpassed": 0, - } - }, - "copilot_current_head_reviewed": True, - "copilot_rabbit_label": True, - }, - "collection_errors": [], - "policy": { - "applicable": True, - "agent_login": "example-agent[bot]", - "run_id": run_id, - "head_sha": head_sha, - }, - } - - -def _fixture(name): - test_root = Path(__file__).resolve().parent - fixture_root = test_root / "fixtures" - if not fixture_root.exists(): - fixture_root = test_root.parent / "fixtures" - return json.loads( - (fixture_root / "agent_completion" / name).read_text(encoding="utf-8") - ) - - -def _repo_root(): - for candidate in Path(__file__).resolve().parents: - if (candidate / "scripts" / "ci" / "agent_completion_gate.py").exists(): - return candidate - raise AssertionError("repository root not found") - - -def _javascript_functions(source, signature): - """Extract repeated JavaScript function declarations with balanced braces.""" - - functions = [] - cursor = 0 - while True: - start = source.find(signature, cursor) - if start < 0: - return functions - opening = source.index("{", start) - depth = 0 - for index in range(opening, len(source)): - if source[index] == "{": - depth += 1 - elif source[index] == "}": - depth -= 1 - if depth == 0: - functions.append(source[start:index + 1]) - cursor = index + 1 - break - else: - raise AssertionError(f"unclosed JavaScript function: {signature}") - - -def _github_script_bodies(workflow): - """Extract YAML literal bodies assigned to github-script's script input.""" - - lines = workflow.splitlines() - bodies = [] - for index, line in enumerate(lines): - if line.lstrip() != "script: |": - continue - key_indent = len(line) - len(line.lstrip()) - body_lines = [] - for candidate in lines[index + 1:]: - if candidate.strip(): - indent = len(candidate) - len(candidate.lstrip()) - if indent <= key_indent: - break - body_lines.append(candidate) - content_indents = [ - len(candidate) - len(candidate.lstrip()) - for candidate in body_lines - if candidate.strip() - ] - if content_indents: - content_indent = min(content_indents) - bodies.append("\n".join( - candidate[content_indent:] if candidate.strip() else "" - for candidate in body_lines - )) - return bodies - - -class CompletionGateTests(unittest.TestCase): - def test_blank_issue_description_is_blocked(self): - payload = _valid_payload() - payload["issue"]["description"] = " " - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("blank_issue_description", result["reasons"]) - - def test_missing_acceptance_criteria_is_blocked(self): - payload = _valid_payload() - payload["issue"]["acceptance_criteria"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_acceptance_criteria", result["reasons"]) - - def test_missing_declared_scope_is_blocked(self): - payload = _valid_payload() - payload["issue"]["declared_files"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_declared_scope", result["reasons"]) - - def test_pr_813_declared_two_files_but_changed_five_is_scope_drift(self): - payload = _valid_payload() - payload["issue"]["declared_files"] = [ - "src/agents/specialized/quality_agent.py", - "src/youtube_extension/backend/ai_code_generator.py", - ] - payload["pull_request"]["changed_files"] = [ - "apps/web/package.json", - "package-lock.json", - "src/agents/specialized/quality_agent.py", - "src/youtube_extension/backend/ai_code_generator.py", - "tests/unit/test_cloud_routes.py", - ] - payload["pull_request"]["present_changed_files"] = list( - payload["pull_request"]["changed_files"] - ) - payload["evidence"]["behavior_changed_files"] = [] - payload["evidence"]["focused_test_files"] = [] - payload["evidence"]["focused_test_results"] = {} - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("scope_drift", result["reasons"]) - self.assertEqual( - result["details"]["undeclared_files"], - [ - "apps/web/package.json", - "package-lock.json", - "tests/unit/test_cloud_routes.py", - ], - ) - - def test_declared_files_are_required_not_only_allowed(self): - payload = _valid_payload() - payload["pull_request"]["changed_files"] = [ - "tests/unit/test_agent_completion_gate.py" - ] - payload["pull_request"]["present_changed_files"] = [ - "tests/unit/test_agent_completion_gate.py" - ] - payload["evidence"]["behavior_changed_files"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_declared_files", result["reasons"]) - self.assertEqual( - result["details"]["missing_declared_files"], - ["src/youtube_extension/agent_lock/completion_gate.py"], - ) - - def test_deleted_or_renamed_declared_file_is_missing(self): - required = "src/youtube_extension/agent_lock/completion_gate.py" - test_path = "tests/unit/test_agent_completion_gate.py" - replacement = "src/youtube_extension/agent_lock/replacement.py" - cases = ( - ([required, test_path], [test_path], []), - ( - [required, replacement, test_path], - [replacement, test_path], - [replacement], - ), - ) - for changed, present, allowed_extras in cases: - with self.subTest(changed=changed, present=present): - payload = _valid_payload() - payload["pull_request"]["changed_files"] = changed - payload["pull_request"]["present_changed_files"] = present - payload["issue"]["allowed_extra_files"] = allowed_extras - payload["evidence"]["behavior_changed_files"] = present - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_declared_files", result["reasons"]) - self.assertEqual( - result["details"]["missing_declared_files"], - [required], - ) - self.assertNotIn("scope_drift", result["reasons"]) - - def test_rename_source_still_counts_toward_scope_drift(self): - payload = _valid_payload() - replacement = "src/youtube_extension/agent_lock/replacement.py" - payload["issue"]["declared_files"] = [ - replacement, - "tests/unit/test_agent_completion_gate.py", - ] - payload["pull_request"]["changed_files"] = [ - "src/youtube_extension/agent_lock/completion_gate.py", - replacement, - "tests/unit/test_agent_completion_gate.py", - ] - payload["pull_request"]["present_changed_files"] = [ - replacement, - "tests/unit/test_agent_completion_gate.py", - ] - payload["evidence"]["behavior_changed_files"] = [replacement] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("scope_drift", result["reasons"]) - self.assertEqual( - result["details"]["undeclared_files"], - ["src/youtube_extension/agent_lock/completion_gate.py"], - ) - - def test_declared_rename_with_allowed_previous_path_is_ready(self): - payload = _valid_payload() - previous = "src/youtube_extension/agent_lock/completion_gate.py" - replacement = "src/youtube_extension/agent_lock/replacement.py" - test_path = "tests/unit/test_agent_completion_gate.py" - payload["issue"]["declared_files"] = [replacement, test_path] - payload["issue"]["allowed_extra_files"] = [previous] - payload["pull_request"]["changed_files"] = [ - previous, - replacement, - test_path, - ] - payload["pull_request"]["present_changed_files"] = [ - replacement, - test_path, - ] - payload["evidence"]["behavior_changed_files"] = [replacement] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - self.assertEqual(result["reasons"], []) - - def test_present_changed_files_must_be_subset_of_all_changed_files(self): - payload = _valid_payload() - payload["pull_request"]["present_changed_files"].append( - "src/not-in-the-diff.py" - ) - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertIn( - "pull_request.present_changed_files", - result["details"]["invalid_fields"], - ) - - def test_present_changed_files_is_required(self): - payload = _valid_payload() - del payload["pull_request"]["present_changed_files"] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertIn( - "pull_request.present_changed_files", - result["details"]["invalid_fields"], - ) - - def test_allowed_extra_files_remain_optional(self): - payload = _valid_payload() - payload["issue"]["allowed_extra_files"] = ["docs/optional.md"] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - - def test_explicit_unrestricted_scope_allows_undeclared_files(self): - payload = _valid_payload() - payload["issue"]["declared_files"] = [] - payload["issue"]["scope_unrestricted"] = True - payload["pull_request"]["changed_files"].append("docs/extra.md") - payload["pull_request"]["present_changed_files"].append( - "docs/extra.md" - ) - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - self.assertNotIn("scope_drift", result["reasons"]) - - def test_unrestricted_scope_does_not_waive_declared_deliverables(self): - payload = _valid_payload() - payload["issue"]["scope_unrestricted"] = True - payload["pull_request"]["changed_files"] = [ - "tests/unit/test_agent_completion_gate.py", - "docs/extra.md", - ] - payload["pull_request"]["present_changed_files"] = list( - payload["pull_request"]["changed_files"] - ) - payload["evidence"]["behavior_changed_files"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_declared_files", result["reasons"]) - self.assertNotIn("scope_drift", result["reasons"]) - - def test_completion_followed_by_error_is_contradictory(self): - payload = _valid_payload() - payload["events"] = [ - { - "kind": "completed", - "sequence": 1, - "author": "example-agent[bot]", - "run_id": "agent-run-123", - "head_sha": "a" * 40, - }, - { - "kind": "error", - "sequence": 2, - "author": "example-agent[bot]", - "run_id": "agent-run-123", - "head_sha": "a" * 40, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("contradictory_terminal_events", result["reasons"]) - - def test_artifact_ready_followed_by_error_is_failed_not_completed(self): - payload = _valid_payload() - payload["events"] = [ - { - "kind": "artifact_ready", - "sequence": 1, - "author": "example-agent[bot]", - "run_id": "agent-run-123", - "head_sha": "a" * 40, - }, - { - "kind": "error", - "sequence": 2, - "author": "example-agent[bot]", - "run_id": "agent-run-123", - "head_sha": "a" * 40, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("agent_run_failed", result["reasons"]) - - def test_missing_agent_result_is_blocked(self): - payload = _valid_payload() - payload["events"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_agent_result", result["reasons"]) - - def test_successful_new_run_can_recover_from_an_old_run_error(self): - payload = _valid_payload() - payload["policy"]["run_id"] = "new-run" - payload["events"] = [ - { - "kind": "error", - "author": "example-agent[bot]", - "run_id": "old-run", - "head_sha": "a" * 40, - "sequence": 1, - }, - { - "kind": "completed", - "author": "example-agent[bot]", - "run_id": "new-run", - "head_sha": "a" * 40, - "sequence": 2, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - - def test_historical_success_cannot_satisfy_a_silent_current_run(self): - payload = _valid_payload() - payload["policy"]["run_id"] = "new-run" - payload["events"] = [ - { - "kind": "completed", - "author": "example-agent[bot]", - "run_id": "old-run", - "head_sha": "a" * 40, - "sequence": 1, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_agent_result", result["reasons"]) - - def test_unscoped_success_cannot_satisfy_a_correlated_run(self): - payload = _valid_payload() - payload["policy"]["run_id"] = "new-run" - payload["events"] = [ - { - "kind": "completed", - "sequence": 1, - "author": "example-agent[bot]", - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_agent_result", result["reasons"]) - - def test_success_from_an_old_head_cannot_satisfy_current_head(self): - payload = _valid_payload() - payload["policy"].update( - {"run_id": "new-run", "head_sha": "b" * 40} - ) - payload["events"] = [ - { - "kind": "completed", - "author": "example-agent[bot]", - "run_id": "new-run", - "head_sha": "a" * 40, - "sequence": 1, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_agent_result", result["reasons"]) - - def test_head_sha_comparison_is_case_insensitive(self): - payload = _valid_payload() - payload["policy"]["head_sha"] = "A" * 40 - payload["events"][0]["head_sha"] = "a" * 40 - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - - def test_legacy_ready_plus_correlated_error_is_contradictory(self): - payload = _valid_payload() - payload["policy"]["run_id"] = "1892762060881911102" - payload["events"] = [ - { - "kind": "artifact_ready", - "run_id": None, - "sequence": 1, - "author": "example-agent[bot]", - }, - { - "kind": "error", - "run_id": "1892762060881911102", - "sequence": 2, - "author": "example-agent[bot]", - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("contradictory_terminal_events", result["reasons"]) - - def test_legacy_ready_plus_unscoped_error_is_contradictory(self): - payload = _valid_payload() - payload["events"] = [ - { - "kind": "artifact_ready", - "sequence": 1, - "author": "example-agent[bot]", - }, - { - "kind": "error", - "sequence": 2, - "author": "example-agent[bot]", - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("agent_run_failed", result["reasons"]) - self.assertIn("contradictory_terminal_events", result["reasons"]) - - def test_unscoped_error_still_blocks_a_correlated_run(self): - payload = _valid_payload() - payload["policy"]["run_id"] = "new-run" - payload["events"] = [ - { - "kind": "error", - "sequence": 1, - "author": "example-agent[bot]", - }, - { - "kind": "completed", - "author": "example-agent[bot]", - "run_id": "new-run", - "head_sha": "a" * 40, - "sequence": 2, - }, - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("agent_run_failed", result["reasons"]) - - def test_green_ci_does_not_override_unresolved_blocking_review(self): - payload = _valid_payload() - payload["pull_request"]["required_checks_passed"] = True - payload["reviews"] = [ - { - "blocking": True, - "resolved": False, - "source": "discussion_r3599972900", - } - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("unresolved_review", result["reasons"]) - self.assertEqual( - result["details"]["unresolved_reviews"], - ["discussion_r3599972900"], - ) - - def test_agent_pr_requires_copilot_review_contract(self): - cases = ( - ( - "copilot_current_head_reviewed", - "missing_copilot_current_head_review", - ), - ("copilot_rabbit_label", "missing_copilot_rabbit_label"), - ) - for field, reason in cases: - with self.subTest(field=field): - payload = _valid_payload() - payload["evidence"][field] = False - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn(reason, result["reasons"]) - - def test_failed_required_checks_are_blocked(self): - payload = _valid_payload() - payload["pull_request"]["required_checks_passed"] = False - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("required_checks_failed", result["reasons"]) - - def test_behavior_change_without_focused_test_evidence_is_blocked(self): - payload = _valid_payload() - payload["evidence"]["focused_test_files"] = [] - payload["evidence"]["focused_test_results"] = {} - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("missing_test_evidence", result["reasons"]) - - def test_empty_pr_diff_cannot_be_ready(self): - payload = _valid_payload() - payload["pull_request"]["changed_files"] = [] - payload["pull_request"]["present_changed_files"] = [] - payload["evidence"]["behavior_changed_files"] = [] - payload["evidence"]["focused_test_files"] = [] - payload["evidence"]["focused_test_results"] = {} - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("empty_pr_diff", result["reasons"]) - - def test_focused_tests_must_pass(self): - for field, value in (("passed", 0), ("failed", 1), ("errors", 1)): - with self.subTest(field=field): - payload = _valid_payload() - results = payload["evidence"]["focused_test_results"] - results["tests/unit/test_agent_completion_gate.py"][field] = value - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("focused_tests_failed", result["reasons"]) - - def test_focused_test_results_require_exact_declared_paths(self): - for results in ( - {}, - { - "tests/unit/test_agent_completion_gate.py": { - "passed": 1, - "failed": 0, - "errors": 0, - "skipped": 0, - "xfailed": 0, - "xpassed": 0, - }, - "tests/unit/test_extra.py": { - "passed": 1, - "failed": 0, - "errors": 0, - "skipped": 0, - "xfailed": 0, - "xpassed": 0, - }, - }, - ): - with self.subTest(results=results): - payload = _valid_payload() - payload["evidence"]["focused_test_results"] = results - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "evidence.focused_test_results", - result["details"]["invalid_fields"], - ) - - def test_focused_test_path_must_be_present_in_the_pr(self): - payload = _valid_payload() - missing_path = "tests/unit/test_not_changed.py" - payload["evidence"]["focused_test_files"] = [missing_path] - payload["evidence"]["focused_test_results"] = { - missing_path: { - "passed": 1, - "failed": 0, - "errors": 0, - "skipped": 0, - "xfailed": 0, - "xpassed": 0, - } - } - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertIn( - "evidence.focused_test_files", - result["details"]["invalid_fields"], - ) - - def test_focused_test_result_counts_are_nonnegative_integers(self): - for count in (True, -1, 1.5, "1"): - with self.subTest(count=count): - payload = _valid_payload() - results = payload["evidence"]["focused_test_results"] - results["tests/unit/test_agent_completion_gate.py"][ - "passed" - ] = count - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "evidence.focused_test_results", - result["details"]["invalid_fields"], - ) - - def test_unmerged_pr_can_be_ready_but_never_completed(self): - payload = _valid_payload() - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "ready") - self.assertNotEqual(result["verdict"], "completed") - - def test_merged_pr_without_post_merge_checks_is_blocked(self): - payload = _valid_payload() - payload["pull_request"]["merged"] = True - payload["pull_request"]["post_merge_checks_passed"] = False - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("post_merge_checks_failed", result["reasons"]) - - def test_fully_verified_merged_work_is_completed(self): - payload = _valid_payload() - payload["pull_request"]["merged"] = True - payload["pull_request"]["post_merge_checks_passed"] = True - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "completed") - self.assertEqual(result["reasons"], []) - self.assertEqual( - result["details"]["identity_projection"], - { - "issue_number": 813, - "agent_login": "example-agent[bot]", - "run_id": "agent-run-123", - }, - ) - - def test_draft_agent_pr_is_blocked(self): - payload = _valid_payload() - payload["pull_request"]["draft"] = True - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("draft_pr", result["reasons"]) - - def test_invalid_pr_title_is_blocked(self): - payload = _valid_payload() - payload["pull_request"]["title_valid"] = False - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_pr_title", result["reasons"]) - - def test_non_agent_pr_is_not_applicable(self): - payload = _valid_payload() - payload["policy"]["applicable"] = False - - result = _evaluate(payload) - - self.assertEqual( - result, - {"verdict": "not_applicable", "reasons": [], "details": {}}, - ) - - def test_malformed_payload_fails_closed(self): - result = _evaluate([]) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - - def test_applicable_policy_requires_bound_identity(self): - cases = ( - ("applicable", None, "policy.applicable"), - ("agent_login", None, "policy.agent_login"), - ("run_id", None, "policy.run_id"), - ("head_sha", None, "policy.head_sha"), - ("agent_login", " ", "policy.agent_login"), - ("run_id", 123, "policy.run_id"), - ("head_sha", "not-a-sha", "policy.head_sha"), - ) - for field, value, invalid_field in cases: - with self.subTest(field=field, value=value): - payload = _valid_payload() - if value is None: - del payload["policy"][field] - else: - payload["policy"][field] = value - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - invalid_field, - result["details"]["invalid_fields"], - ) - - def test_explicit_false_policy_remains_not_applicable(self): - result = _evaluate({"policy": {"applicable": False}}) - - self.assertEqual( - result, - {"verdict": "not_applicable", "reasons": [], "details": {}}, - ) - - def test_event_fields_are_strictly_validated(self): - class UnhashableString(str): - __hash__ = None - - cases = ( - ("kind", "unknown", "events[0].kind"), - ("kind", [], "events[0].kind"), - ("kind", {}, "events[0].kind"), - ("kind", UnhashableString("completed"), "events[0].kind"), - ("sequence", "first", "events[0].sequence"), - ("run_id", 123, "events[0].run_id"), - ("head_sha", "short", "events[0].head_sha"), - ("comment_id", "123", "events[0].comment_id"), - ) - for field, value, invalid_field in cases: - with self.subTest(field=field): - payload = _valid_payload() - payload["events"][0][field] = value - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - invalid_field, - result["details"]["invalid_fields"], - ) - - def test_event_author_must_match_policy_agent(self): - cases = (None, "", "attacker[bot]", 123) - for author in cases: - with self.subTest(author=author): - payload = _valid_payload() - if author is None: - del payload["events"][0]["author"] - else: - payload["events"][0]["author"] = author - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "events[0].author", - result["details"]["invalid_fields"], - ) - - def test_missing_evidence_sections_are_invalid(self): - for field in ( - "issue", - "pull_request", - "events", - "reviews", - "evidence", - "collection_errors", - ): - with self.subTest(field=field): - payload = _valid_payload() - del payload[field] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn(field, result["details"]["invalid_fields"]) - - def test_malformed_nested_fields_return_a_verdict(self): - payload = _valid_payload() - payload["events"] = ["not-an-event"] - - try: - result = _evaluate(payload) - except (AttributeError, TypeError): - self.fail("malformed nested evidence raised instead of failing closed") - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - - def test_falsey_malformed_object_sections_fail_closed(self): - for field in ("policy", "issue", "pull_request", "evidence"): - with self.subTest(field=field): - payload = _valid_payload() - payload[field] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn(field, result["details"]["invalid_fields"]) - - def test_review_flags_must_be_booleans(self): - for field in ("blocking", "resolved"): - with self.subTest(field=field): - payload = _valid_payload() - payload["reviews"] = [ - {"blocking": True, "resolved": False, "source": "thread-1"} - ] - payload["reviews"][0][field] = "false" - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - f"reviews[0].{field}", - result["details"]["invalid_fields"], - ) - - def test_review_source_must_be_nonempty_text(self): - payload = _valid_payload() - payload["reviews"] = [ - {"blocking": True, "resolved": False, "source": ""} - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "reviews[0].source", - result["details"]["invalid_fields"], - ) - - def test_non_string_path_entries_return_a_verdict(self): - payload = _valid_payload() - payload["pull_request"]["changed_files"] = [{}] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "pull_request.changed_files", - result["details"]["invalid_fields"], - ) - - payload = _valid_payload() - payload["pull_request"]["present_changed_files"] = [{}] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "pull_request.present_changed_files", - result["details"]["invalid_fields"], - ) - - def test_blank_list_entries_fail_closed(self): - cases = ( - ("issue", "acceptance_criteria"), - ("issue", "declared_files"), - ("issue", "allowed_extra_files"), - ("pull_request", "changed_files"), - ("pull_request", "present_changed_files"), - ("evidence", "behavior_changed_files"), - ("evidence", "focused_test_files"), - ) - for section, field in cases: - with self.subTest(section=section, field=field): - payload = _valid_payload() - payload[section][field] = [" "] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - f"{section}.{field}", - result["details"]["invalid_fields"], - ) - - def test_unhashable_focused_test_path_fails_closed(self): - payload = _valid_payload() - payload["evidence"]["focused_test_files"] = [{}] - payload["evidence"]["focused_test_results"] = {} - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "evidence.focused_test_files", - result["details"]["invalid_fields"], - ) - self.assertIn( - "evidence.focused_test_results", - result["details"]["invalid_fields"], - ) - - def test_non_boolean_policy_fields_cannot_bypass_rules(self): - for section, field in ( - ("policy", "applicable"), - ("issue", "scope_unrestricted"), - ("pull_request", "draft"), - ("pull_request", "merged"), - ("pull_request", "title_valid"), - ("pull_request", "required_checks_passed"), - ("pull_request", "post_merge_checks_passed"), - ("evidence", "copilot_current_head_reviewed"), - ("evidence", "copilot_rabbit_label"), - ): - with self.subTest(section=section, field=field): - payload = _valid_payload() - payload[section][field] = "true" - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - f"{section}.{field}", - result["details"]["invalid_fields"], - ) - - def test_applicable_issue_number_must_be_a_positive_json_integer(self): - for value in ("813", 0, -1, True, None, [], {}): - with self.subTest(value=value): - payload = _valid_payload() - payload["issue"]["number"] = value - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_payload", result["reasons"]) - self.assertIn( - "issue.number", - result["details"]["invalid_fields"], - ) - - def test_evidence_collection_errors_fail_closed(self): - payload = _valid_payload() - payload["collection_errors"] = ["invalid_agent_lock_manifest"] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertIn("evidence_collection_failed", result["reasons"]) - self.assertEqual( - result["details"]["collection_errors"], - ["invalid_agent_lock_manifest"], - ) - - def test_pr_813_fixture_is_blocked_for_every_expected_reason(self): - result = _evaluate(_fixture("pr_813.json")) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual( - set(result["reasons"]), - { - "blank_issue_description", - "missing_acceptance_criteria", - "scope_drift", - "agent_run_failed", - "contradictory_terminal_events", - "unresolved_review", - "draft_pr", - "invalid_pr_title", - "missing_test_evidence", - "missing_copilot_current_head_review", - "missing_copilot_rabbit_label", - }, - ) - self.assertEqual( - result["details"]["undeclared_files"], - [ - "apps/web/package.json", - "package-lock.json", - "tests/unit/test_cloud_routes.py", - ], - ) - - def test_cli_prints_json_and_returns_nonzero_for_blocked_input(self): - module = _gate_module() - if not hasattr(module, "main"): - self.fail("completion gate CLI is not implemented") - payload = _valid_payload() - payload["issue"]["description"] = "" - with tempfile.NamedTemporaryFile("w", encoding="utf-8") as handle: - json.dump(payload, handle) - handle.flush() - output = io.StringIO() - with redirect_stdout(output): - exit_code = module.main([handle.name]) - - self.assertEqual(exit_code, 1) - self.assertEqual(json.loads(output.getvalue())["verdict"], "blocked") - - def test_cli_fails_closed_for_invalid_json(self): - module = _gate_module() - with tempfile.NamedTemporaryFile("w", encoding="utf-8") as handle: - handle.write("{not-json") - handle.flush() - output = io.StringIO() - try: - with redirect_stdout(output): - exit_code = module.main([handle.name]) - except json.JSONDecodeError: - self.fail("CLI raised instead of returning a fail-closed verdict") - - result = json.loads(output.getvalue()) - self.assertEqual(exit_code, 1) - self.assertEqual(result["verdict"], "blocked") - self.assertIn("invalid_json", result["reasons"]) - - def test_cli_fails_closed_when_input_cannot_be_read(self): - module = _gate_module() - output = io.StringIO() - missing = str(Path(tempfile.gettempdir()) / "agent-lock-missing.json") - - try: - with redirect_stdout(output): - exit_code = module.main([missing]) - except OSError: - self.fail("CLI raised instead of returning an input-read verdict") - - result = json.loads(output.getvalue()) - self.assertEqual(exit_code, 1) - self.assertEqual(result["verdict"], "blocked") - self.assertIn("input_read_failed", result["reasons"]) - - def test_cli_returns_zero_for_ready_input(self): - module = _gate_module() - with tempfile.NamedTemporaryFile("w", encoding="utf-8") as handle: - json.dump(_valid_payload(), handle) - handle.flush() - output = io.StringIO() - with redirect_stdout(output): - exit_code = module.main([handle.name]) - - self.assertEqual(exit_code, 0) - self.assertEqual(json.loads(output.getvalue())["verdict"], "ready") - - def test_invalid_payload_surfaces_the_underlying_collection_errors(self): - """A malformed payload must still explain *why* the fields are missing. - - Reproduces the production failure that blocked ~47 open PRs: branches - matching the agent heuristic (``claude/*``, ``codex/*``, ...) are marked - applicable, but with no AgentTask issue the collector emits - ``agent_login``/``run_id`` as null. The gate correctly blocks, yet - previously reported a bare ``invalid_payload`` and discarded the - ``collection_errors`` that name the actual remediation. - """ - - payload = _valid_payload() - payload["policy"]["agent_login"] = None - payload["policy"]["run_id"] = None - payload["collection_errors"] = [ - "missing_linked_issue", - "missing_agent_run_id", - "missing_agent_login", - ] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertIn("policy.agent_login", result["details"]["invalid_fields"]) - self.assertIn("policy.run_id", result["details"]["invalid_fields"]) - self.assertEqual( - result["details"]["collection_errors"], - [ - "missing_linked_issue", - "missing_agent_run_id", - "missing_agent_login", - ], - ) - - def test_invalid_payload_omits_collection_errors_when_there_are_none(self): - payload = _valid_payload() - payload["policy"]["agent_login"] = None - payload["collection_errors"] = [] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertNotIn("collection_errors", result["details"]) - - def test_invalid_payload_tolerates_unusable_collection_errors(self): - for unusable in (None, "missing_agent_login", {"a": 1}, 7): - with self.subTest(collection_errors=unusable): - payload = _valid_payload() - payload["policy"]["agent_login"] = None - payload["collection_errors"] = unusable - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertNotIn("collection_errors", result["details"]) - - def test_invalid_payload_drops_blank_collection_errors(self): - payload = _valid_payload() - payload["policy"]["run_id"] = None - payload["collection_errors"] = ["", " ", "stale_head"] - - result = _evaluate(payload) - - self.assertEqual(result["details"]["collection_errors"], ["stale_head"]) - - def test_early_invalid_payload_paths_still_surface_collection_errors(self): - """Collector diagnostics must survive the *early* ``invalid_payload`` - returns, not only the late field-validation path. - - Regression for the reviewer's example: a payload whose ``policy`` is - malformed short-circuits before field validation, so it previously - returned a bare ``invalid_payload`` and discarded the - ``collection_errors`` the collector had already recorded. Every - invalid-payload response now routes through ``_invalid_payload`` and - carries those diagnostics consistently. - """ - - cases = ( - # policy is not a dict -> earliest invalid_fields return - ({"policy": "nope"}, "policy"), - # policy is a dict but missing `applicable` -> the exact example - # from the review thread - ({"policy": {}}, "policy.applicable"), - ) - for base, expected_field in cases: - with self.subTest(invalid_field=expected_field): - payload = dict(base) - payload["collection_errors"] = ["missing_linked_issue"] - - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertIn( - expected_field, result["details"]["invalid_fields"] - ) - self.assertEqual( - result["details"]["collection_errors"], - ["missing_linked_issue"], - ) - - def test_non_dict_payload_reports_no_collection_errors(self): - """A payload that is not even a dict has no diagnostics to surface and - must keep returning an empty ``details`` (unchanged behaviour).""" - - for payload in ([], "nope", 7, None): - with self.subTest(payload=payload): - result = _evaluate(payload) - - self.assertEqual(result["verdict"], "blocked") - self.assertEqual(result["reasons"], ["invalid_payload"]) - self.assertEqual(result["details"], {}) - - -class CompletionGateWorkflowTests(unittest.TestCase): - def _workflow(self): - return ( - _repo_root() / ".github" / "workflows" / "pr-checks.yml" - ).read_text(encoding="utf-8") - - def test_workflow_runs_from_trusted_default_branch(self): - workflow = self._workflow() - - self.assertIn("pull_request_target:", workflow) - self.assertIn("workflow_run:", workflow) - self.assertIn("issue_comment:", workflow) - self.assertIn("issues:", workflow) - self.assertIn("schedule:", workflow) - self.assertNotIn("\n pull_request_review:", workflow) - self.assertIn("github.event.repository.default_branch", workflow) - self.assertIn("persist-credentials: false", workflow) - self.assertNotIn("github.event.pull_request.head.sha }}", workflow) - - def test_workflow_cannot_leave_a_stale_green_status(self): - workflow = self._workflow() - publish_step = workflow[ - workflow.index("name: Publish stable status and comment"): - workflow.index("name: Finalize failed gate publication") - ] - finalizer_step = workflow[ - workflow.index("name: Finalize failed gate publication"): - workflow.index("name: Enforce verdict") - ] - - self.assertIn("id: resolve", workflow) - self.assertIn("state: 'pending'", workflow) - self.assertEqual( - publish_step.count("description: ownedDescription("), - 1, - ) - self.assertEqual( - publish_step.count("github.rest.repos.createCommitStatus({"), - 1, - ) - self.assertIn("'gate-owner:' + process.env.PENDING_STATUS_ID", publish_step) - self.assertIn("'gate-owner:' + process.env.PENDING_STATUS_ID", finalizer_step) - self.assertIn( - "PR_NUMBER: ${{ steps.resolve.outputs.pr_number }}", - finalizer_step, - ) - self.assertIn( - "'agent-completion/truth-gate/pr-' +\n" - " process.env.PR_NUMBER", - finalizer_step, - ) - self.assertIn("description,", finalizer_step) - self.assertIn("'pending_status_id'", workflow) - self.assertIn("String(pendingStatus.data.id)", workflow) - self.assertIn( - "PENDING_STATUS_ID: ${{ steps.resolve.outputs.pending_status_id }}", - workflow, - ) - self.assertIn("if: always() && steps.resolve.outputs.pr_number != ''", workflow) - self.assertIn("id: upload", workflow) - self.assertIn("steps.upload.outcome", workflow) - self.assertIn("id: publish", workflow) - self.assertIn("steps.publish.outcome != 'success'", workflow) - self.assertIn("target === currentRunUrl", workflow) - self.assertEqual( - publish_step.count("if (!await currentRunMayPublish(latest))"), - 2, - ) - self.assertIn("Current run has no proven gate-status lease", publish_step) - self.assertIn("core.setFailed", publish_step) - self.assertIn("disposition === 'successor'", finalizer_step) - self.assertIn("disposition === 'already_failed'", finalizer_step) - self.assertIn( - "Overriding success after failed gate publication", - finalizer_step, - ) - self.assertNotRegex( - finalizer_step, - r"already_failed'\s*\|\|\s*" - r"disposition === 'already_succeeded", - ) - self.assertIn("state: 'failure'", finalizer_step) - self.assertEqual( - finalizer_step.count("github.rest.repos.createCommitStatus({"), - 1, - ) - self.assertLess( - workflow.index("name: Upload gate evidence"), - workflow.index("name: Publish stable status and comment"), - ) - - def test_resolve_collect_and_publish_commits_must_match(self): - workflow = self._workflow() - collect_step = workflow[ - workflow.index("name: Collect repository evidence"): - workflow.index("name: Evaluate completion evidence") - ] - publish_step = workflow[ - workflow.index("name: Publish stable status and comment"): - workflow.index("name: Finalize failed gate publication") - ] - functions = _javascript_functions( - workflow, - "function commitEvidenceDisposition(", - ) - self.assertEqual(len(functions), 1) - self.assertIn( - "COLLECTED_HEAD_SHA: ${{ steps.collect.outputs.head_sha }}", - publish_step, - ) - self.assertIn( - "COLLECTED_BASE_SHA: ${{ steps.collect.outputs.base_sha }}", - publish_step, - ) - self.assertIn( - "BASE_SHA: ${{ steps.resolve.outputs.base_sha }}", - publish_step, - ) - self.assertIn( - "RESOLVED_HEAD_SHA: ${{ steps.resolve.outputs.head_sha }}", - collect_step, - ) - self.assertIn( - "RESOLVED_BASE_SHA: ${{ steps.resolve.outputs.base_sha }}", - collect_step, - ) - self.assertEqual( - collect_step.count("github.rest.pulls.get("), - 2, - ) - self.assertGreaterEqual( - collect_step.count("collectionErrors.push('stale_head')"), - 2, - ) - self.assertGreaterEqual( - collect_step.count("collectionErrors.push('stale_base')"), - 2, - ) - self.assertIn("compareCommitsWithBasehead", collect_step) - self.assertIn("basehead: pr.base.sha + '...' + pr.head.sha", collect_step) - self.assertNotIn("pulls.listFiles", collect_step) - - assertions = r""" -const a = 'a'.repeat(40); -const b = 'b'.repeat(40); -const rows = [ - [a, a, a, 'current_commit'], - [a.toUpperCase(), a, a, 'current_commit'], - [a, b, a, 'stale_commit'], - [a, a, b, 'stale_commit'], - [a, b, b, 'stale_commit'], - [a, '', a, 'stale_commit'], - ['not-a-sha', a, a, 'stale_commit'] -]; -for (const [resolved, collected, current, expected] of rows) { - const actual = commitEvidenceDisposition(resolved, collected, current); - if (actual !== expected) { - throw new Error(`${resolved}/${collected}/${current}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_workflow_reserves_status_capacity_for_finalization(self): - workflow = self._workflow() - resolve = workflow[ - workflow.index(" - name: Resolve pull request"): - workflow.index(" - name: Check out trusted gate") - ] - - self.assertIn("listCommitStatusesForRef", resolve) - self.assertIn( - "status.context === gateContext", - resolve, - ) - self.assertIn("gateStatuses.length >= 998", resolve) - self.assertIn("status_capacity_exhausted", resolve) - self.assertLess( - resolve.index("gateStatuses.length >= 998"), - resolve.index("createCommitStatus"), - ) - - def test_finalizer_publishes_only_with_proven_lease(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function mayFinalizeFailure(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - ['current_pending', true], - ['predecessor', true], - ['already_succeeded', true], - ['already_failed', false], - ['successor', false], - ['fail_closed', false], - ['', false], - [null, false] -]; -for (const [disposition, expected] of rows) { - const actual = mayFinalizeFailure(disposition); - if (actual !== expected) { - throw new Error(`${disposition}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - finalizer_step = workflow[ - workflow.index("name: Finalize failed gate publication"): - workflow.index("name: Enforce verdict") - ] - self.assertIn("if (!mayFinalizeFailure(disposition))", finalizer_step) - self.assertIn("No proven lease for failure publication", finalizer_step) - - def test_gate_status_disposition_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function gateStatusDisposition(", - ) - self.assertEqual(len(functions), 2) - - assertions = r""" -const prefix = 'https://github.com/acme/repo/actions/runs/'; -const runUrl = prefix + '100'; -const rows = [ - [null, '', 'fail_closed'], - [{id: 100, state: 'pending', target_url: runUrl}, '100', 'current_pending'], - [{id: 99, state: 'failure', target_url: runUrl}, '100', 'fail_closed'], - [{id: 101, state: 'failure', target_url: runUrl, description: 'gate-owner:100 blocked'}, '100', 'already_failed'], - [{id: 101, state: 'success', target_url: runUrl, description: 'gate-owner:100 ready'}, '100', 'already_succeeded'], - [{id: 101, state: 'pending', target_url: runUrl}, '100', 'fail_closed'], - [{id: 101, state: 'pending', target_url: prefix + '101'}, '100', 'successor'], - [{id: 900, state: 'success', target_url: prefix + '500', description: 'gate-owner:101 ready'}, '100', 'successor'], - [{id: 101, state: 'success', target_url: prefix + '99', description: 'gate-owner:99 ready'}, '100', 'predecessor'], - [{id: 103, state: 'success', target_url: prefix + '900', description: 'gate-owner:100 stale'}, '102', 'predecessor', prefix + '101'], - [{id: 103, state: 'success', target_url: prefix + '100'}, '102', 'fail_closed', prefix + '101'], - [{id: 101, state: 'pending', target_url: 'https://example.test/101'}, '100', 'fail_closed'], - [{id: 101, state: 'success', target_url: prefix + '101?attempt=2', description: 'gate-owner:101 ready'}, '100', 'fail_closed'], - [{id: 101, state: 'pending', target_url: prefix + '101'}, '', 'fail_closed'] -]; -for (const [status, pendingId, expected, currentUrl = runUrl] of rows) { - const actual = gateStatusDisposition(status, pendingId, currentUrl, prefix); - if (actual !== expected) { - throw new Error(`${JSON.stringify(status)}: ${actual} !== ${expected}`); - } -} -""" - for function in functions: - with self.subTest(function=function): - completed = subprocess.run( - ["node", "-e", function + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_copilot_review_must_be_current_head_and_submitted(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function isCurrentCopilotReview(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const reviewers = new Set(['copilot-pull-request-reviewer[bot]']); -const submittedStates = new Set(['APPROVED', 'COMMENTED', 'CHANGES_REQUESTED']); -const head = 'a'.repeat(40); -const rows = [ - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'COMMENTED', commit_id: head}, true], - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'APPROVED', commit_id: head}, true], - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'CHANGES_REQUESTED', commit_id: head}, true], - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'DISMISSED', commit_id: head}, false], - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'PENDING', commit_id: head}, false], - [{user: {login: 'copilot-pull-request-reviewer[bot]'}, state: 'COMMENTED', commit_id: 'b'.repeat(40)}, false], - [{user: {login: 'someone-else[bot]'}, state: 'COMMENTED', commit_id: head}, false], - [null, false] -]; -for (const [review, expected] of rows) { - const actual = isCurrentCopilotReview( - review, reviewers, submittedStates, head - ); - if (actual !== expected) { - throw new Error(`${JSON.stringify(review)}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_refresh_dispatch_trust_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function mayDispatchEvidenceRefresh(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const associations = new Set(['OWNER', 'MEMBER', 'COLLABORATOR']); -const agents = new Set([ - 'google-labs-jules[bot]', - 'github-copilot[bot]', - 'openai-codex[bot]', - 'chatgpt-codex-connector[bot]', - 'github-actions[bot]' -]); -const rows = [ - ['OWNER', 'person', true], - ['MEMBER', 'person', true], - ['COLLABORATOR', 'person', true], - ['NONE', 'google-labs-jules[bot]', true], - ['NONE', 'github-copilot[bot]', true], - ['NONE', 'openai-codex[bot]', true], - ['NONE', 'chatgpt-codex-connector[bot]', true], - ['NONE', 'external-user', false], - ['NONE', 'github-actions[bot]', true], - [null, null, false] -]; -for (const [association, actor, expected] of rows) { - const actual = mayDispatchEvidenceRefresh( - association, actor, associations, agents - ); - if (actual !== expected) { - throw new Error(`${association}/${actor}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_comment_refresh_requires_declared_agent_result_evidence(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function issueCommentAffectsEvidence(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const pull = { - pull_request: {}, - body: '' -}; -const issue = { - body: '## Agent login\n\ngoogle-labs-jules[bot]\n\n## Objective\nTest' -}; -const rows = [ - ['declared ready', pull, 'google-labs-jules[bot]', - 'Ready for a review! A PR has been created.', null, true], - ['declared structured', issue, 'google-labs-jules[bot]', - '', null, true], - ['event edited away', pull, 'google-labs-jules[bot]', - 'ordinary update', 'Jules encountered an unexpected error', true], - ['declared chatter', pull, 'google-labs-jules[bot]', - 'ordinary update', null, false], - ['human discussion', pull, 'maintainer', - 'Ready for a review! A PR has been created.', null, false], - ['unrelated known bot', pull, 'openai-codex[bot]', - 'Ready for a review! A PR has been created.', null, false], - ['gate comment', pull, 'github-actions[bot]', - '', null, false], - ['intent invalidation', issue, 'github-actions[bot]', - '', null, true], - ['missing contract', {body: ''}, 'google-labs-jules[bot]', - 'Ready for a review! A PR has been created.', null, false] -]; -for (const [name, target, actor, body, previousBody, expected] of rows) { - const actual = issueCommentAffectsEvidence( - target, - {user: {login: actor}, body}, - previousBody - ); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - dispatch = workflow[ - workflow.index(" dispatch-evidence-refresh:"): - workflow.index(" validate:") - ] - self.assertIn("issueCommentAffectsEvidence(", dispatch) - self.assertIn("context.payload.changes", dispatch) - - def test_issue_refresh_uses_sender_permission_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function mayDispatchIssueRefresh(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const trustedPermissions = new Set([ - 'admin', 'maintain', 'write', 'triage' -]); -const rows = [ - ['maintainer', {permission: 'admin', role_name: 'admin'}, true], - ['maintainer', {permission: 'write', role_name: 'maintain'}, true], - ['maintainer', {permission: 'write', role_name: 'write'}, true], - ['custom-writer', {permission: 'write', role_name: 'release'}, true], - ['triager', {permission: 'read', role_name: 'triage'}, true], - ['external-user', {permission: 'read', role_name: 'read'}, false], - ['custom-reader', {permission: 'read', role_name: 'observe'}, false], - ['github-actions[bot]', {permission: 'none', role_name: 'none'}, false], - ['google-labs-jules[bot]', {permission: 'none', role_name: 'none'}, false], - ['github-copilot[bot]', {permission: 'read', role_name: 'read'}, false], - ['openai-codex[bot]', {permission: 'write', role_name: 'write'}, true], - [null, {permission: 'admin', role_name: 'admin'}, false] -]; -for (const [actor, response, expected] of rows) { - const actual = mayDispatchIssueRefresh( - actor, response.permission, response.role_name, trustedPermissions - ); - if (actual !== expected) { - throw new Error(`${actor}/${JSON.stringify(response)}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - dispatch = workflow[ - workflow.index(" dispatch-evidence-refresh:"): - workflow.index(" validate:") - ] - self.assertNotIn("!knownAgentCommenters.has(actor)", dispatch) - self.assertIn("roleName = result.data.role_name", dispatch) - - def test_every_known_agent_is_treated_as_an_ai_reviewer(self): - workflow = self._workflow() - reviewer_set = workflow[ - workflow.index("function aiReviewerLoginSet("): - workflow.index("function isCurrentCopilotReview(") - ] - - self.assertNotIn("...knownAgents", reviewer_set) - self.assertIn("'google-labs-jules[bot]'", reviewer_set) - self.assertIn("'chatgpt-codex-connector[bot]'", reviewer_set) - self.assertIn("'vercel[bot]'", reviewer_set) - self.assertIn("const aiReviewerLogins = aiReviewerLoginSet()", reviewer_set) - - set_functions = _javascript_functions( - workflow, - "function aiReviewerLoginSet(", - ) - self.assertEqual(len(set_functions), 1) - set_assertions = r""" -const reviewers = aiReviewerLoginSet(); -const required = [ - 'google-labs-jules', - 'github-copilot', - 'copilot-swe-agent', - 'openai-codex', - 'chatgpt-codex-connector', - 'copilot-pull-request-reviewer', - 'coderabbitai', - 'vercel' -]; -for (const login of required) { - if (!reviewers.has(login)) { - throw new Error(`missing reviewer alias: ${login}`); - } -} -if (reviewers.has('human-reviewer')) { - throw new Error('human reviewer must not be treated as an AI reviewer'); -} -""" - completed = subprocess.run( - ["node", "-e", set_functions[0] + set_assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - functions = _javascript_functions( - workflow, - "function normaliseBotLogin(", - ) - self.assertEqual(len(functions), 1) - assertions = r""" -const reviewers = new Set([ - 'google-labs-jules[bot]', - 'github-copilot[bot]', - 'copilot-swe-agent[bot]', - 'openai-codex[bot]', - 'chatgpt-codex-connector[bot]', - 'copilot-pull-request-reviewer[bot]', - 'coderabbitai[bot]', - 'vercel[bot]' -].map(normaliseBotLogin)); -const rows = [ - ['google-labs-jules', true], - ['google-labs-jules[bot]', true], - ['github-copilot', true], - ['copilot-swe-agent', true], - ['openai-codex', true], - ['chatgpt-codex-connector', true], - ['copilot-pull-request-reviewer', true], - ['coderabbitai', true], - ['vercel', true], - ['human-reviewer', false], - ['', false], - [null, false] -]; -for (const [login, expected] of rows) { - const actual = reviewers.has(normaliseBotLogin(login)); - if (actual !== expected) { - throw new Error(`${login}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - thread_collector = workflow[ - workflow.index("const threads ="): - workflow.index("function focusedTestResultsFromLog(") - ] - self.assertRegex( - thread_collector, - r"aiReviewerLogins\.has\(\s*" - r"normaliseBotLogin\(comment\.author\.login\)\s*\)", - ) - - def test_unrestricted_scope_checkbox_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function checkboxChecked(", - ) - self.assertEqual(len(functions), 2) - - assertions = r""" -const rows = [ - ['- [ ] Yes, this task explicitly permits repository-wide changes.', false], - ['- [x] Yes, this task explicitly permits repository-wide changes.', true], - ['- [X] Yes, this task explicitly permits repository-wide changes.', true], - ['yes', true], - [' true ', true], - ['no', false], - ['_No response_', false], - ['The word yes in prose is not approval.', false], - ['', false] -]; -for (const [value, expected] of rows) { - const actual = checkboxChecked(value); - if (actual !== expected) { - throw new Error(`${JSON.stringify(value)}: ${actual} !== ${expected}`); - } -} -""" - for function in functions: - with self.subTest(function=function): - completed = subprocess.run( - ["node", "-e", function + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_snapshot_label_actor_permission_decision_table(self): - workflow = self._workflow() - snapshot = workflow[ - workflow.index(" snapshot-agent-task-intent:"): - workflow.index(" refresh-open-pull-requests:") - ] - functions = _javascript_functions( - workflow, - "function hasTrustedSnapshotPermission(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - [{permission: 'admin', role_name: 'admin'}, true], - [{permission: 'write', role_name: 'maintain'}, true], - [{permission: 'write', role_name: 'write'}, true], - [{permission: 'write', role_name: 'release'}, true], - [{permission: 'read', role_name: 'triage'}, true], - [{permission: 'read', role_name: 'read'}, false], - [{permission: 'read', role_name: 'observe'}, false], - [{permission: 'none', role_name: 'none'}, false], - [{permission: '', role_name: ''}, false], - [{permission: null, role_name: null}, false] -]; -for (const [response, expected] of rows) { - const actual = hasTrustedSnapshotPermission( - response.permission, response.role_name - ); - if (actual !== expected) { - throw new Error(`${JSON.stringify(response)}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - self.assertNotIn("author_association", snapshot) - self.assertNotIn( - "if (context.payload.action === 'labeled')", - snapshot, - ) - self.assertIn("username: context.actor", snapshot) - self.assertIn("roleName = result.data.role_name", snapshot) - self.assertNotIn("github.event.action == 'opened'", snapshot) - self.assertIn("github.event.action == 'labeled'", snapshot) - self.assertIn("github.event.action == 'edited'", snapshot) - self.assertIn("github.event.action == 'unlabeled'", snapshot) - self.assertIn("agent-lock-intent-invalidated:v1", snapshot) - self.assertEqual(snapshot.count("workflow_run_id: workflowRunId"), 2) - self.assertIn("/^[1-9]\\d*$/.test(workflowRunId)", snapshot) - self.assertIn("'scopeunrestrictedapproved'", snapshot) - self.assertIn("'agenttask'", snapshot) - self.assertIn("'mcpagent'", snapshot) - self.assertLess( - snapshot.index("agent-lock-intent-invalidated:v1"), - snapshot.index("getCollaboratorPermissionLevel"), - ) - - def test_snapshot_event_disposition_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function snapshotEventDisposition(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - [false, 'opened', 'agenttask', '', '101', 'ignore'], - [false, 'labeled', 'agenttask', '', '101', 'snapshot'], - [false, 'labeled', 'mcpagent', '', '102', 'snapshot'], - [false, 'labeled', 'priority', '', '103', 'ignore'], - [true, 'labeled', 'agenttask', '101', '101', 'same_snapshot_run'], - [true, 'labeled', 'agenttask', '101', '102', 'invalidate'], - [true, 'edited', '', '101', '103', 'invalidate'], - [true, 'unlabeled', 'agenttask', '101', '104', 'invalidate'], - [true, 'opened', 'agenttask', '101', '105', 'ignore'] -]; -for (const [snapshot, action, label, sourceRun, run, expected] of rows) { - const actual = snapshotEventDisposition( - snapshot, action, label, sourceRun, run - ); - if (actual !== expected) { - throw new Error(`${JSON.stringify([snapshot, action, label, sourceRun, run])}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_linked_issue_contract_decision_table(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function linkedIssueContract(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - [[870, 870, 870], 'ok'], - [[0, 870, 870], 'missing'], - [[870, 0, 870], 'missing'], - [[870, 870, 0], 'missing'], - [[870, 871, 870], 'conflicting'], - [[870, 870, 871], 'conflicting'], - [['870', 870, 870], 'missing'], - [[-1, 870, 870], 'missing'] -]; -for (const [values, expected] of rows) { - const actual = linkedIssueContract(...values); - if (actual !== expected) { - throw new Error(`${JSON.stringify(values)}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_every_github_script_body_compiles(self): - scripts = _github_script_bodies(self._workflow()) - self.assertEqual(len(scripts), 8) - compiler = ( - "const AsyncFunction = Object.getPrototypeOf(" - "async function(){}).constructor;" - "new AsyncFunction('github','context','core','require'," - ) - for index, script in enumerate(scripts): - with self.subTest(index=index): - completed = subprocess.run( - ["node", "-e", compiler + json.dumps(script) + ");"], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_workflow_pins_every_third_party_action(self): - workflow = self._workflow() - - self.assertNotRegex(workflow, r"uses:\s+actions/[^@\s]+@v\d+") - self.assertIn( - "actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0", - workflow, - ) - self.assertIn( - "actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3", - workflow, - ) - self.assertIn( - "actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a", - workflow, - ) - - def test_workflow_freezes_issue_intent_before_agent_execution(self): - workflow = self._workflow() - - self.assertIn("types: [opened, edited, labeled, unlabeled, closed, reopened]", workflow) - self.assertIn("snapshot-agent-task-intent:", workflow) - self.assertIn("agent-lock-intent-snapshot:v1", workflow) - self.assertGreaterEqual( - workflow.count("agent-lock-intent-invalidated:v1"), - 3, - ) - self.assertIn("createHash('sha256')", workflow) - self.assertIn("missing_intent_snapshot", workflow) - self.assertIn("intent_changed_after_dispatch", workflow) - - def test_snapshot_must_strictly_predate_pull(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function snapshotPredatesPull(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - ['2026-07-18T03:00:00.000Z', '2026-07-18T03:00:01.000Z', true], - ['2026-07-18T03:00:00.000Z', '2026-07-18T03:00:00.000Z', false], - ['2026-07-18T03:00:01.000Z', '2026-07-18T03:00:00.000Z', false], - ['not-a-date', '2026-07-18T03:00:00.000Z', false], - ['2026-07-18T03:00:00.000Z', null, false] -]; -for (const [snapshot, pull, expected] of rows) { - const actual = snapshotPredatesPull(snapshot, pull); - if (actual !== expected) { - throw new Error(`${snapshot}/${pull}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_snapshot_requires_privileged_actor_for_relabel(self): - workflow = self._workflow() - header = workflow[ - workflow.index(" snapshot-agent-task-intent:"): - workflow.index(" concurrency:", workflow.index( - " snapshot-agent-task-intent:" - )) - ] - - self.assertNotIn("github.event.action == 'opened'", header) - self.assertIn("github.event.action == 'labeled'", header) - self.assertNotIn("author_association", header) - self.assertIn("getCollaboratorPermissionLevel", workflow) - self.assertIn("context.actor", workflow) - - def test_workflow_publishes_one_machine_readable_gate(self): - workflow = self._workflow() - - self.assertIn("scripts/ci/agent_completion_gate.py", workflow) - self.assertIn("agent-completion/truth-gate", workflow) - self.assertIn("agent-completion-truth-gate:v1", workflow) - self.assertIn("actions/upload-artifact@", workflow) - - def test_status_context_is_bound_to_the_pull_request(self): - workflow = self._workflow() - - self.assertIn( - "return 'agent-completion/truth-gate/pr-' + pullNumber", - workflow, - ) - self.assertGreaterEqual( - workflow.count( - "'agent-completion/truth-gate/pr-' + prNumber" - ), - 1, - ) - self.assertEqual(workflow.count("context: gateContext"), 3) - self.assertIn("context: pendingRecovery.gateContext", workflow) - self.assertIn("context: terminalInvalidation.gateContext", workflow) - self.assertNotRegex( - workflow, - r"context:\s*'agent-completion/truth-gate'", - ) - - def test_workflow_uses_required_ci_and_separate_post_merge_evidence(self): - workflow = self._workflow() - - self.assertIn("run.name === 'CI'", workflow) - self.assertIn("pr.merge_commit_sha", workflow) - self.assertNotIn("getCombinedStatusForRef", workflow) - - def test_focused_tests_require_per_path_passing_ci_log(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function focusedTestResultsFromLog(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const log = [ - 'PASSED [1%]', - '2026-07-18T03:00:00Z tests/unit/test_alpha.py::test_one PASSED [33%]', - '2026-07-18T03:00:01Z tests/unit/test_beta.py::test_two SKIPPED [66%]', - '\u001b[32mtests/unit/test_beta.py::test_three PASSED\u001b[0m [100%]', - 'tests/unit/test_alpha.py.evil::test_spoof PASSED [100%]', - 'tests/unit/test_gamma.py::test_failure FAILED [100%]', - 'tests/unit/test_only_skipped.py::test_skip SKIPPED [100%]', - 'tests/unit/test_param.py::test_x[PASSED fake] SKIPPED [100%]', - 'tests/unit/test_live.py::test_pass', - '-------------------------------- live log call ---------------------------------', - 'INFO example:test_live.py:10 still running', - '\u001b[32m2026-07-18T03:00:02Z PASSED\u001b[0m [10%]', - 'tests/unit/test_live_failed.py::test_fail', - '-------------------------------- live log call ---------------------------------', - 'WARNING example:test_live.py:20 log says PASSED [99%]', - 'FAILED [20%]', - 'tests/unit/test_live_xpass.py::test_expected_failure', - '-------------------------------- live log call ---------------------------------', - 'XPASS (known bug: now passing) [25%]', - 'tests/unit/test_shadowed.py::test_pending', - '-------------------------------- live log call ---------------------------------', - 'tests/unit/untracked file.py::test_other', - '-------------------------------- live log call ---------------------------------', - 'PASSED [30%]', - 'tests/unit/test_boundary.py::test_pending', - '-------------------------------- live log call ---------------------------------', - '============================= test session summary =============================', - 'PASSED [40%]', - 'tests/unit/test_param_spoof.py::test_x[value] PASSED [42%]', - '-------------------------------- live log call ---------------------------------', - 'SKIPPED [50%]', - '\uFEFF2026-07-18T03:00:03Z tests/unit/test_bom.py::test_pass PASSED [55%]', - ' tests/unit/test_indented.py::test_spoof', - '-------------------------------- live log call ---------------------------------', - 'PASSED [60%]', - 'tests/unit/test_blank_tail.py:: PASSED [65%]', - 'tests/unit/test_actions_boundary.py::test_pending', - '-------------------------------- live log call ---------------------------------', - '##[group]post-test diagnostics', - 'PASSED [70%]' -].join('\n'); -const actual = focusedTestResultsFromLog(log, [ - 'tests/unit/test_alpha.py', - 'tests/unit/test_beta.py', - 'tests/unit/test_gamma.py', - 'tests/unit/test_only_skipped.py', - 'tests/unit/test_param.py', - 'tests/unit/test_live.py', - 'tests/unit/test_live_failed.py', - 'tests/unit/test_live_xpass.py', - 'tests/unit/test_shadowed.py', - 'tests/unit/test_boundary.py', - 'tests/unit/test_param_spoof.py', - 'tests/unit/test_bom.py', - 'tests/unit/test_indented.py', - 'tests/unit/test_blank_tail.py', - 'tests/unit/test_actions_boundary.py' -]); -const expected = { - 'tests/unit/test_alpha.py': {passed: 1, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_beta.py': {passed: 1, failed: 0, errors: 0, skipped: 1, xfailed: 0, xpassed: 0}, - 'tests/unit/test_gamma.py': {passed: 0, failed: 1, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_only_skipped.py': {passed: 0, failed: 0, errors: 0, skipped: 1, xfailed: 0, xpassed: 0}, - 'tests/unit/test_param.py': {passed: 0, failed: 0, errors: 0, skipped: 1, xfailed: 0, xpassed: 0}, - 'tests/unit/test_live.py': {passed: 1, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_live_failed.py': {passed: 0, failed: 1, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_live_xpass.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 1}, - 'tests/unit/test_shadowed.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_boundary.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_param_spoof.py': {passed: 0, failed: 0, errors: 0, skipped: 1, xfailed: 0, xpassed: 0}, - 'tests/unit/test_bom.py': {passed: 1, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_indented.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_blank_tail.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0}, - 'tests/unit/test_actions_boundary.py': {passed: 0, failed: 0, errors: 0, skipped: 0, xfailed: 0, xpassed: 0} -}; -if (JSON.stringify(actual) !== JSON.stringify(expected)) { - throw new Error(`${JSON.stringify(actual)} !== ${JSON.stringify(expected)}`); -} -const crOnly = focusedTestResultsFromLog( - [ - 'tests/unit/test_cr.py::test_pass', - '-------------------------------- live log call ---------------------------------', - 'PASSED [100%]' - ].join('\r'), - ['tests/unit/test_cr.py'] -); -if (crOnly['tests/unit/test_cr.py'].passed !== 1) { - throw new Error(`CR-only parsing failed: ${JSON.stringify(crOnly)}`); -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - self.assertIn("listJobsForWorkflowRun", workflow) - self.assertIn("downloadJobLogsForWorkflowRun", workflow) - self.assertIn( - "focused_test_results: focusedTestResults", - workflow, - ) - self.assertNotIn( - "focused_tests_passed: focusedTestFiles.length > 0 && " - "requiredChecksPassed", - workflow, - ) - - def test_scheduled_sweep_can_list_pull_requests(self): - workflow = self._workflow() - sweep = workflow[ - workflow.index(" refresh-open-pull-requests:"): - workflow.index(" dispatch-evidence-refresh:") - ] - - self.assertIn("issues: read", sweep) - self.assertIn("pull-requests: read", sweep) - self.assertIn("statuses: write", sweep) - self.assertIn("github.paginate(", sweep) - self.assertIn("github.rest.pulls.list", sweep) - self.assertIn("state: 'open'", sweep) - self.assertIn("github.rest.repos.listCommitStatusesForRef", sweep) - self.assertIn("closingIssuesReferences", sweep) - self.assertIn("github.rest.issues.listComments", sweep) - self.assertIn("github.rest.pulls.listReviews", sweep) - self.assertIn("github.rest.actions.listWorkflowRunsForRepo", sweep) - self.assertIn("github.rest.actions.getWorkflowRun", sweep) - self.assertIn("run.data.name === 'PR Checks'", sweep) - self.assertIn( - "run.data.path === '.github/workflows/pr-checks.yml'", - sweep, - ) - self.assertIn("ownerRunId !== terminalRunId", sweep) - self.assertIn("scheduledRefreshDecision(", sweep) - self.assertIn("if (decision === 'dispatch')", sweep) - self.assertIn("group: agent-completion-scheduled-scan", sweep) - self.assertIn("timeout-minutes: 14", sweep) - self.assertIn("Skipping stale scheduled decision", sweep) - self.assertGreaterEqual( - sweep.count("listCommitStatusesForRef"), - 2, - ) - self.assertIn("github.rest.actions.createWorkflowDispatch", sweep) - self.assertIn("workflow_id: 'pr-checks.yml'", sweep) - self.assertIn("ref: context.payload.repository.default_branch", sweep) - self.assertIn("inputs: {pull_request: String(pull.number)}", sweep) - - self.assertIn('cron: "*/15 * * * *"', workflow) - - def test_scheduled_sweep_dispatches_only_for_evidence_delta(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function scheduledRefreshDecision(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const base = { - apiComplete: true, - statusCount: 2, - latestState: 'success', - latestCreatedAt: '2026-07-18T04:30:00Z', - ownerPendingAt: '2026-07-18T04:20:00Z', - ownerRunState: 'completed', - now: '2026-07-18T05:00:00Z', - pullUpdatedAt: '2026-07-18T04:25:00Z', - issueUpdatedAt: '2026-07-18T04:15:00Z', - commentUpdatedAt: '2026-07-18T04:15:00Z', - ciUpdatedAt: '2026-07-18T04:15:00Z', - reviewProjectionChanged: false, - intentProjectionChanged: false, - policyProjectionChanged: false, - terminalAlreadyInvalidated: false -}; -const rows = [ - ['no status', {...base, latestState: null}, 'dispatch'], - ['unchanged terminal', base, 'skip'], - ['status capacity', {...base, statusCount: 998}, 'at_capacity'], - ['capacity evidence delta', { - ...base, - statusCount: 998, - commentUpdatedAt: '2026-07-18T04:21:00Z' - }, 'invalidate_terminal'], - ['last-slot evidence delta', { - ...base, - statusCount: 999, - reviewProjectionChanged: true - }, 'invalidate_terminal'], - ['capacity failure evidence delta', { - ...base, - statusCount: 998, - latestState: 'failure', - commentUpdatedAt: '2026-07-18T04:21:00Z' - }, 'at_capacity'], - ['scheduled recovery failure evidence delta', { - ...base, - statusCount: 999, - latestState: 'failure', - reviewProjectionChanged: true - }, 'at_capacity'], - ['settled capacity invalidation', { - ...base, - statusCount: 999, - reviewProjectionChanged: true, - terminalAlreadyInvalidated: true - }, 'at_capacity'], - ['exhausted evidence delta', { - ...base, - statusCount: 1000, - reviewProjectionChanged: true - }, 'at_capacity'], - ['API uncertainty', {...base, apiComplete: false}, 'retry'], - ['malformed terminal lease', {...base, ownerPendingAt: null}, 'dispatch'], - ['irrelevant PR timestamp', {...base, pullUpdatedAt: '2026-07-18T04:31:00Z'}, 'skip'], - ['irrelevant issue timestamp', {...base, issueUpdatedAt: '2026-07-18T04:21:00Z'}, 'skip'], - ['comment delta', {...base, commentUpdatedAt: '2026-07-18T04:21:00Z'}, 'dispatch'], - ['CI delta', {...base, ciUpdatedAt: '2026-07-18T04:21:00Z'}, 'dispatch'], - ['review delta', {...base, reviewProjectionChanged: true}, 'dispatch'], - ['intent delta', {...base, intentProjectionChanged: true}, 'dispatch'], - ['policy delta', {...base, policyProjectionChanged: true}, 'dispatch'], - ['fresh pending', { - ...base, - latestState: 'pending', - latestCreatedAt: '2026-07-18T04:30:00Z', - ownerPendingAt: '2026-07-18T04:30:00Z', - ownerRunState: 'in_progress', - pendingLeaseValid: true - }, 'skip'], - ['stale pending', { - ...base, - latestState: 'pending', - latestCreatedAt: '2026-07-18T03:59:59Z', - ownerPendingAt: '2026-07-18T03:59:59Z', - ownerRunState: 'in_progress', - pendingLeaseValid: true - }, 'finalize_pending'], - ['completed pending', { - ...base, - latestState: 'pending', - ownerPendingAt: '2026-07-18T04:30:00Z', - ownerRunState: 'completed', - pendingLeaseValid: true - }, 'finalize_pending'], - ['missing pending run', { - ...base, - latestState: 'pending', - ownerPendingAt: '2026-07-18T04:30:00Z', - ownerRunState: 'missing', - pendingLeaseValid: true - }, 'finalize_pending'], - ['pending 998 recovery', { - ...base, - statusCount: 998, - latestState: 'pending', - latestCreatedAt: '2026-07-18T04:30:00Z', - ownerPendingAt: '2026-07-18T04:30:00Z', - ownerRunState: 'completed', - pendingLeaseValid: true - }, 'finalize_pending'], - ['malformed pending at capacity', { - ...base, - statusCount: 998, - latestState: 'pending', - latestCreatedAt: '2026-07-18T04:30:00Z', - ownerPendingAt: '2026-07-18T04:30:00Z', - ownerRunState: 'missing', - pendingLeaseValid: false - }, 'at_capacity'] -]; -for (const [name, input, expected] of rows) { - const actual = scheduledRefreshDecision(input); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_scheduled_review_projection_fails_safe_without_churning(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function verdictProjection(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -function comment(verdict) { - return { - user: {login: 'github-actions[bot]'}, - body: '' + - '
' + JSON.stringify(verdict) + '
' - }; -} -const invalid = verdictProjection([comment({ - verdict: 'blocked', - reasons: ['invalid_payload'], - details: {} -})]); -if (!invalid || invalid.projectionKnown !== false || - invalid.projectionUsable !== false) { - throw new Error('invalid payload projection must be unknown'); -} -const failedCollection = verdictProjection([comment({ - verdict: 'blocked', - reasons: ['evidence_collection_failed'], - details: {unresolved_reviews: ['thread-b']} -})]); -if (!failedCollection || failedCollection.projectionKnown !== false || - failedCollection.projectionUsable !== false) { - throw new Error('partial collection projection must be unknown'); -} -const recordedCollection = verdictProjection([comment({ - verdict: 'blocked', - reasons: ['evidence_collection_failed'], - details: { - collection_errors: ['conflicting_linked_issue'], - identity_projection: { - issue_number: 870, - agent_login: 'example-agent[bot]', - run_id: 'run-1' - } - } -})]); -if (!recordedCollection || recordedCollection.projectionKnown !== false || - recordedCollection.projectionUsable !== true || - JSON.stringify(recordedCollection.collectionErrors) !== - JSON.stringify(['conflicting_linked_issue']) || - recordedCollection.identityProjection.issue_number !== 870) { - throw new Error('recorded collection projection must remain usable'); -} -const partialIdentity = verdictProjection([comment({ - verdict: 'blocked', - reasons: ['evidence_collection_failed'], - details: { - collection_errors: ['missing_agent_login'], - identity_projection: { - issue_number: 870, - agent_login: null, - run_id: 'run-1' - } - } -})]); -if (!partialIdentity || - JSON.stringify(partialIdentity.identityProjection) !== JSON.stringify({ - issue_number: 870, - agent_login: null, - run_id: 'run-1' - })) { - throw new Error('partial identity projection must be preserved'); -} -const evaluated = verdictProjection([comment({ - verdict: 'blocked', - reasons: [ - 'missing_copilot_current_head_review', - 'unresolved_review', - 'draft_pr', - 'missing_copilot_rabbit_label' - ], - details: { - unresolved_reviews: ['thread-b', 'thread-a'], - collection_errors: ['agent_login_mismatch'], - identity_projection: { - issue_number: 870, - agent_login: 'example-agent[bot]', - run_id: 'run-1' - } - } -})]); -if (!evaluated || evaluated.projectionKnown !== true || - evaluated.projectionUsable !== true || - evaluated.copilotCurrentHeadReviewed !== false || - JSON.stringify(evaluated.unresolved) !== - JSON.stringify(['thread-a', 'thread-b']) || - JSON.stringify(evaluated.mutablePolicyErrors) !== JSON.stringify([ - 'agent_login_mismatch', - 'draft_pr', - 'missing_copilot_rabbit_label' - ]) || evaluated.identityProjection.run_id !== 'run-1') { - throw new Error('evaluated projection was not preserved'); -} -const exempt = verdictProjection([comment({ - verdict: 'not_applicable', - reasons: [], - details: {} -})]); -if (!exempt || exempt.projectionKnown !== true || - exempt.projectionUsable !== true || - exempt.notApplicable !== true) { - throw new Error('not-applicable projection must be stable'); -} -if (verdictProjection([]) !== null) { - throw new Error('missing verdict comment must remain unknown'); -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - sweep = workflow[ - workflow.index(" refresh-open-pull-requests:"): - workflow.index(" dispatch-evidence-refresh:") - ] - self.assertIn("latest.state === 'success'", sweep) - self.assertIn("previousProjection.projectionKnown", sweep) - - def test_scheduled_scanner_ignores_ordinary_comments(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function scheduledCommentAffectsEvidence(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const agents = new Set(['google-labs-jules[bot]']); -const rows = [ - ['structured', 'google-labs-jules[bot]', - '', true], - ['ready', 'google-labs-jules[bot]', - 'Ready for a review! A PR has been created.', true], - ['error', 'google-labs-jules[bot]', - "Jules wasn't able to complete the task", true], - ['ordinary agent chatter', 'google-labs-jules[bot]', - 'Here is a progress update.', false], - ['maintainer discussion', 'maintainer', - 'Ready for a review! A PR has been created.', false], - ['gate publication', 'github-actions[bot]', - '', false] -]; -for (const [name, actor, body, expected] of rows) { - const actual = scheduledCommentAffectsEvidence( - {user: {login: actor}, body}, - agents - ); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - sweep = workflow[ - workflow.index(" refresh-open-pull-requests:"): - workflow.index(" dispatch-evidence-refresh:") - ] - self.assertIn("const evidenceComments =", sweep) - self.assertNotIn("externalPullComments", sweep) - - def test_scheduled_scanner_detects_frozen_intent_changes(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function intentContractErrors(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const crypto = require('crypto'); -const body = '## Objective\n\nKeep immutable intent.'; -const digest = crypto.createHash('sha256') - .update(body, 'utf8').digest('hex'); -const issue = { - number: 870, - body, - labels: {nodes: [{name: 'agent-task'}]} -}; -function snapshot(overrides = {}) { - const value = { - issue_number: 870, - body_sha256: digest, - scope_unrestricted_approved: false, - ...overrides - }; - return { - user: {login: 'github-actions[bot]'}, - created_at: '2026-07-18T03:00:00Z', - body: '' - }; -} -function invalidation() { - return { - user: {login: 'github-actions[bot]'}, - created_at: '2026-07-18T03:00:02Z', - body: '' - }; -} -const rows = [ - ['valid', issue, [snapshot()], []], - ['missing', issue, [], ['missing_intent_snapshot']], - ['changed body', {...issue, body: body + ' changed'}, [snapshot()], - ['intent_changed_after_dispatch']], - ['restored body after edit', issue, [snapshot(), invalidation()], - ['intent_changed_after_dispatch']], - ['approval changed', { - ...issue, - labels: {nodes: [ - {name: 'agent-task'}, - {name: 'scope-unrestricted-approved'} - ]} - }, [snapshot()], ['unrestricted_approval_changed_after_dispatch']], - ['label and snapshot missing', { - ...issue, - labels: {nodes: []} - }, [], ['linked_issue_not_agent_task', 'missing_intent_snapshot']], - ['bad issue number', issue, [snapshot({issue_number: 999})], - ['invalid_intent_snapshot']] -]; -for (const [name, candidate, comments, expected] of rows) { - const actual = intentContractErrors( - candidate, - comments, - '2026-07-18T03:00:01Z' - ); - if (JSON.stringify(actual) !== JSON.stringify(expected)) { - throw new Error(`${name}: ${JSON.stringify(actual)} !== ` + - JSON.stringify(expected)); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - sweep = workflow[ - workflow.index(" refresh-open-pull-requests:"): - workflow.index(" dispatch-evidence-refresh:") - ] - self.assertIn("intentProjectionChanged", sweep) - self.assertIn("previousProjection.collectionErrors", sweep) - - projection_functions = _javascript_functions( - workflow, - "function scheduledIntentProjectionChanged(", - ) - self.assertEqual(len(projection_functions), 1) - projection_assertions = r""" -const rows = [ - ['N/A remains N/A', {notApplicable: true, collectionErrors: []}, - ['missing_closing_issue_reference'], 0, false, false], - ['N/A becomes applicable', {notApplicable: true, collectionErrors: []}, - [], 1, true, true], - ['applicable becomes N/A', {notApplicable: false, collectionErrors: []}, - [], 1, false, true], - ['new missing link', {notApplicable: false, collectionErrors: []}, - ['missing_closing_issue_reference'], 0, true, true], - ['settled missing link', { - notApplicable: false, - collectionErrors: ['missing_closing_issue_reference'] - }, ['missing_closing_issue_reference'], 0, true, false], - ['settled multiple links', { - notApplicable: false, - collectionErrors: [ - 'missing_closing_issue_reference', - 'multiple_closing_issues', - 'missing_intent_snapshot' - ] - }, [ - 'missing_closing_issue_reference', - 'multiple_closing_issues' - ], 2, true, false], - ['link repaired', { - notApplicable: false, - collectionErrors: ['missing_closing_issue_reference'] - }, [], 1, true, true], - ['body changed', {notApplicable: false, collectionErrors: []}, - ['intent_changed_after_dispatch'], 1, true, true], - ['body failure settled', { - notApplicable: false, - collectionErrors: ['intent_changed_after_dispatch'] - }, ['intent_changed_after_dispatch'], 1, true, false], - ['new conflicting contract', { - notApplicable: false, - collectionErrors: [] - }, ['conflicting_linked_issue'], 1, true, true], - ['settled conflicting contract', { - notApplicable: false, - collectionErrors: ['conflicting_linked_issue'] - }, ['conflicting_linked_issue'], 1, true, false], - ['contract repaired', { - notApplicable: false, - collectionErrors: ['incomplete_linked_issue_contract'] - }, [], 1, true, true], - ['settled unavailable conflicting target', { - notApplicable: false, - collectionErrors: [ - 'conflicting_linked_issue', - 'linked_issue_unavailable', - 'missing_linked_issue' - ] - }, [ - 'conflicting_linked_issue', - 'linked_issue_unavailable', - 'missing_linked_issue' - ], 1, true, false], - ['hard unknown failure stays settled', { - notApplicable: false, - projectionUsable: false, - collectionErrors: [] - }, ['conflicting_linked_issue'], 1, true, false] -]; -for (const [name, previous, current, count, applicable, expected] of rows) { - const actual = scheduledIntentProjectionChanged( - previous, current, count, applicable - ); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - [ - "node", - "-e", - projection_functions[0] + projection_assertions, - ], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - sweep = workflow[ - workflow.index(" refresh-open-pull-requests:"): - workflow.index(" dispatch-evidence-refresh:") - ] - self.assertIn("const applicableWithoutSelectedIssue =", sweep) - self.assertIn("linkProjection.errors.length > 0", sweep) - self.assertIn("if (error.status !== 404", sweep) - self.assertIn("selectedIssueUnavailable = true", sweep) - self.assertIn("['linked_issue_unavailable']", sweep) - self.assertNotIn("selectedIssue || linkedIssues[0]", sweep) - self.assertLess( - sweep.index("const linkProjection = scheduledLinkProjection("), - sweep.index("await github.rest.issues.get("), - ) - - applicable_functions = _javascript_functions( - workflow, - "function agentTaskApplicable(", - ) - self.assertEqual(len(applicable_functions), 2) - self.assertEqual(applicable_functions[0], applicable_functions[1]) - applicable_assertions = r""" -const CONTRACT = [ - '### Agent Login', '', '`google-labs-jules[bot]`', '', - '### Agent Run ID', '', '`run-42`' -].join('\n'); -const issue = (number, labels, body) => ({ - number, - labels: {nodes: labels.map(name => ({name}))}, - body: body || '' -}); -const base = { - user: {login: 'maintainer'}, - head: {ref: 'feature/ordinary'}, - labels: [], - body: '' -}; -const PLAIN = issue(1, [], ''); -const rows = [ - ['human', base, null, false], - // A bare agent label on the linked issue is a topic tag applied by label - // automation, not a dispatch. It only asserts agent work when the issue - // actually declares the contract the gate goes on to require (#1130). - ['issue label without contract', base, issue(1, ['mcp/agent']), false], - ['issue label with contract', base, issue(1, ['mcp/agent'], CONTRACT), true], - // Pull-side provenance identifies the producer; it does not supply a - // contract to score against. With no linked issue there is no intent - // snapshot, so `policy.agent_login`, `policy.run_id` and `issue.number` - // can never be populated and the verdict is permanently `invalid_payload`. - // Each of these arms the gate only once an issue exists to verify against. - ['PR label, no issue', {...base, labels: [{name: 'agent-task'}]}, null, false], - ['PR label + issue', {...base, labels: [{name: 'agent-task'}]}, PLAIN, true], - ['branch, no issue', {...base, head: {ref: 'codex/fix'}}, null, false], - ['branch + issue', {...base, head: {ref: 'codex/fix'}}, PLAIN, true], - ['manifest, no issue', - {...base, body: ''}, null, false], - ['manifest + issue', - {...base, body: ''}, PLAIN, true], - ['known agent, no issue', - {...base, user: {login: 'google-labs-jules[bot]'}}, null, false], - ['known agent + issue', - {...base, user: {login: 'google-labs-jules[bot]'}}, PLAIN, true], - ['dependabot excluded', { - ...base, - user: {login: 'dependabot[bot]'}, - labels: [{name: 'agent-task'}] - }, issue(1, ['agent-task'], CONTRACT), false] -]; -for (const [name, pull, selected, expected] of rows) { - const actual = agentTaskApplicable(pull, selected); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - for function in applicable_functions: - completed = subprocess.run( - ["node", "-e", function + applicable_assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - selection_functions = _javascript_functions( - workflow, - "function scheduledLinkProjection(", - ) - self.assertEqual(len(selection_functions), 1) - selection_assertions = r""" -const issue = number => ({number}); -const rows = [ - ['valid contract', { - body: '\nFixes #7' - }, [issue(7)], 7, []], - ['multi authoritative uses textual first', { - body: 'Fixes #7\nFixes #8' - }, [issue(7), issue(8)], 7, [ - 'incomplete_linked_issue_contract', - 'missing_closing_issue_reference', - 'multiple_closing_issues' - ]], - ['textual non-authoritative fallback', { - body: 'Fixes #9' - }, [], 9, [ - 'incomplete_linked_issue_contract', - 'missing_closing_issue_reference' - ]], - ['manifest wins', { - body: '\nFixes #9' - }, [issue(9)], 10, ['conflicting_linked_issue']], - ['invalid manifest falls back', { - body: '\nFixes #11' - }, [], 11, [ - 'incomplete_linked_issue_contract', - 'invalid_agent_lock_manifest', - 'missing_closing_issue_reference' - ]], - ['no issue', {body: ''}, [], 0, [ - 'incomplete_linked_issue_contract', - 'missing_closing_issue_reference' - ]] -]; -for (const [name, pull, issues, expectedNumber, expectedErrors] of rows) { - const actual = scheduledLinkProjection(pull, issues); - if (actual.selectedIssueNumber !== expectedNumber || - JSON.stringify(actual.errors) !== JSON.stringify(expectedErrors)) { - throw new Error(`${name}: ${JSON.stringify(actual)}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", selection_functions[0] + selection_assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - combined_assertions = r""" -const first = {number: 1, labels: {nodes: []}, body: ''}; -const second = { - number: 2, - labels: {nodes: [{name: 'agent-task'}]}, - body: '### Agent Login\n\n`google-labs-jules[bot]`\n\n' + - '### Agent Run ID\n\n`run-42`' -}; -const multi = { - user: {login: 'maintainer'}, - head: {ref: 'feature/ordinary'}, - labels: [], - body: 'Fixes #1\nFixes #2' -}; -const selected = scheduledLinkProjection(multi, [first, second]); -if (selected.selectedIssueNumber !== 1 || - agentTaskApplicable(multi, first) !== false) { - throw new Error('multi-link applicability must use selected issue only'); -} -const textual = {...multi, body: 'Fixes #2'}; -if (scheduledLinkProjection(textual, []).selectedIssueNumber !== 2 || - agentTaskApplicable(textual, second) !== true) { - throw new Error('textual fallback issue label must affect applicability'); -} -""" - completed = subprocess.run( - [ - "node", - "-e", - selection_functions[0] + applicable_functions[0] + - combined_assertions, - ], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - policy_functions = _javascript_functions( - workflow, - "function scheduledMutablePolicyErrors(", - ) - self.assertEqual(len(policy_functions), 1) - policy_assertions = r""" -const issue = { - body: '## Agent login\nexample-agent[bot]\n' + - '## Agent run id\nrun-1' -}; -const manifest = ''; -const base = { - title: 'ci: enforce agent truth gate', - draft: false, - labels: [{name: 'copilot-rabbit'}], - body: manifest + '\nFixes #870' -}; -const rows = [ - ['valid', base, issue, true, []], - ['draft', {...base, draft: true}, issue, true, ['draft_pr']], - ['invalid title', {...base, title: 'bad'}, issue, true, - ['invalid_pr_title']], - ['label removed', {...base, labels: []}, issue, true, - ['missing_copilot_rabbit_label']], - ['invalid manifest', { - ...base, - body: '\nFixes #870' - }, issue, true, [ - 'agent_login_mismatch', - 'agent_run_id_mismatch', - 'invalid_agent_lock_manifest' - ]], - ['run mismatch', { - ...base, - body: '\nFixes #870' - }, issue, true, ['agent_run_id_mismatch']], - ['login mismatch', { - ...base, - body: '\nFixes #870' - }, issue, true, ['agent_login_mismatch']], - ['missing headings', base, {body: ''}, true, [ - 'missing_agent_login', - 'missing_agent_run_id' - ]], - ['not applicable', {...base, draft: true, labels: []}, issue, false, []] -]; -for (const [name, pull, selected, applicable, expected] of rows) { - const actual = scheduledMutablePolicyErrors( - pull, selected, applicable - ); - if (JSON.stringify(actual) !== JSON.stringify(expected)) { - throw new Error(`${name}: ${JSON.stringify(actual)} !== ` + - JSON.stringify(expected)); - } -} -""" - completed = subprocess.run( - ["node", "-e", policy_functions[0] + policy_assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - policy_projection_functions = _javascript_functions( - workflow, - "function scheduledMutablePolicyProjectionChanged(", - ) - self.assertEqual(len(policy_projection_functions), 1) - policy_projection_assertions = r""" -const ready = {notApplicable: false, mutablePolicyErrors: []}; -const blocked = { - notApplicable: false, - mutablePolicyErrors: ['missing_copilot_rabbit_label'] -}; -const rows = [ - ['success to violation', ready, - ['missing_copilot_rabbit_label'], true, true], - ['settled violation', blocked, - ['missing_copilot_rabbit_label'], true, false], - ['violation repaired', blocked, [], true, true], - ['not applicable', {notApplicable: true, mutablePolicyErrors: []}, - ['draft_pr'], true, false], - ['currently exempt', ready, ['draft_pr'], false, false], - ['hard unknown failure', { - notApplicable: false, - projectionUsable: false, - mutablePolicyErrors: [] - }, ['missing_copilot_rabbit_label'], true, false] -]; -for (const [name, previous, current, applicable, expected] of rows) { - const actual = scheduledMutablePolicyProjectionChanged( - previous, current, applicable - ); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - [ - "node", - "-e", - policy_projection_functions[0] + - policy_projection_assertions, - ], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - identity_functions = _javascript_functions( - workflow, - "function scheduledContractIdentity(", - ) - identity_projection_functions = _javascript_functions( - workflow, - "function scheduledIdentityProjectionChanged(", - ) - self.assertEqual(len(identity_functions), 1) - self.assertEqual(len(identity_projection_functions), 1) - identity_assertions = r""" -const issueA = { - body: '## Agent login\nexample-agent[bot]\n' + - '## Agent run id\nrun-a' -}; -const issueB = { - body: '## Agent login\nexample-agent[bot]\n' + - '## Agent run id\nrun-b' -}; -const identityA = scheduledContractIdentity(870, issueA, true); -const identityB = scheduledContractIdentity(871, issueB, true); -const allNull = scheduledContractIdentity(0, null, true); -const missingLogin = scheduledContractIdentity(870, { - body: '## Agent run id\nrun-a' -}, true); -const missingRun = scheduledContractIdentity(870, { - body: '## Agent login\nexample-agent[bot]' -}, true); -if (JSON.stringify(identityA) !== JSON.stringify({ - issue_number: 870, - agent_login: 'example-agent[bot]', - run_id: 'run-a' -}) || scheduledContractIdentity(870, issueA, false) !== null) { - throw new Error('contract identity extraction failed'); -} -const previous = { - notApplicable: false, - projectionUsable: true, - identityProjection: identityA -}; -const rows = [ - ['unchanged', previous, identityA, true, false], - ['issue and run switched', previous, identityB, true, true], - ['missing baseline', {...previous, identityProjection: null}, - identityA, true, true], - ['settled all-null identity', {...previous, identityProjection: allNull}, - allNull, true, false], - ['settled missing login', {...previous, identityProjection: missingLogin}, - missingLogin, true, false], - ['settled missing run', {...previous, identityProjection: missingRun}, - missingRun, true, false], - ['partial identity repaired', {...previous, identityProjection: missingRun}, - identityA, true, true], - ['hard unknown', {...previous, projectionUsable: false}, - identityB, true, false], - ['not applicable', {...previous, notApplicable: true}, - identityB, true, false], - ['currently exempt', previous, null, false, false] -]; -for (const [name, prior, current, applicable, expected] of rows) { - const actual = scheduledIdentityProjectionChanged( - prior, current, applicable - ); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - [ - "node", - "-e", - identity_functions[0] + identity_projection_functions[0] + - identity_assertions, - ], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_validation_replaces_obsolete_failure_comment(self): - workflow = self._workflow() - validate = workflow[ - workflow.index(" validate:"): - workflow.index(" truth-gate:") - ] - - self.assertIn("✅ Current validation passed.", validate) - self.assertIn("comment.user.login === 'github-actions[bot]'", validate) - self.assertLess( - validate.index("const marker = ''"), - validate.index("if (findings.length === 0)"), - ) - self.assertIn("issues.updateComment", validate) - - def test_commented_review_does_not_clear_changes_requested(self): - workflow = self._workflow() - - self.assertIn( - "['APPROVED', 'CHANGES_REQUESTED', 'DISMISSED'].includes(", - workflow, - ) - self.assertIn("reviewDecision", workflow) - - def test_workflow_requires_copilot_review_label_and_committed_tests(self): - workflow = self._workflow() - - self.assertIn("copilot-pull-request-reviewer[bot]", workflow) - self.assertIn("function isCurrentCopilotReview(", workflow) - self.assertIn("copilot-rabbit", workflow) - self.assertNotIn("!thread.isOutdated", workflow) - self.assertIn( - "copilot_current_head_reviewed: copilotCurrentHeadReviewed", - workflow, - ) - self.assertIn("copilot_rabbit_label: copilotRabbitLabel", workflow) - self.assertIn("expectedTests.every", workflow) - self.assertIn("presentChangedFiles.has(path)", workflow) - - def test_adapter_fails_closed_on_pagination_and_binds_trusted_ci(self): - workflow = self._workflow() - - self.assertIn("changed_files_truncated", workflow) - self.assertIn("run.path === '.github/workflows/ci.yml'", workflow) - self.assertIn("run.event === expectedEvent", workflow) - self.assertIn("pull.number === prNumber", workflow) - self.assertIn("trusted_ci_workflow_changed", workflow) - self.assertIn("file.previous_filename", workflow) - self.assertIn("file.status !== 'removed'", workflow) - self.assertIn( - "present_changed_files: [...presentChangedFiles]", - workflow, - ) - - def test_gate_runs_are_serialized_and_coalesced_by_pr_number(self): - workflow = self._workflow() - dispatch = workflow[ - workflow.index(" dispatch-evidence-refresh:"): - workflow.index(" validate:") - ] - truth_gate_header = workflow[ - workflow.index(" truth-gate:"): - workflow.index(" steps:", workflow.index(" truth-gate:")) - ] - - self.assertIn("dispatch-evidence-refresh:", workflow) - self.assertIn("group: agent-completion-${{", workflow) - self.assertIn("inputs.pull_request || github.event.pull_request.number", workflow) - self.assertIn("cancel-in-progress: false", truth_gate_header) - self.assertNotIn("cancel-in-progress: true", truth_gate_header) - self.assertIn("timeout-minutes: 20", truth_gate_header) - self.assertIn("trustedCommentAssociations", workflow) - self.assertIn("comment.author_association", workflow) - self.assertIn("google-labs-jules[bot]", workflow) - self.assertIn("getCollaboratorPermissionLevel", dispatch) - self.assertIn("context.payload.sender", dispatch) - self.assertNotIn("issue.author_association", dispatch) - self.assertIn("!cancelled()", truth_gate_header) - self.assertNotIn("always()", truth_gate_header) - - def test_collector_binds_structured_agent_event_to_head(self): - workflow = self._workflow() - - self.assertIn("agent-lock-event", workflow) - self.assertIn("head_sha: pr.head.sha", workflow) - self.assertIn("structuredEvent.head_sha", workflow) - self.assertIn("ready for (?:a )?review", workflow) - self.assertIn("\\[PR\\]", workflow) - - def test_legacy_run_id_requires_an_unambiguous_delimiter(self): - workflow = self._workflow() - functions = _javascript_functions( - workflow, - "function legacyRunId(", - ) - self.assertEqual(len(functions), 1) - - assertions = r""" -const rows = [ - ['Run failed', null], - ['failed to complete task because the worker stopped', null], - ['run id: provider-123', 'provider-123'], - ['run_id=provider.456', 'provider.456'], - ['task-id/abc:789', 'abc:789'], - ['https://jules.google.com/task/1892762060881911102', - '1892762060881911102'], - ['https://example.test/tasks/task-42.', 'task-42'], - ['Run: failed', null], - ['Task: failed', null], - ['runner:wrong', null], - ['', null] -]; -for (const [body, expected] of rows) { - const actual = legacyRunId(body); - if (actual !== expected) { - throw new Error(`${body}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", functions[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_collector_uses_authoritative_closing_issue_references(self): - workflow = self._workflow() - - self.assertIn("closingIssuesReferences", workflow) - self.assertIn("incomplete_linked_issue_contract", workflow) - self.assertIn("multiple_closing_issues", workflow) - - def test_snapshot_validates_contract_and_freezes_approval_state(self): - workflow = self._workflow() - - self.assertIn("pre-dispatch confirmation", workflow.lower()) - self.assertIn("incomplete_agent_task_contract", workflow) - self.assertIn("scope_unrestricted_approved", workflow) - self.assertIn("intent_snapshot_after_dispatch", workflow) - self.assertIn("function hasResponse(value)", workflow) - self.assertIn("response !== '_No response_'", workflow) - self.assertIn( - "Boolean(hasResponse(declaredScope) || unrestrictedRequested)", - workflow, - ) - - def test_adapter_recognizes_agent_labels_scripts_and_blocking_threads(self): - workflow = self._workflow() - - self.assertIn("'agenttask'", workflow) - self.assertIn("explicitlyNonBehavioral", workflow) - self.assertIn("!explicitlyNonBehavioral", workflow) - self.assertIn("VADE-RECOMMENDATION", workflow) - - def test_agent_applicability_requires_provenance_or_declared_contract(self): - """A bare `agent-task` label on a linked issue is not a dispatch. - - Label automation applies `agent-task` and `mcp/agent` as topic tags to - issues that were never created from the agent task template. Before - this guard the gate unioned pull request labels with linked issue - labels, so any pull request closing such an issue was judged an agent - completion and then measured against a contract the issue had never - declared -- producing a permanent ``blocked``/``invalid_payload`` - verdict that no author could satisfy (#1130). - """ - - workflow = self._workflow() - applicable = _javascript_functions( - workflow, - "function agentTaskApplicable(", - ) - self.assertEqual(len(applicable), 2) - - assertions = r""" -const CONTRACT = [ - '### Agent Login', - '', - '`google-labs-jules[bot]`', - '', - '### Agent Run ID', - '', - '`run-42`' -].join('\n'); -const human = { - user: {login: 'groupthinking'}, - head: {ref: 'feature/thing'}, - labels: [], - body: 'Fixes #7' -}; -const rows = [ - // The regression this guard exists for: a human pull request closing an - // issue that automation mislabelled `agent-task` without a contract. - ['mislabelled linked issue', { - user: {login: 'groupthinking'}, - head: {ref: 'fix/agentic-workflow-noop-terminal-state-1091'}, - labels: [{name: 'documentation'}, {name: 'ci/cd'}], - body: 'Fixes #1091' - }, { - number: 1091, - labels: [{name: 'agent-task'}, {name: 'bug'}], - body: '## Summary\nSomething broke.\n' - }, false], - // A genuine issue-side dispatch is still gated. - ['declared contract', human, { - number: 7, labels: [{name: 'agent-task'}], body: CONTRACT - }, true], - ['contract via graphql label nodes', human, { - number: 7, labels: {nodes: [{name: 'agent-task'}]}, body: CONTRACT - }, true], - // Only the two contract labels arm the issue side. The generic `agent` - // label is never recognised by the snapshot job or the collector, so an - // issue carrying it (even with contract headings) must not arm the gate: - // it would be permanently blocked as linked_issue_not_agent_task with no - // snapshot to satisfy. - ['generic agent label on issue', human, { - number: 7, labels: [{name: 'agent'}], body: CONTRACT - }, false], - // Unfilled issue form fields render as the placeholder, not a contract. - ['no response placeholder', human, { - number: 7, - labels: [{name: 'agent-task'}], - body: '### Agent Login\n\n_No response_\n\n' + - '### Agent Run ID\n\n_No response_\n' - }, false], - ['half declared contract', human, { - number: 7, - labels: [{name: 'agent-task'}], - body: '### Agent Login\n\n`jules`\n' - }, false], - ['unlabelled issue with contract', human, { - number: 7, labels: [{name: 'bug'}], body: CONTRACT - }, false], - // Pull request provenance applies against a linked issue -- an agent - // producing work against a contract-less issue is still applicable, and - // therefore still blocked. - ['known agent author', { - user: {login: 'google-labs-jules[bot]'}, - head: {ref: 'feature/thing'}, labels: [], body: '' - }, {number: 7, labels: [{name: 'agent-task'}], body: '## Summary\n'}, true], - // ...but with no linked issue there is no intent snapshot and no declared - // run id or login, so the required payload fields are unsatisfiable and the - // gate would block permanently rather than ever reaching a verdict. These - // are `not_applicable`, not violations. - ['agent branch prefix, no issue', { - user: {login: 'groupthinking'}, - head: {ref: 'jules/thing'}, labels: [], body: '' - }, null, false], - ['agent branch prefix with issue', { - user: {login: 'groupthinking'}, - head: {ref: 'jules/thing'}, labels: [], body: '' - }, {number: 7, labels: [], body: ''}, true], - ['agent label on the pull request, no issue', { - user: {login: 'groupthinking'}, - head: {ref: 'feature/thing'}, - labels: [{name: 'agent'}], body: '' - }, null, false], - ['lock manifest in the pull request body, no issue', { - user: {login: 'groupthinking'}, - head: {ref: 'feature/thing'}, - labels: [], - body: '' - }, null, false], - // Dependabot is excluded regardless of every other signal. - ['dependabot', { - user: {login: 'dependabot[bot]'}, - head: {ref: 'jules/bump'}, - labels: [{name: 'agent-task'}], body: '' - }, {number: 7, labels: [{name: 'agent-task'}], body: CONTRACT}, false], - ['plain human pull request', human, { - number: 7, labels: [{name: 'bug'}], body: '' - }, false], - ['no linked issue', human, null, false] -]; -for (const [name, pull, issue, expected] of rows) { - const actual = agentTaskApplicable(pull, issue); - if (actual !== expected) { - throw new Error(`${name}: ${actual} !== ${expected}`); - } -} -""" - completed = subprocess.run( - ["node", "-e", applicable[0] + assertions], - check=False, - capture_output=True, - text=True, - ) - self.assertEqual(completed.returncode, 0, completed.stderr) - - def test_agent_applicability_copies_stay_identical(self): - """The scheduled sweep and the per-pull-request collector must agree. - - Both jobs publish to the same commit status context, so divergent - applicability logic would let one job block what the other skips. The - contract check is nested inside ``agentTaskApplicable`` so the - function stays self-contained for the ``node -e`` extraction harness - above. - """ - - workflow = self._workflow() - copies = _javascript_functions(workflow, "function agentTaskApplicable(") - self.assertEqual(len(copies), 2) - self.assertEqual(copies[0], copies[1]) - self.assertIn("function declaresAgentContract(", copies[0]) - - def test_mislabelled_agent_task_is_reported_rather_than_blocked(self): - """The mislabel is surfaced as an annotation, never as a gate reason. - - ``agent_completion_gate.evaluate`` short-circuits to - ``not_applicable`` before it reads ``collection_errors``, so a - diagnostic pushed there would be silently discarded. It is emitted - with ``core.notice`` instead, and it is keyed on the missing contract - itself rather than on inapplicability, so pull requests that are - inapplicable for unrelated reasons (Dependabot's unconditional - exclusion) never receive a notice falsely claiming a declared - contract is missing. - """ - - workflow = self._workflow() - - self.assertIn("mislabelled_agent_task", workflow) - self.assertNotIn( - "collectionErrors.push('mislabelled_agent_task')", - workflow, - ) - notice = workflow[workflow.index("mislabelled_agent_task") - 600:] - self.assertIn("core.notice(", notice[:800]) - self.assertIn("!contractDeclared", notice[:800]) - self.assertNotIn("!applicable", notice[:800]) - - def test_workflow_does_not_approve_or_merge(self): - workflow = self._workflow() - - self.assertNotIn("createReview", workflow) - self.assertNotIn("pulls.merge", workflow) - self.assertNotIn("mergePullRequest", workflow) - self.assertNotIn("event: 'APPROVE'", workflow) - - -class CompletionGateDocumentationTests(unittest.TestCase): - def test_agent_task_template_requires_pre_dispatch_evidence(self): - template = ( - _repo_root() / ".github" / "ISSUE_TEMPLATE" / "agent-task.yml" - ).read_text(encoding="utf-8") - - for field in ( - "Objective", - "Acceptance criteria", - "Declared file scope", - "Focused test paths", - ): - self.assertIn(field, template) - declared_scope = template[ - template.index(" id: declared_scope"): - template.index(" id: allowed_extra") - ] - self.assertIn("validations:", declared_scope) - self.assertIn("required: true", declared_scope) - self.assertNotIn("id: unrestricted", template) - self.assertNotIn("scope-unrestricted-approved", template) - - def test_pr_template_uses_an_inert_example_marker(self): - template = ( - _repo_root() / ".github" / "pull_request_template.md" - ).read_text(encoding="utf-8") - - self.assertIn("agent-lock-example", template) - self.assertNotIn("