fix(ci): make the Dependabot merge gate able to merge, and gate it on real CI - #1487
Conversation
… real CI
The `merge` job in dependabot-auto-merge.yml had two independent defects.
Defect 1 — it could never merge. The job read the semver impact from
`GET /pulls/{n}` behind a `dorian` media-type preview:
metadata.data?.dependency?.update_type
The pulls schema carries no dependency metadata, and `dorian` gated draft
PRs, not Dependabot fields. `updateType` was always undefined, so every PR
hit the "could not determine update type" branch. Read it instead from the
`updated-dependencies` trailer Dependabot writes into the head commit
message — the same source `dependabot/fetch-metadata` uses, which exists
precisely because there is no such API field. All entries are checked, so a
grouped update containing one major is blocked.
Defect 2 — the readiness gate read a surface with no CI in it.
`getCombinedStatusForRef` returns only legacy commit statuses; every gate in
MERGE_POLICY.md gate 2 is a check run. On a typical Dependabot head the
combined status is `success` off two Vercel statuses while build/test/guards
are still queued, so this would merge to protected main with CI unfinished —
or red. Now scans check runs too: unfinished blocks, non-success/skipped/
neutral blocks, commit statuses still checked as before.
Defect 1 was masking defect 2. Fixing the update-type lookup alone would
have armed unsafe merges, so both are fixed together.
Adds the `checks: read` permission the readiness scan needs, without which
`checks.listForRef` 403s, and excludes this workflow's own approve/merge
check runs so the gate cannot deadlock against itself.
The existing tests asserted the script *contained* "pulls.merge" and
"semver-major" — both true of the no-op version. Replaced with behavioural
tests that extract the script and execute it against a stubbed octokit:
8 of them fail against the previous workflow and pass against this one.
No behaviour change today: `vars.DEPENDABOT_AUTO_MERGE_ENABLED` is not
'true', and this commit deliberately does not set it.
Closes #1476
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe workflow now classifies Dependabot updates from head commit messages and fails closed for unsupported types. It also checks required GitHub check runs, excludes its own jobs, and blocks merging until checks complete successfully. ChangesDependabot auto-merge
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Dependabot
participant Workflow
participant GitHubCommits
participant GitHubChecks
participant GitHubPulls
Dependabot->>Workflow: Open dependency update pull request
Workflow->>GitHubCommits: Read head commit message
GitHubCommits-->>Workflow: Return update-type entries
Workflow->>GitHubChecks: List latest check runs
GitHubChecks-->>Workflow: Return required check statuses
Workflow->>GitHubPulls: Merge eligible pull request
Suggested reviewers: ✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review Auto-review skipped again with "Auto reviews are limited based on label configuration". That is the #1424 defect, not a missing label: This change modifies a workflow that can merge to protected
One thing I would rather you catch than take on trust: defect 1 was masking defect 2. The old code never resolved an update type, so it never reached the status check — which is the only reason it was not merging PRs with red CI. If my reasoning there is wrong, then the severity framing in the PR body is wrong too. Note Generated by Claude Code |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Changed files and workflow diff =='
git diff --name-status origin/main...HEAD
git diff -- .github/workflows/dependabot-auto-merge.yml
echo '== Workflow outline and content =='
ast-grep outline .github/workflows/dependabot-auto-merge.yml --items all || true
cat -n .github/workflows/dependabot-auto-merge.yml
echo '== Candidate supporting files =='
fd -a 'test_dependabot_automation_workflow.py|dependabot_merge_gate_driver.js|coderabbit\.ya?ml|MERGE_POLICY\.md' . -x sh -c 'echo "--- $1"; cat -n "$1"' sh {} \;
echo '== Workflow references and local workflow job names =='
rg -n -C 3 --glob '*.yml' --glob '*.yaml' \
'(dependabot-auto-merge|^\s*(approve|merge):|updated-dependencies|listForRef|getCombinedStatusForRef)' \
.github .coderabbit.yaml .coderabbit.yml 2>/dev/null || true
echo '== PR metadata and changed-file list =='
gh pr view 1487 --repo groupthinking/EventRelay --json number,headRefOid,baseRefName,files,url \
--jq '{number, headRefOid, baseRefName, url, files: [.files[].path]}'Length of output: 30291 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Merge base and exact changed workflow diff =='
base="$(git merge-base origin/main HEAD)"
printf 'merge-base: %s\n' "$base"
git diff --no-ext-diff --unified=80 "$base" HEAD -- .github/workflows/dependabot-auto-merge.yml
echo '== Changed fixture and unit test =='
for f in \
tests/fixtures/dependabot_merge_gate_driver.js \
tests/unit/test_dependabot_automation_workflow.py
do
echo "--- $f"
cat -n "$f"
done
echo '== Current checks and status for the PR head =='
sha="$(gh pr view 1487 --repo groupthinking/EventRelay --json headRefOid --jq .headRefOid)"
gh api "repos/groupthinking/EventRelay/commits/$sha/check-runs?per_page=100" \
--jq '{total_count, check_runs: [.check_runs[] | {id, name, status, conclusion, app: .app.slug, details_url}]}'
gh api "repos/groupthinking/EventRelay/commits/$sha/status" \
--jq '{state, total_count, statuses: [.statuses[] | {context, state, creator: .creator.login}]}'
echo '== Existing workflow job names that collide with approve or merge =='
rg -n -U --glob '*.yml' --glob '*.yaml' \
'^\s*(name:\s*(approve|merge)\s*$|(?:approve|merge):\s*(?:#.*)?$)' \
.github/workflows || trueLength of output: 12302 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo '== Changed paths present in HEAD =='
git ls-tree -r --name-only HEAD | rg -n \
'(^\.github/workflows/dependabot-auto-merge\.yml$|dependabot_merge_gate_driver\.js$|test_dependabot_automation_workflow\.py$)' || true
echo '== Fixture blob at HEAD =='
git show HEAD:tests/fixtures/dependabot_merge_gate_driver.js 2>&1 || true
echo '== Unit-test blob at HEAD =='
git show HEAD:tests/unit/test_dependabot_automation_workflow.py 2>&1 || true
echo '== PR changed-file metadata =='
gh api 'repos/groupthinking/EventRelay/pulls/1487/files?per_page=100' \
--jq '.[] | {filename, status, sha, previous_filename, additions, deletions}'
sha="$(gh pr view 1487 --repo groupthinking/EventRelay --json headRefOid --jq .headRefOid)"
echo '== Check runs on PR head =='
gh api "repos/groupthinking/EventRelay/commits/$sha/check-runs?per_page=100" \
--jq '{total_count, check_runs: [.check_runs[] | {id, name, status, conclusion, app: .app.slug}]}'
echo '== Combined commit status on PR head =='
gh api "repos/groupthinking/EventRelay/commits/$sha/status" \
--jq '{state, total_count, statuses: [.statuses[] | {context, state, creator: .creator.login}]}'
echo '== Check suites associated with PR head =='
gh api "repos/groupthinking/EventRelay/commits/$sha/check-suites?per_page=100" \
--jq '{total_count, check_suites: [.check_suites[] | {id, status, conclusion, app: .app.slug, head_sha, latest_check_runs_count}]}'Length of output: 24733
The trailer is the correct source for the The commit-status gate remains active. A non- The The severity framing is correct. The old undefined
|
… collisions Three fail-open holes in the merge gate from the previous commit. Two were raised by CodeRabbit; the required-checks one I raised against my own diff and fixed more strictly than the minimum. 1. An empty check-run set merged. `unfinished` and `failed` are both empty when no relevant check runs exist, so the gate merged on no CI at all. This is reachable: the job fires on a *completed check suite*, and the first suite to complete can be this workflow's own, before other workflows have created their check runs. Rather than only blocking the empty case, require every check MERGE_POLICY.md gate 2 lists as "required for every pull request" to be present -- absence is not success, and a partial list was just as unsafe as an empty one. Also passes `filter: 'latest'` so a superseded failure from a re-run cannot block a head that is now green. 2. The update-type gate denied major rather than allowing patch/minor, so any value it did not recognise passed -- a new Dependabot update kind, or a value mangled by future parser drift. Now an explicit allowlist of semver-patch/semver-minor, with every entry of a grouped update required to qualify. 3. The self-exclusion matched the bare names `approve` and `merge`, which would also swallow a failing job of that name from an unrelated workflow. Renamed this workflow's jobs to `dependabot-auto-merge-approve` and `dependabot-auto-merge-merge` and narrowed the exclusion to those. Nothing outside the tests referenced the old names, and neither is in branch protection. Drops the test assertion that the merge script contains "semver-major". It was true of the version that could merge nothing, and stopped being true when the policy got stricter -- it tracked wording, not behaviour, which is the defect #1476 calls out. The behavioural tests cover the semantics. +4 behavioural tests (16 total). Against the previous workflow: 15 fail.
All three findings fixed in
|
Correction: there is a competing PR — #1478. My handoff checkbox is wrong.This PR's body claims "No competing PR implements the same issue — searched open PRs for Flagging rather than quietly letting This is not the only collision
Comparison, so the choice is informed rather than first-comeI have not closed anything. Both are good-faith reads of #1476 and they agree on the diagnosis; they differ in what they do about it.
The first three rows are the substantive ones. #1478 carries the same three fail-open holes this PR had at The one place #1478 is more capable is its title-fallback classifier, which recovers an update type when the trailer omits it. I deliberately skip that case instead. I think skipping is right — inferring semver intent from Recommendation: take this PR's gating logic and, if the title fallback is wanted, add it here as a follow-up. I am not asserting that as a decision — it is a competing-implementation call, which the runbook routes to a human, and no automated tiebreak should close another session's work.
Generated by Claude Code |
Competing implementation: #1478Cross-link for reviewers. #1478 is an independent fix for the same issue (#1476), opened four minutes before this PR by a parallel session of the same routine. This PR's handoff says "searched open PRs for The two rewrite the same hunks of This PR is the superset: 12 behavioural tests driving the extracted job script against a stubbed octokit, versus #1478's 5 substring assertions. Recommend consolidating here and closing #1478. One thing worth porting across before doing so: #1478 falls back to the major components in the PR title when the commit block names exactly one dependency and carries no No action taken on either PR. Generated by Claude Code |
Duplicate cluster: #1478 ⊂ #1487Flagging from an unattended PR-remediation sweep. #1478 and this PR change the same two files (
This PR is the broader of the two. Beyond #1478 it adds: workflow-qualified job names (so the self-exclusion cannot swallow an unrelated One genuine behavioural divergence, worth a decision rather than a coin-flip:
#1487's is the conservative read — inferring semver impact from a title is a weak signal on a job whose failure mode is an unreviewed merge to protected Suggested resolution: keep this PR as canonical, close #1478 as superseded, and settle the title-fallback question explicitly on this one. Flagging rather than closing #1478 myself — the divergence above is a policy call, not a mechanical dedup. Generated by Claude Code |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
github-actions
Bot
added
ci/cd
javascript
🔍 PR Validation |
Correction: the competing-PR gate has not run on either PR, and will not while both are draftsI said above that #1478 and this PR are "currently blocking each other" via
if (pr.draft) {
await publish("neutral", "Governance deferred for draft PR", ...);
return;
}The competing-PR scan lives well below that What misled me is worth flagging on its own: the two check runs here do not mean what their names suggest. The job is named What this changes
I am not proposing a fix to that here; it is out of scope for #1476 and belongs to whoever owns the fleet-coordination problem. Recording it because the seven-way #1452 collision is otherwise hard to explain, and "the gate would have caught it" is not the explanation. Everything else in my earlier comparison stands — including the three fail-open holes #1478 still carries. Only the claim about when the gate bites was wrong. CI on
|
The open risk recorded when this PR was written -- that the merge gate hard-codes gate 2's check list and will drift from the policy it claims to enforce -- materialised in three weeks. #1449/#1480 made `test-frontend` a required check and updated MERGE_POLICY.md, but nothing pointed at this gate. Its list still ended at `test`, which runs Python pytest only, so it would have merged a Dependabot PR with the apps/web vitest suite unrun -- including the CWE-209 and billing-disclosure regressions that suite carries. That is the same fail-open class this PR exists to close, arriving through drift rather than through a coding error. Adds `test-frontend` to REQUIRED_CHECKS, and adds the test I previously declined to write. I argued then that parsing the policy prose was "its own small parser with its own failure mode"; having now seen the drift happen, that trade is clearly wrong -- the parse is one regex over a stable sentence, and the alternative is a gate that silently under-enforces. The test fails with the exact missing names, so the next addition to gate 2 lands here instead of in production. Also merges origin/main (56 commits), which brings the concurrency group added to this workflow by #1510. Verified present after the merge. 17 tests pass. Removing `test-frontend` from the gate fails the new test with `missing from gate: ['test-frontend']`.
|
@coderabbitai review Incremental, for What changed: Worth attacking specifically:
Context for why this commit exists: the drift I flagged as a hypothetical risk when opening this PR actually happened.
Generated by Claude Code |
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
base="bdca4c5"
head="877da4c5"
echo '== Commit range =='
git log --oneline --decorate "${base}..${head}"
echo
echo '== Changed files =='
git diff --name-status "${base}" "${head}"
echo
echo '== Incremental diff: workflow, test, policy =='
git diff --no-ext-diff --unified=100 "${base}" "${head}" -- \
.github/workflows/dependabot-auto-merge.yml \
tests/unit/test_dependabot_automation_workflow.py \
MERGE_POLICY.md
echo
echo '== Workflow outline =='
ast-grep outline .github/workflows/dependabot-auto-merge.yml --items all || true
echo
echo '== Current workflow with line numbers =='
cat -n .github/workflows/dependabot-auto-merge.yml
echo
echo '== Current test file with line numbers =='
cat -n tests/unit/test_dependabot_automation_workflow.py
echo
echo '== Policy locations and surrounding content =='
rg -n -C 12 --fixed-strings 'Required for every pull request:' MERGE_POLICY.md
rg -n -C 6 --fixed-strings 'Generate and Upload Coverage' MERGE_POLICY.md
rg -n -C 6 --fixed-strings 'E2E Pipeline Tests' MERGE_POLICY.md
echo
echo '== Workflow trigger/concurrency keys and required checks =='
rg -n -C 4 '^(on:|concurrency:|permissions:|jobs:)|check_suite:|head_sha|REQUIRED_CHECKS|test-frontend' \
.github/workflows/dependabot-auto-merge.yml
echo
echo '== Exact objects in the policy section, including punctuation and backtick spans =='
python3 - <<'PY'
from pathlib import Path
import re
policy = Path("MERGE_POLICY.md").read_text()
for m in re.finditer(r"Required for every pull request:", policy):
start = max(0, m.start() - 120)
end = min(len(policy), m.start() + 2500)
print(repr(policy[start:end]))
print("---")
PY
echo '== PR head and live workflow check names =='
sha="$(gh pr view 1487 --repo groupthinking/EventRelay --json headRefOid --jq .headRefOid)"
printf 'PR head: %s\n' "$sha"
gh api "repos/groupthinking/EventRelay/commits/$sha/check-runs?per_page=100" \
--jq '.check_runs[] | {name, status, conclusion, app: .app.slug}' || trueLength of output: 74413 🏁 Script executed: #!/bin/bash
set -euo pipefail
head="877da4c5"
base="bdca4c5"
echo '== Locate relevant paths in HEAD tree =='
git ls-tree -r --name-only "$head" | rg \
'(^|/)(test_dependabot_automation_workflow\.py|dependabot-auto-merge\.yml|MERGE_POLICY\.md)$' || true
echo
echo '== Test blob from HEAD =='
git show "$head:tests/unit/test_dependabot_automation_workflow.py" | nl -ba | sed -n '1,560p'
echo
echo '== Policy blob from HEAD: required-check section through conditional section =='
git show "$head:MERGE_POLICY.md" | nl -ba | sed -n '55,155p'
echo
echo '== Evaluate the committed parity parser against the HEAD policy =='
git show "$head:MERGE_POLICY.md" > /tmp/merge-policy-1487.md
git show "$head:tests/unit/test_dependabot_automation_workflow.py" > /tmp/test-dependabot-1487.py
python3 - <<'PY'
from pathlib import Path
import re
policy = Path("/tmp/merge-policy-1487.md").read_text()
match = re.search(r"Required for every pull request:(.*?)\.\n", policy, re.S)
print("match_found:", bool(match))
if match:
print("matched_span:", match.span())
print("matched_text_repr:", repr(match.group(1)))
print("parsed_checks:", re.findall(r"`([^`]+)`", match.group(1)))
print("period_positions_after_anchor:", [
(m.start(), repr(policy[m.start()-35:m.start()+20]))
for m in re.finditer(r"\.", policy[match.start():match.end()+500])
][:20])
print("anchor_count:", policy.count("Required for every pull request:"))
PY
echo
echo '== Compare concurrency and trigger sections: origin/main parent vs HEAD =='
main_parent="$(git rev-parse 8dc737ae^)"
for ref in "$main_parent" 8dc737ae "$head"; do
echo "--- $ref"
git show "$ref:.github/workflows/dependabot-auto-merge.yml" |
nl -ba | sed -n '1,60p'
done
echo
echo '== Check for duplicate workflow keys and all auto-merge entry points =='
git show "$head:.github/workflows/dependabot-auto-merge.yml" > /tmp/dependabot-auto-merge-1487.yml
printf 'concurrency key count: '
grep -Ec '^concurrency:' /tmp/dependabot-auto-merge-1487.yml || true
printf 'check_suite trigger count: '
grep -Ec '^[[:space:]]+check_suite:' /tmp/dependabot-auto-merge-1487.yml || true
printf 'head_sha references: '
grep -Ec 'check_suite\.head_sha' /tmp/dependabot-auto-merge-1487.yml || true
rg -n -C 4 'enablePullRequestAutoMerge|pulls\.merge|update-type !=|ALLOWED_UPDATE_TYPES' \
/tmp/dependabot-auto-merge-1487.ymlLength of output: 2905 🤖 web_search executed:
Length of output: 13553 🏁 Script executed: #!/bin/bash
set -euo pipefail
head="877da4c5"
echo '== Committed parity test =='
git show "$head:tests/unit/test_dependabot_automation_workflow.py" | sed -n '430,560p'
echo
echo '== Policy required-check section =='
git show "$head:MERGE_POLICY.md" | sed -n '65,145p'
echo
echo '== Exact committed-parser result =='
git show "$head:MERGE_POLICY.md" > /tmp/merge-policy-1487.md
python3 - <<'PY'
from pathlib import Path
import re
policy = Path("/tmp/merge-policy-1487.md").read_text()
pattern = r"Required for every pull request:(.*?)\.\n"
match = re.search(pattern, policy, re.S)
print(f"anchor_count={policy.count('Required for every pull request:')}")
print(f"match_found={match is not None}")
if match:
print(f"span={match.span()}")
print("captured=" + repr(match.group(1)))
print("checks=" + repr(re.findall(r"`([^`]+)`", match.group(1))))
tail = policy[match.start():match.end() + 1000]
print("periods_after_anchor=" + repr([
(m.start(), tail[max(0, m.start()-40):m.start()+30])
for m in re.finditer(r"\.", tail)
]))
PY
echo
echo '== Concurrency section before `#1510`, at `#1510`, and at HEAD =='
parent="$(git rev-parse 8dc737ae^)"
for ref in "$parent" 8dc737ae "$head"; do
echo "--- $ref"
git show "$ref:.github/workflows/dependabot-auto-merge.yml" | sed -n '1,52p'
done
echo
echo '== Count and locations of semantic keys at HEAD =='
git show "$head:.github/workflows/dependabot-auto-merge.yml" > /tmp/dependabot-auto-merge-1487.yml
for pattern in '^concurrency:' '^[[:space:]]+check_suite:' 'check_suite\.head_sha' \
'enablePullRequestAutoMerge' 'github\.rest\.pulls\.merge'; do
echo "--- $pattern"
grep -nE "$pattern" /tmp/dependabot-auto-merge-1487.yml || true
done
echo
echo '== Current GitHub repository auto-merge and branch-protection configuration =='
gh api repos/groupthinking/EventRelay --jq \
'{allow_auto_merge, default_branch, archived, visibility}' || true
gh api repos/groupthinking/EventRelay/branches/main/protection/required_status_checks \
--jq '{strict, contexts, checks}' 2>&1 || trueLength of output: 19261
High — native auto-merge bypasses the patch/minor allowlist. The repository has This bypass also applies to grouped updates. The approval job does not verify that every grouped entry is patch or minor. Remove the The requested incremental changes are otherwise correct:
|
2 participants
Canonical issue
Closes #1476
Outcome
The
mergejob independabot-auto-merge.ymlcan merge a green patch/minor Dependabot PR — which it could not do before — and it only does so when the checks that actually carry this repo's CI have passed.Defect 1 — it could never merge
GET /pulls/{n}carries no dependency metadata, anddoriangated draft PRs, not Dependabot fields.updateTypewas alwaysundefined, so every PR hit thecould not determine update typebranch. Theapprovejob in the same file gets this right viadependabot/fetch-metadata@v3— an action that exists because there is no such API field.Fixed by reading the
updated-dependenciestrailer Dependabot writes into the head commit message (the same sourcefetch-metadataparses), then allowlistingsemver-patch/semver-minorrather than merely denying major, so an unrecognised value fails closed. Every entry of a grouped update must qualify.Defect 2 — the readiness gate read a surface with no CI in it
getCombinedStatusForRefreturns only legacy commit statuses. Every gate inMERGE_POLICY.mdgate 2 is a check run, which that endpoint cannot see. Observed live on #1459: combined statussuccessoff two Vercel statuses whiletest,build,guardsandtrivywere stillqueued.Now scans check runs as well: all 18 gate-2 checks must be present (absence is not success), none unfinished, none failed — with
skipped/neutralaccepted per gate 2's conditional list, andfilter: 'latest'so a superseded failure from a re-run cannot block a head that is now green. Commit statuses are still checked, as an addition rather than a replacement.The two are coupled
Defect 1 was masking defect 2. The obvious one-line fix — swap in
fetch-metadata— would have armed a gate that merges to protectedmainon a green that means nothing. That is why this PR does not fix them separately.Scope
dependabot-auto-merge.yml— trailer-based update-type resolution with a patch/minor allowlist; check-run readiness scan requiring all 18 gate-2 checks;checks: readpermission; jobs renamed todependabot-auto-merge-approve/-mergeso the self-exclusion cannot swallow an unrelated workflow'smerge.tests/unit/test_dependabot_automation_workflow.py— +14 tests, permissions assertion updated, one vocabulary assertion removed.tests/fixtures/dependabot_merge_gate_driver.js— new; runs the extracted job script against a stubbed octokit.vars.DEPENDABOT_AUTO_MERGE_ENABLED. Still not'true', and this PR does not set it. Both jobs remain skipped until a human flips it, so this change is inert on merge.approvejob. Correct already; untouched.MERGE_POLICY.mdadoption step 6. This makes the mechanism work; adopting it is a separate call.Risk
MERGE_POLICY.mdgains a check and this does not, the gate under-enforces silently. That already happened once (see Verification) and is now pinned by a test. The permissive failure modes present in the first revision — empty/partial check-run list, unrecognised update type, name collision — are closed and each has a test.git revert. The variable is off, so there is nothing live to roll back.Verification
Head
877da4c5. Measured, not inferred.Focused tests —
tests/unit/test_dependabot_automation_workflow.py: 17 passed.Non-vacuous. Against
origin/main's workflow the behavioural suite fails 15 of 16; the sole survivor is the eslint-ignore test, which touches none of this. Each guard has a test that fails when only that guard is removed:checkRuns: []and partial-list casestest-frontendrequiredmissing from gate: ['test-frontend']semver-unknownand quoted-value casesmergecaseBoth original defects reproduced against the old script, not inferred from reading it:
Defect 2 was isolated by force-feeding the old code the metadata it tries to read — the counterfactual showing that fixing defect 1 alone is unsafe.
The drift risk is not hypothetical. When this PR was opened I recorded that hard-coding gate 2's list would drift, and declined to test it on the grounds that parsing the policy prose was "its own small parser with its own failure mode". Three weeks later ci: no workflow runs the apps/web unit suite — 292 tests, including every CWE-209 regression test, gate nothing #1449/ci: run apps/web vitest as required test-frontend job (#1449) #1480 made
test-frontendrequired and updatedMERGE_POLICY.md; nothing pointed at this gate, whose list still ended attest— Python pytest only. It would have merged a Dependabot PR with the apps/web vitest suite unrun, including the CWE-209 and billing-disclosure regressions it carries. That judgement was wrong and is reversed:test-frontendis added, andtest_required_checks_match_merge_policynow pins the two together.Trailer parse checked against a real commit — the hono bump on build(deps): bump hono from 4.12.32 to 4.13.1 #1459 (
d5a4a43) carriesdependency-type: indirectand noupdate-type, which is why "cannot classify → skip" is the conservative branch rather than a guess.Rebased on current
main(56 commits), which brings the concurrency group fix(ci): add concurrency groups so the Actions queue can drain #1510 added to this workflow; verified present after the merge.Lint —
ruff checkclean on the changed test file. YAML parses; the extracted job script passesnode --check.Required CI — green on
bdca4c5:validate,guards,lint-python,lint-frontend,build,test,CodeQL,gitleaks (working tree),dependency-review,PR Governance,Canonical issue and evidence, both Security Scans,bandit,python-safety,npm-audit,trivy, coverage. Re-running on877da4c5.Review threads resolved — CodeRabbit reviewed and raised two fail-open findings; both fixed in
bdca4c5, along with a third I raised against my own diff. No open threads.Note on the wider suite
pytest tests/unit/reports collection errors in this sandbox (ModuleNotFoundError: No module named 'fastapi') — identical on cleanmainwith these changes stashed. The sandbox lacks the extras CI installs viapip install -e ".[dev,youtube]". Not caused by this change.Production evidence
Not applicable as a preview: a GitHub Actions workflow with no
apps/web/**surface, which is what gate 4 ofMERGE_POLICY.mdscopes previews to.The runtime evidence that matters is the execution transcript above — the real job script run in both directions against a stubbed octokit — plus the live check-run/commit-status divergence observed on #1459 and #1433, which is what makes defect 2 concrete rather than theoretical.
Agent handoff
mergejob can never merge, and its CI gate reads a surface with no CI in it #1476PR Governanceverified this on the current head once the PR left draft.mergejob can never merge, and its CI gate reads a surface with no CI in it #1476bdca4c5; re-running on877da4c5DEPENDABOT_AUTO_MERGE_ENABLED