Skip to content

ci: make the coverage ratchet survive a repo setting it cannot grant - #47

Open
sroberts wants to merge 1 commit into
mainfrom
ci/ratchet-visible-failure
Open

ci: make the coverage ratchet survive a repo setting it cannot grant#47
sroberts wants to merge 1 commit into
mainfrom
ci/ratchet-visible-failure

Conversation

@sroberts

Copy link
Copy Markdown
Owner

The finding

coverage-ratchet.yml has failed on every run since at least June 8 — ten consecutive weeks:

2026-08-10 failure    2026-06-29 failure
2026-08-03 failure    2026-06-22 failure
2026-07-27 failure    2026-06-15 failure
2026-07-20 failure    2026-06-08 failure
2026-07-13 failure    ...
2026-07-06 failure

Always the same step, always the same error:

##[error]GitHub Actions is not permitted to create or approve pull requests.

Steps 1–6 pass every time. It checks out, builds, measures coverage, decides a bump, writes .coverage-floor — then dies on open ratchet PR. Nobody watches a scheduled workflow that has always been red, so nobody saw it.

.coverage-floor has sat at 60 since it was created. Not because coverage never rose — because the thing that raises it has never once completed.

This is the same shape as the dead Codecov pipe from #42, one level up: an automation that looks wired, fails at the last hop, and reports success to anyone who isn't reading logs. The difference is that this one is the L4/L5 self-improvement loop plumbline cites as evidence of its own maturity.

What this PR can and can't do

The blocker is can_approve_pull_request_reviews: false on the repo — a settings toggle. The workflow's permissions: block is already correct (contents: write, pull-requests: write) and cannot grant it. So this PR does not fix the loop. It stops the loop from failing silently:

  • open ratchet PR becomes continue-on-error with an id, so its failure no longer swallows the result.
  • A fallback step files an issue carrying the measured total, current floor, proposed floor, and the exact setting to enable. Deduped by proposed floor, so it's one issue per bump rather than one per week.
  • The run still exit 1s with an ::error::, because the loop is broken until the setting changes. Loud and actionable beats quiet and green.

CONTRIBUTING.md gains a table of repo settings the automation depends on but can't request — docs-signals.yml and canary-repos.yml use the same action and will hit this the moment their conditional PR branch is taken.

The actual fix

Settings → Actions → General → Workflow permissions → "Allow GitHub Actions to create and approve pull requests." One toggle, not something a PR can do. Worth doing regardless of whether this merges — with it on, the ratchet lands its own bump next Monday and the floor starts tracking real coverage (currently ~87% against a floor of 60).

Verification

The heredoc in the fallback is rendered and shell-parsed locally rather than trusted — it initially broke on an apostrophe inside $(cat <<EOF), a bash parsing quirk, and was reworded. bash -n clean, YAML valid, body renders with variables substituted.

Snapshot regenerated: the new step registers as a third evidence path for l3.metrics-acted-on, which is correct — it does act on the coverage number.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XtNwLDzRYmPrXcYeQypTf9

The weekly ratchet has failed on every run since at least June 8 — ten
consecutive weeks — at its final step:

    ##[error]GitHub Actions is not permitted to create or approve
    pull requests.

Every run measured coverage, decided a bump, wrote .coverage-floor, and
then died. Nothing surfaced it, because nobody watches a scheduled
workflow that has always been red. The floor has sat at 60 the whole time,
not because coverage never rose but because the thing that raises it could
never finish.

The blocker is repos/.../actions/permissions/workflow →
can_approve_pull_request_reviews: false, a repo-level toggle. The
permissions: block in the workflow is already correct and cannot grant it.
So this does not fix the loop — it stops the loop from failing silently:

- The PR step is continue-on-error, so a failure there no longer eats the
  result.
- A fallback files an issue with the measured total, the current floor, the
  proposed floor, and the exact setting to flip. One issue per proposed
  floor, not one per week.
- The run still exits 1 and emits ::error::, because the loop genuinely is
  broken until someone flips the setting. Loud and actionable beats quiet.

CONTRIBUTING.md gains a table of repo settings the automation depends on
but cannot request, with this failure as the worked example.

Enabling the setting is the actual fix, and it is not something a PR can do.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XtNwLDzRYmPrXcYeQypTf9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant