Skip to content

Record that gh pr checks omits check runs, and correct fully-clean's fifth case - #1079

Merged
d-morrison merged 2 commits into
mainfrom
fix/check-surface-completeness
Aug 3, 2026
Merged

Record that gh pr checks omits check runs, and correct fully-clean's fifth case#1079
d-morrison merged 2 commits into
mainfrom
fix/check-surface-completeness

Conversation

@d-morrison

Copy link
Copy Markdown
Collaborator

Closes #1074

What changed

Two files, both shared/workflow/.

1. fully-clean.md criterion 1 gains a block on gh pr checks incompleteness

The criterion requiring every check to be completed and passing now says that gh pr checks is not a complete enumeration of a head's check runs, and names commits/<sha>/check-runs as the authority for "has everything finished".

It is explicit that this is a different gap from the one the criterion already documents two paragraphs above. That passage warns a workflow run may produce no check run, so a check-runs query cannot see it, and sends the reader to the raw workflow runs. Here the check run exists and the check-runs endpoint returns it; it is gh pr checks that omits it, so the raw-workflow-run cross-check finds nothing missing and confirms the wrong surface.

Per the issue's "Not yet established" section, no mechanism is asserted. The block names the three untested candidates (app filter, required/branch-protection set, in-progress omission) and records the counts that disqualify each without supplying a replacement.

2. The fifth case's "no check run at all / by construction" claim is corrected

That section stated the reviewer contributed "no check run at all" on #1005 and #1008 and generalized to "the check surface is silent about that reviewer by construction". Both observations are now kept with their dates. No claim is made that the behaviour changed -- two dates cannot establish that, and no change is needed to explain the original record, since the two surfaces disagree on those same PRs on the same day.

The section's operative advice (settle it from get_reviews filtered by login) survives and is stated as strengthened, because there are now two distinct failure modes landing on the same remedy from opposite sides: a check run the rollup omits, and a green check run with no review behind it.

3. An incidental correction that runs the other way

The fifth case carried a retraction of a 04:50:41Z figure, calling it an invented particular. That retraction is itself wrong, and it was produced by exactly the gap this PR documents -- a query against the surface that omits the check run, read as establishing the check run does not exist.

metacognitive-monitoring.md asserted the same refuted claim ("#1008 carries no Copilot-attributable check run at all"), so it is corrected in the same pass rather than left contradicting fully-clean.md. Flagging it here because it is outside the issue's stated scope.

Verification

All re-measured on 2026-08-03, not taken from the issue.

PR head gh pr checks check-runs API Copilot in rollup Copilot in API
#1056 cbf39b64 10 13 0 1, success
#1005 a17f1021 8 9 0 1, success
#1008 7abfed6b 9 11 0 1, success

Checks run before pushing

sync-codex-skill-wrappers.py (no wrapper changes), validate-skills.py, check-links.py (947 links, 434 files), check-vendored-drift.py, markdownlint-cli2@0.22.1 (456 files, 0 errors), check-memory-file-size.py, test_check_memory_file_size.py (13 passed).

check-new-line-breaks.py run after committing against origin/main...HEAD: no lines missing semantic breaks. Added lines scanned for banned punctuation: 131 examined, 0 hits.

fully-clean.md criterion 1 gains a block saying gh pr checks is not a
complete enumeration of a head's check runs, naming the commit
check-runs endpoint as the authority for "has everything finished",
and distinguishing this from the existing workflow-run-vs-check-run
gap two paragraphs above. It states plainly that the reason for the
omission is unestablished, names the three untested candidates, and
records the counts that disqualify each.

The fifth case's "no check run at all / by construction" claim is
corrected to what was measured, with both dates kept. On #1005 and
#1008 the commit check-runs endpoint returns one
copilot-pull-request-reviewer run each, conclusion success, while
gh pr checks returns zero for both. That also reinstates the
04:50:41Z figure a later revision retracted: check run 91327863807
on 7abfed6 reads completed_at 2026-08-01T04:50:41Z, success.

metacognitive-monitoring.md's sixth-instance paragraph asserted the
same refuted claim, so it is corrected in the same pass.

Closes #1074
Copilot AI review requested due to automatic review settings August 3, 2026 03:26
@d-morrison

Copy link
Copy Markdown
Collaborator Author

Incidental observation while opening this PR, worth recording because it is the same shape one surface over.

The Copilot review request landed, and the two surfaces disagree about it:

$ gh api repos/Morrison-Lab/ai-config/pulls/1079/requested_reviewers --jq '.users[].login'
Copilot
$ gh pr view 1079 --json reviewRequests
{"reviewRequests":[]}

Not claiming a shared mechanism with the check-run omission this PR documents -- requested_reviewers is a different endpoint, and I have not tested why these two disagree either. Recording it as a second, independent instance of a gh convenience view returning zero Copilot-attributable entries where the REST endpoint returns one. Will fold it into the fragment only if a reviewer thinks it earns a line.

@github-actions
github-actions Bot removed the request for review from Copilot August 3, 2026 03:26
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


No issues found. Checked for bugs and CLAUDE.md compliance — this is a documentation-only PR (shared/workflow/fully-clean.md, shared/workflow/metacognitive-monitoring.md) that I fact-checked directly against the live GitHub API (check-run IDs, SHAs, timestamps, counts for PRs #1056, #1005, #1008), and every empirical claim verified exactly. Style rules (no em-dashes, cross-reference integrity) are satisfied.

Verdict

Ready for merge

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

💰 Cost: $6.7187 (review) — run

@d-morrison
d-morrison merged commit 1736266 into main Aug 3, 2026
9 checks passed
@d-morrison
d-morrison deleted the fix/check-surface-completeness branch August 3, 2026 03:36
@d-morrison
d-morrison requested a review from Copilot August 3, 2026 03:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Documents gaps in gh pr checks and corrects earlier conclusions about Copilot check runs.

Changes:

  • Recommends direct check-run API verification.
  • Corrects evidence for PRs #1005, #1008, and #1056.
  • Updates the related metacognitive case record.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
shared/workflow/fully-clean.md Revises fully-clean criteria and Copilot review guidance.
shared/workflow/metacognitive-monitoring.md Corrects the check-run verification history.
Suppressed comments (2)

shared/workflow/fully-clean.md:717

  • Optional style fix: this touched paragraph packs several major clauses into each long line, contrary to the one-clause-per-line convention in shared/writing/semantic-line-breaks.md:1-11. Split the whole paragraph semantically in one pass, as that guide requests for touched sections.
The commit check-runs endpoint disagrees with that rollup on both PRs, returning 9 and 11 respectively, the extra entries including one `copilot-pull-request-reviewer` run apiece.
Two figures in the paragraph above have since been corrected, and the correction runs opposite to the one this record previously carried.
An earlier revision claimed Copilot's own check run completed `success` at `04:50:41Z` on #1008, a later revision retracted that as an invented particular, and the retraction was the wrong one: check run `91327863807` on `7abfed6b` is named `copilot-pull-request-reviewer` and reads `completed_at: 2026-08-01T04:50:41Z`, `conclusion: success`.
It is worth leaving the whole chain visible rather than quietly deleting it, because the retraction was produced by exactly the gap criterion 1 now documents --- a query against a surface that omits the check run, read as establishing that the check run does not exist.
The record's other figure moved too: #1008's rollup count was written as 10 and re-measures at 9, and why was not determined.)

shared/workflow/fully-clean.md:102

  • This endpoint is paginated and returns only 30 check runs by default. Without --paginate, a head with more than 30 runs can still produce an empty result even when a later page contains an unfinished run, defeating the promised complete enumeration.
   gh api "repos/<owner>/<repo>/commits/<head-sha>/check-runs" \
     --jq '.check_runs[] | select(.status != "completed") | "\(.name) \(.status)"'

Comment thread shared/workflow/fully-clean.md
Comment thread shared/workflow/fully-clean.md
Comment thread shared/workflow/fully-clean.md
@d-morrison

Copy link
Copy Markdown
Collaborator Author

This PR merged at 03:36:11Z; Copilot's review posted at 03:46:25Z, ten minutes later. All five findings (3 inline, 2 in the Suppressed comments (2) block) are correct, and all five are addressed against main in #1082, commit 372a9f7.

  • --paginate missing (suppressed, :102) --- Addressed. Added --paginate plus per_page=100, with a paragraph and a Don't: bullet on why it is load-bearing: an unfinished run on page 2 returns the same empty result as a finished head.
  • Endpoint omits legacy commit statuses (:85) --- Addressed. Added the commits/<sha>/status query and reworded the Do: bullet to ask for both rather than either alone. Also recorded that its combined state reads pending with zero statuses.
  • Raw-run claim too categorical (:92) --- Addressed. Your particular checked out: run 30781006647 is Running Copilot Code Review at head_sha: cbf39b64, and gh run list --commit does surface it. Reworded to call the raw-run route indirect and best-effort rather than blind.
  • "An absent check run" (:695, :713) --- Addressed. Both occurrences now name the rollup as the thing that omits it, and :693 was tightened the same way. This was the sharpest of the five, since the ambiguity undercut the correction this PR exists to make.
  • Semantic line breaks (suppressed, :717) --- Addressed. Reflowed one clause per line. check-new-line-breaks passed on it, since it flags multi-sentence lines rather than long single-sentence ones, so CI did not gate this.

Nothing rebutted or deferred.

d-morrison added a commit that referenced this pull request Aug 3, 2026
#1079 merged at 03:36:11Z; Copilot's review posted at 03:46:25Z, ten
minutes later, carrying three inline findings and two suppressed ones.
All five verified against the API before addressing.

- Add --paginate and per_page=100 to the check-runs command. The
  endpoint returns 30 runs per page by default, so an unfinished run on
  page 2 yields the same empty result as a finished head, reintroducing
  the incompleteness the block is about.
- Note the endpoint covers check runs only. gh pr checks folds in legacy
  commit statuses, so swapping one for the other can hide a status
  context. Adds the commits/<sha>/status query and warns that its
  combined state reads pending with zero statuses; measured
  {"state":"pending","n":0} on both ai-config heads checked.
- Soften the raw-workflow-run claim, which was too categorical. Run
  30781006647, "Running Copilot Code Review", sits at #1056's head
  cbf39b6, and gh run list --commit does surface it, so the raw-run
  route is best-effort corroboration rather than blind.
- Name the narrower surface where the text read "an absent check run":
  the run exists and only the rollup omits it.
- Reflow the touched case-record paragraph one clause per line.

Follow-up to #1079, which closed #1074.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gh pr checks omits the Copilot check run, so it reports 0 pending while a review is still in progress

2 participants