feat: add the reviewer registry that makes the triage queue answerable - #407
Conversation
Closes #403. `dotf pr triage-queue` reads harness/review-attestation.json to decide which open PRs carry reviewer output nobody has dispositioned. The file did not exist here, so the command could not answer and exited 1 on every run — and since it exits non-zero both when work is pending and when the question is unanswerable, hive could produce no queue-clear evidence for any PR at all. Every PR in this repo closed with the Definition of Done's evidence item skipped rather than met. Only coderabbitai is registered. dotfiles additionally declares github-actions for PR-Agent; no such workflow runs here, and declaring a reviewer that cannot run would assert a review that never happens. The non-obvious part is which marker identifies its output. CodeRabbit files proper reviews through the reviews API — #401 carries a 5-finding review reading "Actionable comments posted: 5" in reviews[] — but the queue reads `gh pr list --json comments`, which never returns reviews[]. Matching the review text would leave the queue permanently empty on a repository that is actively being reviewed. The registered marker is therefore the walkthrough comment posted alongside every review, which is comment-shaped; verified present on both open PRs, #401 (a review) and #406 (a decline). The limitation this leaves — the walkthrough is edited rather than reposted on re-review, and createdAt does not move on an edit — is stated in the file rather than left to be rediscovered. advisory: true on the decline, measured here and not inherited: on #406 and #405 CodeRabbit was rate limited and its commit status read `success`. A green check meaning no review ran is the failure this mechanism exists to catch, so AGENTS.md now says so in the PR workflow alongside how to run the queue. Verified end to end: the queue now answers, correctly omits #406 (which carries a triage comment) and correctly surfaces #405 (which does not).
|
Warning Review limit reachedNext included review available in 26 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review triageCI: all 7 check runs green ( Review: none ran. CodeRabbit's commit status reads There is a circularity worth naming rather than glossing: the change that teaches this repo not to trust CodeRabbit's green status is itself merging on an unreviewed green status. That is not a reason to hold it — it is the argument for it — but it does mean the registry lands without an independent read of its marker choice, which is the one judgement call in the file. Findings: none to dispose of. Verified in this session, not just asserted:
One wrinkle found while testing, worth knowing before merge: the registry path is CWD-relative ( Recommendation: merge on the human's call, with the unreviewed status on the record. |
Lesson 096, from the #403 / #407 work. It merged before this was written; this carries the part worth keeping. ## The finding `dotf pr triage-queue` calls `gh pr list --json comments`. GitHub exposes a PR's discussion through **three separate surfaces** — issue comments, the reviews API (`reviews[]`), and inline review comments — and that call returns **only the first**. On this repo CodeRabbit files *proper* reviews. #401's five findings live in `reviews[]` as `**Actionable comments posted: 5**`; `comments` held only the walkthrough. So registering the review's own text as the marker would have produced a queue that was **permanently, silently empty on a repository that was being reviewed the whole time**. The dotfiles registry does not hit this because its second reviewer, PR-Agent, publishes *through* the comments API — which is why its marker does not transfer, and why "port the config" was the wrong mental model for #403. ## Why it is worth a lesson rather than a commit message The direction of failure. A wrong marker does not error — it returns *"no reviewer output yet"*, which is also what a genuinely quiet PR looks like. Same family as [087](docs/lessons/lesson-087-a-green-review-check-can-mean-did-not-review.md) and [094](docs/lessons/lesson-094-a-verification-command-that-selects-zero-tests.md): **a check that cannot see its subject reports the subject is fine.** Two rules fall out: 1. When porting a config between repos, verify the **observable** it keys on against live data in the destination. One `gh api repos/…/pulls/N/reviews` versus `…/issues/N/comments` settles it in ten seconds; reading the source registry for an hour does not. 2. Prove a matcher in **both** directions. This one was confirmed by watching the queue correctly omit a triaged PR *and* correctly surface an untriaged one it found on its own — #405, a real open release PR nobody had looked at, invisible until the registry existed. Docs only. All wikilinks verified against the working tree.
🤖 I have created a release *beep* *boop* --- ## [4.2.0](v4.1.0...v4.2.0) (2026-08-25) ### Features * add the reviewer registry that makes the triage queue answerable ([#407](#407)) ([2492151](2492151)) ### Bug Fixes * **spec:** record measured evidence on HIVE-384, and stop AC5 reading green ([#408](#408)) ([5fd5aa6](5fd5aa6)) ### Documentation * correct lesson-094's premise and three reviewer findings from [#401](#401) ([#406](#406)) ([66dbf17](66dbf17)) * record five lessons from the delegate-verb work ([#401](#401)) ([8bc66eb](8bc66eb)) * record why the reviewer registry could not be copied ([#409](#409)) ([2b920ba](2b920ba)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Closes #403.
The problem
That command exits non-zero for two different reasons — work is pending, or the question could not be answered — deliberately, so an unanswerable queue is never mistaken for a clear one. Here it was always the second. The consequence: hive could produce no queue-clear evidence for any PR, ever, so every PR closed with the Definition of Done's evidence item skipped rather than met.
The non-obvious part: which marker identifies a review
This is not a copy of the dotfiles registry, and the reason is worth reading before reviewing the file.
CodeRabbit files proper reviews through the reviews API on this repo. Measured on #401: a 5-finding review reading
**Actionable comments posted: 5**, sitting inreviews[]. But the queue readsgh pr list --json comments, which returns issue comments only and neverreviews[]. Matching the review's own text would have left the queue permanently empty on a repository that is actively being reviewed — the exact false clear the mechanism exists to prevent.So the registered marker is the walkthrough comment CodeRabbit posts alongside every review, which is comment-shaped. Verified present on both PRs open at the time of writing:
The limitation this leaves is stated in the file, not left to be rediscovered: the walkthrough is edited rather than reposted on re-review, and
createdAtdoes not move on an edit — so a second review after a push may not re-open the queue for a PR already triaged once. The queue is a floor, not a ceiling. Closing that needs a consumer that readsreviews[], which is a change to the dotfiles CLI, not to this file.advisory: true, measured here rather than inheritedOn #405 and #406, CodeRabbit was rate limited, posted "Review limit reached", and its commit status read
success. A green check that means no review ran is precisely the failure this whole mechanism exists to catch. Advisory means its decline does not refuse the change — a quota is a billing state of a third party, not a property of the diff — while its review still attests.Note what the two settings do together, and it is intended: the decline notice also carries the walkthrough marker, so a rate-limited PR reads as output awaiting disposition rather than as a clear queue. That is the honest answer.
Deliberately absent
The
escapeandexemptblocks dotfiles carries. Those are read by its review-attestation CI gate, which hive does not run —prtriagereadsreviewersandtriage.markerand nothing else. A registry declaring policy no consumer here enforces is drift with extra steps. The practical consequence: release-please PRs are not exempted and will queue for a one-line disposition.Verified end to end
Both directions confirmed: #406 correctly omitted (it carries a
## Review triagecomment) and #405 correctly surfaced (it does not). The queue found a real open PR I did not know existed.AGENTS.mdgains two bullets in the PR workflow section: how to run the queue and where to record the disposition, and the warning that CodeRabbit's green status can mean it never reviewed.