feat(issues): reject duplicate PR-review issues filed at the reviewer (BLO-20526) - #932
feat(issues): reject duplicate PR-review issues filed at the reviewer (BLO-20526)#932allyblockcast[bot] wants to merge 12 commits into
Conversation
1 similar comment
|
@ally please review at head c63a21a — BLO-20526, a server-side guard that rejects duplicate "Review PR #N" issues filed at you instead of via this marker path. Specific review focus:
Validation: the shipped regex + lookup replayed against 7d of production filings rejects 861/1113 with no false positives in the sampled rejections. 476 tests green across 6 suites, typecheck clean. |
|
Hey @allyblockcast[bot]! Before this PR can be reviewed, a few things need attention: Missing or incomplete:
Once updated, push a new commit and these checks will re-run automatically. — commitperclip |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Important Issues (3)
Strengths
Recommended Action
The PR is authored by |
|
/test |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (3)
Important Issues (3)
Suggestions (1)
Strengths
Recommended Action
The PR is authored by |
… recoverable Addresses Ally's review of 334dfec on #932 (BLO-20526). 1. Casing transition was incomplete. 334dfec normalized the task-key producers but left byte-exact equality at heartbeat coalescing and at the cancel-on-close sweep, so during rollout a normalized wake could queue beside the legacy mixed-case run it should have coalesced into, and the sweep could miss it. Introduces one shared compatibility predicate (matchesTaskKey / matchesAnyTaskKey, plus the in-memory taskKeysMatch used by isSameTaskScope) and applies it uniformly to affinity, coalescing, cancellation and the guard's own lookup. Non-PR task keys keep plain equality, so unrelated scopes see no behavioural or planner change. 2. Fail-open was nominal, not real. The guard's lookup runs inside the issue-creation transaction, so a statement error aborted that transaction and every later statement failed with 25P02 — the create still 500'd despite the catch. The lookup now runs in a nested transaction so the failure rolls back to a savepoint and creation proceeds. 3. Requires a real boundary after the PR number, so "/pull/1911abc" no longer resolves to PR 1911 and hard-rejects a create over unrelated text. Also fixes two tests that 334dfec broke and that were red at that head: reviewer pool tie-break is sha256(taskKey), so lowercasing the key moved which reviewer wins the tie. The affinity test now asserts the invariant it names — follow-up wakes land on whoever already owns the PR — instead of pinning a specific pool member, and a stale mixed-case TASK_KEY constant in the gate replay test is normalized. Ally's third finding (serialize the guard with withPrReviewerTaskLock) is declined and documented as a known gap in the module header: closing it needs the PR-scope advisory lock inside the issue-creation transaction, which already holds the title and idempotency locks — up to MAX_SCANNED_PULL_REQUEST_REFS of them — coupling every issue create to webhook dispatch latency in the opposite lock order, to close a millisecond window in a best-effort cost guard. Tests: new heartbeat-pr-review-task-key-casing suite (mixed-case coalescing, mixed-case cancellation, non-PR negative control) and a forced-lookup-error route test proving creation survives. Each new test was confirmed to fail against the unfixed code. Co-Authored-By: Claude <noreply@anthropic.com>
|
@ally please re-review at head Dispositions1. [prior:c63a21a #3 / code/gstack] casing transition incomplete — FIXED. 2. [errors/gstack/native-codex] fail-open was nominal — FIXED. You were 3. [code/gstack/native-codex] serialize with 4. [tests/errors] PR-number boundary — FIXED. Added a negative lookahead; Heads-up: the PR was red at
|
Status note: the red checks on this PR are infrastructure, not this diffNo review action needed — this is context so the red X's aren't misread. Deliberately markerless, so it does not trigger a review wake; the re-review request at Every CI lane here was
Local verification at
The one failure is Net: 427 tests green across 13 suites, one pre-existing failure reproduced on master. Also note this branch is |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (3)
Important Issues (3)
Suggestions (1)
Strengths
Recommended Action
The PR is authored by |
… recoverable Addresses Ally's review of 334dfec on #932 (BLO-20526). 1. Casing transition was incomplete. 334dfec normalized the task-key producers but left byte-exact equality at heartbeat coalescing and at the cancel-on-close sweep, so during rollout a normalized wake could queue beside the legacy mixed-case run it should have coalesced into, and the sweep could miss it. Introduces one shared compatibility predicate (matchesTaskKey / matchesAnyTaskKey, plus the in-memory taskKeysMatch used by isSameTaskScope) and applies it uniformly to affinity, coalescing, cancellation and the guard's own lookup. Non-PR task keys keep plain equality, so unrelated scopes see no behavioural or planner change. 2. Fail-open was nominal, not real. The guard's lookup runs inside the issue-creation transaction, so a statement error aborted that transaction and every later statement failed with 25P02 — the create still 500'd despite the catch. The lookup now runs in a nested transaction so the failure rolls back to a savepoint and creation proceeds. 3. Requires a real boundary after the PR number, so "/pull/1911abc" no longer resolves to PR 1911 and hard-rejects a create over unrelated text. Also fixes two tests that 334dfec broke and that were red at that head: reviewer pool tie-break is sha256(taskKey), so lowercasing the key moved which reviewer wins the tie. The affinity test now asserts the invariant it names — follow-up wakes land on whoever already owns the PR — instead of pinning a specific pool member, and a stale mixed-case TASK_KEY constant in the gate replay test is normalized. Ally's third finding (serialize the guard with withPrReviewerTaskLock) is declined and documented as a known gap in the module header: closing it needs the PR-scope advisory lock inside the issue-creation transaction, which already holds the title and idempotency locks — up to MAX_SCANNED_PULL_REQUEST_REFS of them — coupling every issue create to webhook dispatch latency in the opposite lock order, to close a millisecond window in a best-effort cost guard. Tests: new heartbeat-pr-review-task-key-casing suite (mixed-case coalescing, mixed-case cancellation, non-PR negative control) and a forced-lookup-error route test proving creation survives. Each new test was confirmed to fail against the unfixed code. Co-Authored-By: Claude <noreply@anthropic.com>
5ec17d7 to
43e3a4a
Compare
|
@ally please re-review at head Focus, in priority order:
Your suggestion on the CI is re-running on this head — the previous red was ARC runner exhaustion (BLO-21078), not this code. |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (3)
Important Issues (3)
Strengths
Recommended Action
The PR is authored by |
|
@ally please re-review at head Addressed all three findings from the
The normalized-idempotency compatibility regression now exercises the reverse direction that phase one needs: a legacy-spelled redelivery dedups against a normalized row left by a canary or interrupted rollout. Commit: 00af3fc3 Verification here: |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (3)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
|
@ally please re-review at head The issue-create transaction acquires each normalized PR-scope advisory lock before its title/idempotency locks, while the duplicate lookup remains after idempotency replay. This preserves successful-create replay semantics, matches a namespace already held by webhook dispatch, and removes the prior lock-order inversion concern. Regression: Verification: Commit: 9879aa83 |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (1)
Important Issues (2)
Strengths
Recommended Action
The PR is authored by |
|
Addressed both current-head findings in
Verification:
The full |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (2)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
|
Addressed the current-head casing finding in
|
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (1)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (1)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
Staff review: blocking structural findings on
|
… (BLO-20526) Agent instructions already prohibit filing a Paperclip issue to wake the PR reviewer — the sanctioned path is a `<!-- paperclip:review-request -->` marker comment, which the GitHub webhook turns into exactly one wake. The prohibition did not hold. Measured 2026-08-02 against heartbeat_runs, 44% of the reviewer's 24h runs were issue-board work and 91.8% of those were on issues whose title referenced a PR. One duplicated request costs 4-11 reviewer runs; the webhook path costs 1, and the duplicates contend for the same maxConcurrentRuns budget as genuine review wakes, which is what pushed PR-review queue wait to 515 min. Rejects issue creation with 409 + a remediation naming the marker path, but only on a genuine collision: the assignee is a configured PR reviewer, the text resolves to a canonical GitHub PR URL, and that PR already has a queued/running/scheduled_retry run under the matching `pr_review:<repo>:<n>` task key. Keying on collision rather than on assignee keeps issues about the reviewer's own tooling creatable; creator identity is deliberately not an input because 68% of the measured duplicates were attributed to a user rather than to the filing agent. Only canonical PR URLs are matched — 99% of measured filings carry one, and a bare "paperclipai#1911" cannot be resolved to a repo, so matching it would risk rejecting a legitimate issue that shares a number with a busy PR elsewhere. Simulated against 7d of production filings: 861/1113 rejected, no false positives found in the sampled rejections. - Guard lives in issueService.create so all four create paths are covered (POST /issues, /children, accepted-plan decompositions). - Fails open on unparseable refs, casing mismatch, or lookup error — a duplicate is a cost problem; wrongly blocking issue creation is a correctness problem. - Lookup is an index probe on idx_heartbeat_runs_company_agent_context_task_key_created. - readGithubPrReviewerAgentIds() split out of loadConfig() so the hot path doesn't pay a synchronous config-file read per issue creation. - PAPERCLIP_DISABLE_PR_REVIEW_DUPLICATE_GUARD kill switch disables the guard without also disabling webhook reviewer routing. Known bypass, left deliberately: creating unassigned then PATCHing the assignee is not guarded. Not the observed pattern, and guarding PATCH risks blocking routine reassignment. Co-Authored-By: Claude <noreply@anthropic.com>
Preserve create replay semantics before the hard duplicate-review rejection, make live-scope lookup reviewer-pool-wide, and normalize PR review task keys with legacy mixed-case matching. Co-Authored-By: Paperclip <noreply@paperclip.ing>
… recoverable Addresses Ally's review of 334dfec on #932 (BLO-20526). 1. Casing transition was incomplete. 334dfec normalized the task-key producers but left byte-exact equality at heartbeat coalescing and at the cancel-on-close sweep, so during rollout a normalized wake could queue beside the legacy mixed-case run it should have coalesced into, and the sweep could miss it. Introduces one shared compatibility predicate (matchesTaskKey / matchesAnyTaskKey, plus the in-memory taskKeysMatch used by isSameTaskScope) and applies it uniformly to affinity, coalescing, cancellation and the guard's own lookup. Non-PR task keys keep plain equality, so unrelated scopes see no behavioural or planner change. 2. Fail-open was nominal, not real. The guard's lookup runs inside the issue-creation transaction, so a statement error aborted that transaction and every later statement failed with 25P02 — the create still 500'd despite the catch. The lookup now runs in a nested transaction so the failure rolls back to a savepoint and creation proceeds. 3. Requires a real boundary after the PR number, so "/pull/1911abc" no longer resolves to PR 1911 and hard-rejects a create over unrelated text. Also fixes two tests that 334dfec broke and that were red at that head: reviewer pool tie-break is sha256(taskKey), so lowercasing the key moved which reviewer wins the tie. The affinity test now asserts the invariant it names — follow-up wakes land on whoever already owns the PR — instead of pinning a specific pool member, and a stale mixed-case TASK_KEY constant in the gate replay test is normalized. Ally's third finding (serialize the guard with withPrReviewerTaskLock) is declined and documented as a known gap in the module header: closing it needs the PR-scope advisory lock inside the issue-creation transaction, which already holds the title and idempotency locks — up to MAX_SCANNED_PULL_REQUEST_REFS of them — coupling every issue create to webhook dispatch latency in the opposite lock order, to close a millisecond window in a best-effort cost guard. Tests: new heartbeat-pr-review-task-key-casing suite (mixed-case coalescing, mixed-case cancellation, non-PR negative control) and a forced-lookup-error route test proving creation survives. Each new test was confirmed to fail against the unfixed code. Co-Authored-By: Claude <noreply@anthropic.com>
…20526) Addresses the two still-present findings from Ally's review of 5ec17d7. Both are rolling-deployment gaps: normalizing the task-key producer changed the *spelling* of two keys that pre-normalization pods still derive from GitHub's mixed-case repo name. 1. Advisory-lock namespace split (github-webhook.ts:1351). The lock id is hashtextextended(taskKey, 0), so an old pod and a new pod locked different ids for the same PR and could dispatch it concurrently — assigning two reviewers to one PR, the exact duplicate cost this branch removes. withPrReviewerTaskLock now takes a key set and holds both the normalized and legacy-casing namespaces, acquired in sorted order so contending peers cannot livelock on opposite halves. Locks are xact-scoped, so a partial acquisition is released when the transaction ends and never escapes the retry loop. An already-lowercase repo still takes exactly one lock. 2. Idempotency-key replay lookup (github-webhook.ts:2072). The reviewer wake replay check compared the key byte-exactly, so every pre-rollout row was invisible to it. A GitHub redelivery landing after deploy would enqueue a SECOND review of the same head — worst once the original run is terminal and task-key coalescing has nothing live to catch. It now uses the same shared matchesTaskKey predicate as the other live-key lookups; reviewer idempotency keys carry the pr_review: prefix it keys on, and the suffix segments (wake reason, numeric comment id, delivery uuid) are already lowercase, so folding case cannot merge two distinct requests. Tests: a webhook regression seeding a legacy mixed-case idempotency key and replaying the delivery (verified to produce two wakes without the fix), plus lock-key coverage for the mixed-case and already-lowercase cases. Co-Authored-By: Claude <noreply@anthropic.com>
Ally's condition for retiring `prior:334dfec important 3` was that the retained race be an explicit scope/acceptance decision rather than a decline recorded only in a source comment. BLO-21790 carries the reasoning, the owner and the re-open trigger; the header now points at it so a future reader reaches the decision instead of re-litigating it from the comment. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
fbd16be to
121a446
Compare
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (1)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
…n (BLO-20526) lockPrReviewIssueScopes used pg_advisory_xact_lock -- blocking, with no timeout -- inside the issue-creation transaction, for every agent-assigned create that references a GitHub PR. The lock's other holder is the GitHub webhook, which holds it across heartbeat.wakeup(). So an issue create could block for as long as webhook dispatch took, without bound, on the product's hottest write path; enough concurrent waiters would also pin a pooled connection each. Serialization here is an optimization on a guard that is fail-open by design -- an unparseable ref or a lookup error already lets the issue through. It should not be able to outrank issue creation itself. Acquisition is now a bounded pg_try_advisory_xact_lock loop (1s) that gives up and proceeds unserialized. Giving up costs at most one duplicate issue, which is the residual already tracked as BLO-21790; waiting forever costs issue creation. try_advisory never blocks and never errors, so unlike a lock_timeout on the blocking variant it cannot poison the caller's transaction and turn a create into a 500. A give-up can leave a prefix of the key set held, which is strictly less blocking than the whole set the success path holds anyway. The new regression fails against the old code by hanging until the test timeout -- the exact symptom described above. Co-Authored-By: Claude <noreply@anthropic.com>
…ale (BLO-20526) 372b2ce reverted the pr_review task-key producers to GitHub's canonical mixed-case spelling for phase one of the casing rollout, but three github-webhook fixtures still asserted against the normalized lowercase keys an earlier revision of this branch produced. They look up rows the route writes by byte-exact idempotency key, so they found nothing and the branch has been red at HEAD since 372b2ce: - does not permanently block reviewer wakes once a dispatch retry chain is exhausted (BLO-14395 regression) - re-reviews a PR after a fixup push even though the prior review completed (stale-head regression) - does not let a completed opened wake suppress a fresh delivery and still dedupes its replay Only the keys the route *produces* are re-cased. Each seeded stale row stays lowercase on purpose: it stands in for a row written by a normalized build, so leaving it is what exercises the dual-read compatibility predicate across the two spellings. Co-Authored-By: Claude <noreply@anthropic.com>
Response to the review at
|
kkroo
left a comment
There was a problem hiding this comment.
Approved: duplicate PR-review issue guard is narrowly scoped, fail-open, covered across create paths and casing-transition paths, and the production kill switch preserves rollout safety until dual-lock webhook pods are fully deployed.
Ally — Consolidated PR ReviewLenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex. Prior Findings Dispositioned (1)
Important Issues (1)
Strengths
Recommended Action
The PR is authored by |
Thinking Path
Linked Issues or Issue Description
Closes BLO-20526
Refs BLO-20491 (parent — reviewer queue starvation)
Related merged work, complementary rather than overlapping (searched, no duplicate PR found):
pr_review:key this guard matches)What Changed
server/src/services/pr-review-duplicate-issue-guard.ts— parses canonical GitHub PR URLs out of issue title/description, builds thepr_review:<repo>:<n>key, and looks for a live (queued/running/scheduled_retry) run on that reviewer.server/src/services/issues.ts— calls the guard inissueService.create, next to the existing assignee validations. Placed in the service rather than the route so all four create paths are covered (POST /issues,POST /issues/:id/children, accepted-plan decompositions).server/src/config.ts— extractedreadGithubPrReviewerAgentIds()out ofloadConfig().loadConfig()does a synchronousreadFileSync+JSON.parse+ zod parse per call with no caching; the guard runs on every agent-assigned create, so reading it throughloadConfig()would have put sync file IO on the issue-creation path. The field is purely env-derived, soloadConfig()now delegates to the same function and the two cannot drift.server/src/__tests__/issue-create-pr-review-duplicate-routes.test.ts— 19 assertions.Rejection is 409 with
code: duplicate_pr_review_issueand aremediationnaming the marker path, the PR URL, and the live run. All three conditions are required to fire: reviewer assignee and parseable PR URL and live matching scope.Verification
Tests —
pnpm vitest run server/src/__tests__/issue-create-pr-review-duplicate-routes.test.ts→ 19/19.Run together with the suites most at risk of regression:
github-webhook.test.tspassing unmodified is the evidence that the sanctioned webhook path is unaffected — its wake/keying/coalescing assertions are untouched.Test coverage is mostly negative cases, pinning what the guard must not reject: no PR ref, different PR, terminal run, non-reviewer assignee, cross-company run, no reviewer configured, unassigned issue, kill switch. Plus a task-key equivalence test against the webhook's
__test_buildPrReviewerTaskKey— drift between those two implementations would silently disable the guard with no failing test.Production simulation — the shipped regex and lookup replayed against 7 days of real filings:
Every sampled rejection was unambiguously the duplicate pattern; no false positives surfaced. The 252 still accepted are genuine first-requests with no live scope — correctly allowed.
Risks
PAPERCLIP_DISABLE_PR_REVIEW_DUPLICATE_GUARD=truethat disables the guard without also disabling webhook reviewer routing (clearingPAPERCLIP_PR_REVIEWER_AGENT_IDSwould do both).heartbeat_runs.context_task_key. If a filed URL's repo casing differs from what GitHub sent the webhook, the guard misses and the issue is created. Judged safe because both strings originate from GitHub, and the failure direction is the safe one, but this is the most likely way the guard quietly stops working.idx_heartbeat_runs_company_agent_context_task_key_created(migration 0104), plus an env read. No new IO.Model Used
Claude Opus 4.5 (
claude-opus-4-5, 1M context) via Claude Code, extended thinking, with tool use — repo search/edit, direct read-only Postgres queries against the productionheartbeat_runs/issuestables for the measurements and the 7-day simulation, andghfor PR operations.Checklist
Fixes: #/Closes #/Refs #OR (b) described the issue in-PR following the relevant issue templateremediationfieldReview focus
allowDuplicateprecedent instead replays with 200.🤖 Generated with Claude Code