claude-code-review: reviewer spawns background agents, then ends its turn waiting for them instead of posting a verdict

[d-morrison]

First execution artifact ever retrieved for a claude-review stub, and it names the cause exactly. This is not #391 — different signature, kept separate deliberately.

Evidence

claude-review-execution-30671528617-1-attempt1, from Morrison-Lab/ai-config#986 at head 147ee69, running claude-code-review.yml@v2.

The single result event:

{
  "is_error": false,
  "subtype": "success",
  "stop_reason": "end_turn",
  "api_error_status": null,
  "terminal_reason": "completed",
  "num_turns": 17,
  "total_cost_usd": 9.7565,
  "result": "Waiting for the remaining background agents to complete."
}

That result string is the whole finding. It is the model's final message, and it is what got posted in place of a verdict.

Tool-use counts across the run: Agent 8, Bash 34, Read 2, ToolSearch 1, TaskList 1, Write 1.

The last three assistant text blocks, in order:

I'll pause here and wait for the background agents' completion notifications before continuing.

I'll stop issuing filler calls and simply wait for the background task notifications to arrive.

Waiting for the remaining background agents to complete.

What happens

The reviewer farms the review out to eight background subagents, then ends its turn waiting for completion notifications. In an interactive session those notifications wake the agent back up. In a headless Actions run the turn simply ends, so the eight agents' findings are never collected and no verdict is ever written.

Every status field reports success, because nothing failed. The work was done and then discarded.

Why it costs real money for nothing

total_cost_usd is $9.76 for this attempt alone, over 17 turns, with 8.9M cache-read tokens. The Fail the check ... (attempt 1) guard then correctly classifies the output as a stub and fires the #185 retry — which runs the same prompt and can reasonably be expected to delegate the same way, doubling the spend.

Two things this rules out

• Not quota, auth, or rate limiting. The two rate_limit_event entries both read "status": "allowed_warning" at 0.62 and 0.63 utilization on a seven-day window.
• Not context overflow. modelUsage reports the reviewer on claude-sonnet-5 with "contextWindow": 1000000. See my retraction on claude-code-review: the guard fails the check on is_error=true even when a complete verdict was posted #391; I had this wrong.

One contributing signal worth checking

permission_denials shows Bash calls denied, all of the shape gh api repos/Morrison-Lab/ai-config/pulls/985 --jq .... The reviewer reached for gh for basic PR state, was refused by the allowlist, and thereafter leaned on Agent. Whether that pushed it toward delegation is a guess, but the denial is real and is independently worth fixing.

Suggested direction

The review prompt should state that the run is headless and single-shot: no background delegation, no waiting on notifications, and the verdict must be written by the top-level agent before its turn ends. Constraining or removing the Agent tool from allowedTools for the reviewer would enforce that mechanically rather than by instruction, which is the stronger form.

Worth checking whether the guard can also treat "final message names background agents" as a distinct outcome from a generic stub, since the retry is unlikely to help and the spend is large.

[d-morrison]

Correcting the fix I proposed above, before anyone acts on it. #393 implements the corrected version.

"Constrain or drop Agent from allowedTools" is wrong. The comment block above claude_args in run-claude-review-attempt/action.yml records a deliberate decision, made when ScheduleWakeup/SendMessage/Monitor were denied for #218, to leave the Agent fan-out alone -- because the code-review plugin fans out to four parallel sub-agents by design. Denying Agent closes this bug by breaking that.

I proposed it without reading the file I was proposing to change.

The actual mechanism is narrower and the fix is smaller. Agent's run_in_background parameter defaults to true. The prompt already forbids deferring work past the end of the run, and then carves out an exception:

(The code-review plugin's own same-run parallel sub-agent fan-out is fine -- this only forbids deferring work past the end of this run.)

Given the default, that carve-out permits exactly what the preceding sentence forbids. Any fan-out the model performs is a background spawn, so the instruction is unsatisfiable as written, and the model resolved it the way the artifact shows.

So the fix is to require run_in_background: false on every Agent/Task call. Parallelism kept, deferral removed.

One thing this sharpens about #218's fix. Denying those three tools did not close the failure mode they were denied for; it closed three routes to it. This run reached the identical end state through the one orchestration tool deliberately left allowed, without calling any of the three. The distinction now stated in the code: those three have no synchronous form and stay denied outright, while Agent is allowed in its synchronous form only.

The two follow-ons from the original comment still stand and are not in that PR: the guard spending a second full attempt on a prompt likely to delegate the same way, and the Bash(gh api ...) denials.

---

Generated by Claude Code

[d-morrison]

Measured effect of the fix, on a controlled comparison

v2 was slid to 70c7fa8 carrying #393. Consecutive review rounds on the same PR, same reviewer, give a before/after:

Round	Run	Tag resolved	Cost
1	30673938581	pre-slide	$17.03
2	30676853877	claude-code-review.yml@v2 at 70c7fa8	$4.18

Both on Morrison-Lab/ai-config#1003. The post-slide run's referenced_workflows[].sha was read directly rather than inferred, and matches the tag's commit.

Other pre-slide rounds the same evening, for scale: $27.57, $13.04, and $14.30 (that last one on ai-config#986, where attempt 1 hit this bug and the gha#185 stub-retry recovered).

$4.18 lands on the single-clean-review baseline, which is what you would expect if the wasted first attempt is gone rather than merely cheaper.

Caveats, since this is one comparison and not a study

• n=1. Four further post-slide rounds are pending on sibling PRs; if they land in the same band that is worth more than this table.
• The diffs differ. Round 2 reviewed a smaller incremental change than round 1, so some of the drop is content, not the fix. The magnitude (4x) is larger than that confound plausibly explains, but it is a confound.
• Cost is a proxy for "did a whole attempt get thrown away". The direct evidence remains the execution output, which is what claude-code-review: reviewer spawns background agents, then ends its turn waiting for them instead of posting a verdict #392 was diagnosed from.

Recording it here rather than in a memory file: it is evidence about this fix, it decays as soon as pricing or the reviewer prompt changes, and a memory entry would read as settled fact rather than as one measurement.

---

Generated by Claude Code

[d-morrison]

Correcting my own comment above: cost is a bad proxy, and the band I predicted is already broken

I wrote that $4.18 "lands on the single-clean-review baseline" and that further post-slide rounds landing "in the same band" would strengthen the case. A post-slide round has now landed well outside it.

Morrison-Lab/ai-config#1008, run 30681344801:

• referenced_workflows[].sha = 70c7fa86, refs/tags/v2 -- read directly, so this is unambiguously post-slide.
• Cost: $26.70.

So two post-slide runs cost $4.18 and $26.70. The band does not exist.

What that does and does not undermine

The paired comparison is unaffected as a pairing. $17.03 then $4.18 on the same PR, consecutive rounds, is still what it was.

The inference I drew from it was wrong. I treated cost as tracking the wasted first attempt, with diff size as a minor confound. It is the other way round: cost is dominated by how much work the review does. The $26.70 run reviewed +120 lines across four files, produced four findings, and spot-checked three PRs against live API output. The $4.18 run re-checked two findings on a small incremental diff. The wasted attempt this issue is about is a roughly $9.76 fixed addend, which is a large share of a $14 review and a small share of a $27 one.

That also means my $4.18 figure was never good evidence for the fix -- a cheap round following an expensive one is what you would expect from a shrinking diff alone.

What would actually settle it

The execution output, which is what #392 was diagnosed from: whether attempt 1 ends with stop_reason: end_turn and a message about waiting on background agents, and whether an attempt-2 artifact exists. That is a direct observation of the behaviour rather than a proxy for it.

Leaving the original comment in place rather than editing it, since this one responds to it. Treat the cost table there as superseded: suggestive of nothing on its own, and not a metric to judge this fix by.

---

Generated by Claude Code