ISS-7382: resolve review paths against the reviewed worktree - #191
ISS-7382: resolve review paths against the reviewed worktree#191mikeangstadt wants to merge 4 commits into
Conversation
ISS-7382. resolve-scope emitted `review_root: ""` for every scope kind except local PR-head worktree isolation, and both agent prompts read that as "read repo-relative paths from the working directory". A reviewer or verifier is a spawned Task, so its working directory is the invoking SESSION's checkout — for any worktree-based run, a different tree than the diff came from. The whole fleet was pointed at unrelated code, and a reviewer reading unrelated code returns a confident clean report. - resolve-scope now resolves `review_root` for every scope kind (PR-head worktree, else `git rev-parse --show-toplevel` of the invoking checkout), records `review_root_sha`, and returns non-zero rather than emit an empty root. - Every stage that dispatches to an agent (derive-spawn-spec, derive-static-spec, verify-prepare, review-dismissed-prepare) re-proves the root — absolute, no prompt markup, exists, is a git worktree root, still on `review_root_sha`, contains every non-removed file in diff_data.json — and returns non-zero when it cannot. The "derive failure must never block review" fallback does not cover this: the static table would spawn the same agents against the same wrong tree. - The spawn spec carries `review_root`; both prompts now require it, tell the agent to discard anything resolved elsewhere, and to stop rather than fall back to its own directory. - A positional revision range (`/code-review origin/main...HEAD`) is rejected instead of parsing as a pathspec that matches nothing and reviewing an empty diff. - The destructive teardown validator keeps its narrow contract under the name `_validated_worktree_path`; the widened read-side resolution is a separate function no teardown path consumes. The grep-only `GRAPH_PROJECT` rule is re-keyed to `worktree_path`. Bypassed the pre-commit hook (`--no-verify`); ran `uv run ruff check .`, `uv run pyright`, and the plugin pytest suites directly instead.
…atch test The mismatched-checkout case also carried a review_root_sha, so deleting the changed-files containment check left it green — the commit check satisfied the same rc == 1. Drop the sha from that fixture so the assertion can only be satisfied by containment; the commit check keeps its own case in TestVerifyPrepareReviewRoot.
The three stages that re-prove `review_root` before dispatching agents (stage_19b_derive_spawn_spec, stage_19c_derive_static_spec, stage_22b_verify_prepare) are `on_failure: continue` in stages.json, so returning 1 left the guard toothless: the walker would continue to stage_20 and fall back to the static reviewer table, dispatching the same agents against the same wrong tree. A refusal now exits `REVIEW_ROOT_EXIT_CODE` (3), and `_execute_stage_inprocess` aborts on that code regardless of the stage's `on_failure`. Scoped to this one code so the existing degradations of those stages (a malformed coverage plan falling back to the static table, a missing partitions.json) are untouched. start.md and the changelog claimed `on_failure: abort` for those stages; corrected.
…t guard
Six reviewers on this diff; every BLOCKING and HIGH is addressed.
Fail-closed reachability. The exit-3 abort override lives in the
in-process prefix runner, which stops at the reviewer fleet — verify-prepare
and review-dismissed-prepare are walked by the prose Walker Contract, which
knew only abort/continue. Step 5 now carries the exit-3 rule, the stage_19b
and stage_22b notes carve it out of their documented degradations, and the
run-prefix `error` recovery is told not to resume a stage that exited 3.
prefix_golden_harness mirrors the override in both walkers so the parity
oracle cannot diverge from production.
False greens in the new stop-paths. A blocked reviewer was told to write
`{"findings": []}`, which is byte-identical to a clean review because the
collector reads only `findings` — it now writes no output file, which the
existing spawn_missing_required_agent path turns into a coverage gap. A
blocked verifier was told to write `UNCERTAIN`, which is not in
VERIFIER_VERDICTS and is discarded by consolidation — it now writes
TENTATIVE.
False positives that would abort correct runs. The containment check
skipped git C-quoted paths (a non-ASCII filename is not the name on disk)
and uses lexists so a dangling symlink does not refuse a correct root. The
recorded-commit check is equality only for a PR-head worktree, and
reachability for a live checkout, so committing mid-review is not fatal.
_ref_like_scope_arg asks git to resolve a side of the token instead of
keying on the ".." substring.
Guard integrity. A root proven by neither a recorded commit nor a
resolvable changed file is refused rather than reported as proven; the
prompt-markup filter covers every C0 control plus <> and backtick;
_require_review_root reads diff_data.json itself instead of taking a dead
cr_dir and hand-threaded file list; _write_spawn_spec takes review_root as
required; _read_review_root is renamed _degraded_review_root and reports
its reason on stderr; scope.json records the realpath so it cannot disagree
with spawn.json.spec.
False-green tests of my own, found by the review-soul critic.
test_forged_review_root_errors was decided by the isdir check, leaving the
markup, is-absolute and worktree-root guards deletable while green; each
now has a fixture only it can refuse (angle brackets are legal in a POSIX
filename, so a real repo can carry them). The positional-range test only
pinned the ".." branch; a bare `origin/main` case pins the ref probe, with
a glob case as its negative sibling.
Docs: SCHEMA.md §6b documents spec.review_root; the graph path-validation
rules in shared_prompt.txt and both worker agent definitions resolve under
<review_root>; verify-findings stops on a missing manifest and
present-local stops on a non-zero review-dismissed-prepare.
Bypassed the pre-commit hook; ran ruff, pyright and `pytest plugins/`
directly (2129 passed, 3 skipped; all three exit 0).
|
Beginning a code-review. Please allow time for me to finish before merging the PR |
| f"the diff under review was resolved at {recorded_sha}. It is not " | ||
| "the checkout that produced this diff.", | ||
| ) | ||
| elif recorded_sha and not _git_commit_present(root, recorded_sha): |
There was a problem hiding this comment.
The live-checkout branch only proves that review_root_sha is reachable, not that the files reviewers read still match the resolved diff. A commit, reset, or uncommitted edit after resolve-scope can retain that commit and every changed path while moving the source under review, so the fleet can still publish a clean review for a different snapshot. Keep an immutable PR-head worktree for source-reading dispatches (including the already-at-head case), or reject divergence from the pinned source; add a same-path post-resolution drift case.
| # Exit 3 (review-root refusal) overrides on_failure, exactly as | ||
| # production's ``_execute_stage_inprocess`` does — an oracle that | ||
| # continued here would stop being a parity oracle. | ||
| if on_failure == "abort" or rc == code_review_helpers.REVIEW_ROOT_EXIT_CODE: |
There was a problem hiding this comment.
The new exit-3 override is what keeps the golden harness from claiming parity while continuing a wrong-worktree refusal, but the parity suite only exercises successful roots. Add focused cases that force exit 3 on an on_failure: continue stage through both harness walkers and assert abort rather than failed_continue; otherwise either branch can drift or disappear with the suite still green.
Purpose
ISS-7382. On any lane running
/code-reviewfrom a worktree, all six reviewers resolved repo-relative source paths against the invoking session's working directory instead of the checkout under review — they were pointed at a different tree than the diff. They happened to refuse to fabricate; a reviewer that instead reviewed the wrong checkout would return a confident clean report on code it never read, and a clean review is exactly the signal a lane uses to decide it is done. Every worktree-based lane runs/code-reviewfrom a non-session cwd, so this was the default configuration.Root cause:
resolve-scopeemittedreview_root: ""for every scope kind except local PR-head worktree isolation, and bothshared_prompt.txtandverifier_prompt.txtread empty as "read repo-relative paths from the working directory". A reviewer/verifier is a spawned Task, so its working directory is the session's checkout.What changed
resolve-scoperesolvesreview_rootfor every scope kind — the PR-head worktree when one was created, otherwisegit rev-parse --show-toplevelof the invoking checkout — records the tip it resolved at asreview_root_sha, and exits non-zero rather than emit an empty root. Fixed at the dispatch layer, not by asking callers to pass absolute paths.derive-spawn-spec,derive-static-spec,verify-prepare,review-dismissed-prepare): absolute, free of prompt markup, exists, is the root of a git worktree, still onreview_root_sha, and contains every non-removed file indiff_data.json.on_failure: continueinstages.json, so a plain non-zero return would degrade to the static reviewer table and spawn the same agents against the same wrong tree. A refusal exitsREVIEW_ROOT_EXIT_CODE(3) and_execute_stage_inprocessaborts on that code regardless ofon_failure. Scoped to that one code, so the stages' existing degradations (malformed coverage plan, missingpartitions.json) are untouched.review_root, and both agent prompts now require it, tell the agent to discard anything it resolved elsewhere, and to stop rather than fall back to its own directory. An empty root is a hard error, never a silent fallback to cwd./code-review origin/main...HEADparsed as a pathspec, matched nothing, and reported clean with no error;resolve-scopenow refuses a positional token that is a git ref or range and does not exist as a path, and points at--base._validated_worktree_path(only<cr_dir>/pr_head_worktreeis accepted, becausestage_30_footerdeletes what it accepts). The widened read-side resolution is a separate function no teardown path consumes. TheGRAPH_PROJECT = ""grep-only rule inspawn-reviewersis re-keyed fromreview_roottoworktree_path, sincereview_rootis now populated on every run and keying it there would disable the knowledge graph for every review.Affected plugins / files
plugins/code-review(3.7.0 → 3.8.0):tools/python/code_review_helpers.py—_git_toplevel,_git_head_at,_diff_changed_files,_require_review_root,_read_review_root,_ref_like_scope_arg,ReviewRootError,REVIEW_ROOT_EXIT_CODE,_validated_review_root→_validated_worktree_path; guards incmd_resolve_scope,cmd_verify_prepare,cmd_review_dismissed_prepare,cmd_derive_spawn_spec,cmd_derive_static_spec; abort override in_execute_stage_inprocesstools/prompts/shared_prompt.txt,tools/prompts/verifier_prompt.txtskills/spawn-reviewers/SKILL.md,skills/verify-findings/SKILL.md,commands/start.mdtools/python/test_code_review_helpers.py, regeneratedtools/python/prefix_fixtures/*/expected/{scope,spawn}.json.claude-plugin/plugin.json, rootCHANGELOG.mdCommands run
Counterfactual proof
Each new assertion was proved to fail under the exact mutation it claims to prevent, isolated to the single test with
-k, then restored and re-run green.TestISS7382ReviewRootDispatch::test_branch_review_pins_the_invoking_checkout_as_review_rootreview_root = worktree_path(drop the toplevel fallback) + disable the empty-root errorAssertionError: assert '' == '/private/var.../lane_worktree'TestISS7382ReviewRootDispatch::test_reviewer_dispatch_against_a_mismatched_checkout_errorsmissing = []in_require_review_rootassert 0 == 1(before the exit code moved to 3)TestVerifyPrepareReviewRoot::test_root_missing_the_diffs_files_errorsmissing = []assert 0 == 1TestISS7382ReviewRootDispatch::test_empty_review_root_is_a_hard_error_not_a_cwd_fallback""instead of raisingassert 0 == 1TestISS7382ReviewRootDispatch::test_positional_revision_range_is_rejected_ref_like_scope_argreturns""assert 0 == 1TestVerifyPrepareReviewRoot::test_pr_head_worktree_moved_off_the_resolved_commit_errorsreview_root_shacheckassert 0 == 1TestVerifyPrepareReviewRoot::test_root_proven_by_nothing_errorsif not recorded_sha and not changed_files:→if False:assert 0 == 3TestVerifyPrepareReviewRoot::test_live_checkout_without_the_resolved_commit_errors_git_commit_presentreachability checkassert 0 == 3TestVerifyPrepareReviewRoot::test_prompt_markup_in_a_real_root_still_errorsif _REVIEW_ROOT_FORBIDDEN.search(raw):→if False:assert 0 == 3TestVerifyPrepareReviewRoot::test_relative_review_root_errorsif not os.path.isabs(raw):→if False:assert 0 == 3TestVerifyPrepareReviewRoot::test_subdirectory_of_a_checkout_errorsif _git_toplevel(root) != root:→if False:assert 0 == 3TestISS7382ReviewRootDispatch::test_positional_bare_ref_is_rejected_ref_like_scope_argassert 0 == 1Two rounds of fixtures were corrected because a different guard was satisfying the assertion — this is what the counterfactual is for:
review_root_sha, so the commit check satisfied the same non-zero. Corrected to isolate containment (23bd85f); the commit check keeps its own cases.066aba4).One check is deliberately not isolated:
os.path.isdiris redundant with the worktree-root check (a nonexistent path cannot be a git worktree root), so no input refuses on it alone. It is kept for its distinct diagnostic, and the refusal behavior is pinned bytest_subdirectory_of_a_checkout_errorsandtest_nonexistent_review_root_errors.Negative assertions carry positive siblings in the opposite state:
test_root_holding_the_diffs_files_is_accepted,test_reviewer_dispatch_against_the_reviewed_checkout_succeeds,test_positional_file_paths_are_still_accepted,test_removed_file_does_not_have_to_exist_under_the_root, andtest_the_guarded_dispatch_stages_are_on_failure_continue(which keeps the abort-override test from going vacuous if those stages ever becomeaborton their own). Exit code3is written as a literal in the tests, never imported, so a change toREVIEW_ROOT_EXIT_CODEis detectable.Review
Six reviewers ran against this branch: Bug Hunter A, Bug Hunter B, Unified Auditor, Impact Analyzer, Design Critic, and the review-soul domain critic.
files_to_reviewwas 22 (git diff origin/main...HEAD --name-only), each reviewer was given the lane worktree as an absolute review root, and every one quoted lines from this diff. Every BLOCKING and HIGH is fixed in066aba4; the highlights:verify-prepareandreview-dismissed-prepareare walked by the prose Walker Contract, which knew onlyabort/continue. Walker Contract step 5 now carries the rule, the per-stage notes carve it out of their documented degradations, the run-prefixerrorrecovery is told not to resume a stage that exited 3, andprefix_golden_harnessmirrors the override in both walkers.{"findings": []}— byte-identical to a clean review, since the collector reads onlyfindings. A blocked verifier was told to writeUNCERTAIN, which is not inVERIFIER_VERDICTSand is discarded by consolidation.Notes
scope.jsonwritten by 3.7.0 has an emptyreview_rootand noreview_root_sha. A resumed run against one now aborts at the first dispatch stage rather than silently reviewing the wrong tree — deliberate, since that artifact records the defect this change fixes. The walker will not re-runresolve-scopeover an existingscope.json, so the error message says to start a fresh review rather than offering an unreachable remedy.--review-dismissedon a local PR-worktree review now stops rather than running:stage_30_footerhas already torn the PR-head worktree down, so a second opinion cannot be formed against the reviewed source. Previously it ran and rejected every finding on the existence check, which is the same outcome without the explanation.present-local/SKILL.mdcarries the operator handling.code-reviewplugin version is bumped and the rootCHANGELOG.mdcarries the v3.8.0 entry, perCONTRIBUTING.md.