Skip to content

Record four lessons from ai-config#1042's review lifecycle - #1104

Merged
dem-ucdh merged 5 commits into
mainfrom
ums/pr-1042-review-lessons
Aug 3, 2026
Merged

Record four lessons from ai-config#1042's review lifecycle#1104
dem-ucdh merged 5 commits into
mainfrom
ums/pr-1042-review-lessons

Conversation

@dem-ucdh

@dem-ucdh dem-ucdh commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

UMS pass capturing lessons from ai-config#1042's multi-round adversarial review of hooks/no-unreviewed-pr.py (the hook is unregistered until post-merge, so this PR touches no hooks/ and does not interact with #1042's branch).

This is a disjoint memories/skills PR: it edits four existing sections/files, adding hand-written prose (no generated trees).

Lessons recorded

  1. Full structural mirror across parallel guard paths -- extended shared/principles/fail-fast.md's existing "partial is worse than absent" section (did not create a new one). The new note covers the parallel-discharge-paths instance: no-unreviewed-pr.py has four sibling open/draft/request/self discharge-and-identity paths, and the fail-safe guard was applied to them one at a time across the review, each subsequent round surfacing the one path still unguarded: the shell-command parser underlying them, then the open path (open_ident), then the self discharge. That is the partial-guard failure stretched across rounds rather than shipped at once. Cross-references the discharge-mechanics section below it. (The earlier round-numbered phrasing was corrected during this PR's own review — the two numbering schemes in play, this session's own round count and Add a Stop hook catching a PR opened without a reviewer request #1042's internal review-comment numbering, did not reconcile, so the pattern is described by path instead.)

  2. bash git commit -m "..." runs backtick spans as command substitution -- new section in memories/git.md. The shell executes the backticked text and drops it from the message; hit live on Add a Stop hook catching a PR opened without a reviewer request #1042 (commit 97bf7d4 lost its backtick spans). The bash counterpart of CLAUDE.md's PowerShell backtick warning, by a different mechanism. Remedy: git commit -F <file> or single-quoted -m. (This PR's own commits used -F.)

  3. Python shlex shell-command-splitting gotchas -- new section in memories/tools.md, alongside the existing shell-gotcha family: (a) shlex drops an unquoted newline rather than emitting it as a separator, merging two lines' commands; (b) punctuation_chars=True uses the default set ();<>|&, conflating redirection (<,>) with control operators, so a redirect wrongly attaches inside a command. Use ;&|() as the separator set and convert newlines to ; first.

  4. Mutation-check each clause of an ANDed guard in isolation -- extended shared/workflow/algorithmatize-checks.md's non-vacuity section (already cites Carry forward the whole-file punctuation-replace hook stranded by #1045's merge #1047's fuzz()). A sibling clause in the same condition masks the mutation the same way a sibling test case aborts the suite: reverting one clause of if a and b and c still passes any case another clause keeps correct. Isolate each clause with a case where only that clause keeps the result correct.

Verification

  • Branched off origin/main; push verified against git ls-remote (matches HEAD).
  • Banned-punctuation scan over origin/main...HEAD: 0 hits on added lines.
  • Positive-controlled multi-sentence-line check over added .md lines: control OK, 0 flagged.

🤖 Generated with Claude Code

https://claude.ai/code/session_015icfsSybE9vMSt8cUrXB6X

From `hooks/no-unreviewed-pr.py`'s 9-round adversarial review (the hook is
unregistered until post-merge):

- `shared/principles/fail-fast.md`: extend the partial-guard section with the
  parallel-discharge-paths instance -- four sibling discharge/identity paths
  (open/draft/request/self) guarded one at a time across review rounds, which
  is the partial-guard failure stretched across rounds instead of shipped.
- `memories/git.md`: `git commit -m "..."` runs backtick spans as shell
  command substitution and drops them from the message; use `-F <file>` or
  single-quoted `-m`. The bash counterpart of CLAUDE.md's PowerShell backtick
  warning.
- `memories/tools.md`: two `shlex` gotchas when splitting a shell command into
  simple commands in Python -- newlines are dropped rather than emitted as
  separators, and the default `punctuation_chars` set conflates redirection
  (`<`,`>`) with control operators.
- `shared/workflow/algorithmatize-checks.md`: mutation-check each clause of an
  ANDed guard condition in isolation; a sibling clause masks the mutation the
  same way a sibling test case aborts the suite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015icfsSybE9vMSt8cUrXB6X
Copilot AI review requested due to automatic review settings August 3, 2026 21:25
@github-actions
github-actions Bot removed the request for review from Copilot August 3, 2026 21:25
Comment thread shared/principles/fail-fast.md
@claude

claude Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code review

Reviewed memories/git.md, memories/tools.md, shared/principles/fail-fast.md, and shared/workflow/algorithmatize-checks.md at head 3a1e67d (96 additions, 0 deletions).

This is a UMS pass recording four lessons from ai-config#1042's review lifecycle. I fact-checked the claims against the actual source PR (#1042, merged) rather than taking them at face value, since the PR description says this content may be partly machine-authored.

Confirmed accurate: the git commit -m backtick-command-substitution bug in memories/git.md (verified against commit 97bf7d4, which genuinely has backtick-spanned text missing from its message exactly as described). The two shlex gotchas in memories/tools.md (verified directly by running both reproduction snippets in Python 3, both behave exactly as claimed). The fuzz()/mutation-testing cross-reference in shared/workflow/algorithmatize-checks.md (the cited #1047 case and the ardi "seen to fail" phrase both check out). All internal cross-reference links ([fail-fast], [ardi]) resolve correctly. No banned punctuation on added lines, and no multi-sentence-line violations found, matching the PR's own verification claims.

One finding, posted inline: the round-number attributions in the shared/principles/fail-fast.md addition ("round 8 the open path, round 9 the self path", "a nine-round review") don't match #1042's actual posted review comments. Round 8 was about a different bug (request-discharge err/RX_REQ_FAILED logic), the open_ident fix is explicitly attributed to round 13 by that round's own comment, and the review had comments labeled through at least round 13. Details and citations are in the inline comment.

No other CLAUDE.md/lab-manual violations found. This is prose-only, and the additions follow the corpus's existing conventions for extending a section vs. creating a new one, semantic line breaks, and citation style.

Verdict

Needs more work. One finding: the round-number citations in the shared/principles/fail-fast.md addition should be corrected against #1042's actual review history before merge, see the inline comment for the specific quotes that contradict the current text.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Code review

Reviewed memories/git.md, memories/tools.md, shared/principles/fail-fast.md, and shared/workflow/algorithmatize-checks.md at head 3a1e67d (96 additions, 0 deletions).

This is a UMS pass recording four lessons from ai-config#1042's review lifecycle. I fact-checked the claims against the actual source PR (#1042, merged) rather than taking them at face value, since the PR description says this content may be partly machine-authored.

Confirmed accurate: the git commit -m backtick-command-substitution bug in memories/git.md (verified against commit 97bf7d4, which genuinely has backtick-spanned text missing from its message exactly as described). The two shlex gotchas in memories/tools.md (verified directly by running both reproduction snippets in Python 3, both behave exactly as claimed). The fuzz()/mutation-testing cross-reference in shared/workflow/algorithmatize-checks.md (the cited #1047 case and the ardi "seen to fail" phrase both check out). All internal cross-reference links ([fail-fast], [ardi]) resolve correctly. No banned punctuation on added lines, and no multi-sentence-line violations found, matching the PR's own verification claims.

One finding, posted inline: the round-number attributions in the shared/principles/fail-fast.md addition ("round 8 the open path, round 9 the self path", "a nine-round review") don't match #1042's actual posted review comments. Round 8 was about a different bug (request-discharge err/RX_REQ_FAILED logic), the open_ident fix is explicitly attributed to round 13 by that round's own comment, and the review had comments labeled through at least round 13. Details and citations are in the inline comment.

No other CLAUDE.md/lab-manual violations found. This is prose-only, and the additions follow the corpus's existing conventions for extending a section vs. creating a new one, semantic line breaks, and citation style.

Verdict

Needs more work. One finding: the round-number citations in the shared/principles/fail-fast.md addition should be corrected against #1042's actual review history before merge, see the inline comment for the specific quotes that contradict the current text.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

💰 Cost: $11.2834 (review) — run

claude added 2 commits August 3, 2026 21:41
Review of #1104 found the round-number attributions in the fail-fast.md
partial-guard case record ("nine-round review", "round 8 the open path",
"round 9 the self path") do not match #1042's actual review history: the
open_ident fix was that PR's round 13, the review ran past round 13, and the
same file's existing #1042 discharge case record already says "~12 review
rounds / rounds 8-10". The two numbering schemes (this session's count vs the
PR's own comment numbering) are irreconcilable and add no value to the lesson.

Describe the pattern by PATH instead (shell-command parser, then the open path,
then the self discharge), dropping the round numbers in both the fail-fast.md
addition and the memories/tools.md shlex citation. The lesson (guard all
parallel sibling paths in one change) does not depend on the numbering.
Copilot AI review requested due to automatic review settings August 3, 2026 21:41
@github-actions
github-actions Bot removed the request for review from Copilot August 3, 2026 21:42
Comment thread memories/git.md Outdated
@claude

claude Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code review

Reviewed memories/git.md, memories/tools.md, shared/principles/fail-fast.md, and shared/workflow/algorithmatize-checks.md at head 090be42 (96 additions, 0 deletions across the branch; this rounds fix commit touched shared/principles/fail-fast.md and memories/tools.md).

Prior finding resolved. The round-number citation mismatch flagged in the previous round (round 8/9/nine-round-review attributions that did not match the source PRs actual review history) is fixed in commit 090be42: the shared/principles/fail-fast.md case record now describes the sequence by path (the shell-command parser underlying them, then the open path (open_ident), then the self discharge) instead of by round number, and the memories/tools.md shlex citation dropped its round reference too.

One new finding, posted inline on memories/git.md line 355: the new backtick-command-substitution memory illustrative example relies on backslash-escaping a backtick inside a single-backtick code span, which CommonMark does not support (backslash escapes are not processed inside code spans, and delimiter matching is purely lexical). I traced the delimiter matching by hand and confirmed live via the rendered blob at this PRs head, it renders as garbled fragments rather than the intended two-span illustration. Suggested fix (verified to render correctly): use a double-backtick delimiter for the span containing the embedded backtick.

Independently re-verified the two shlex reproduction snippets in memories/tools.md (both behave exactly as shown) and confirmed the cited source PR is merged. No other CLAUDE.md/lab-manual violations found; internal cross-reference links resolve correctly, no banned punctuation on added lines, and no multi-sentence-line violations.

Verdict

Needs more work. One finding: fix the broken Markdown code-span rendering in memories/git.md (see inline comment) before merge.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


Review complete. Summary of findings:

PR: Morrison-Lab/ai-config#1104 — "Record four lessons from ai-config#1042's review lifecycle" (UMS pass, prose-only, 4 files, 96 additions)

Prior finding check: The round-number citation mismatch I flagged in the previous review round was fixed in commit 090be42 — the case records now describe the guard-path sequence by name (shell-command parser → open path → self discharge) instead of by round number, resolving the discrepancy against ai-config#1042's actual review history.

New finding (posted inline on memories/git.md:355): The new memory's illustrative example — `git commit -m "fix the \`slast\` guard"` commits `fix the guard` — relies on backslash-escaping a backtick inside a single-backtick code span. CommonMark doesn't process backslash escapes inside code spans (delimiter matching is purely lexical, matching the spec's own `foo\`bar` example). I traced the delimiter matching by hand and confirmed via the actual rendered blob on GitHub that this renders as garbled fragments rather than the intended two-span illustration. Posted a verified committable suggestion (double-backtick delimiter for the span containing the embedded backtick).

Independently re-verified: the two shlex reproduction snippets (both behave exactly as claimed), that PR #1042 is merged, that internal cross-reference links resolve, no banned punctuation on added lines, and no multi-sentence-line violations.

Verdict

Needs more work. One finding — the broken Markdown rendering in memories/git.md, with a verified fix suggested inline.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

💰 Cost: $10.0680 (review) — run

claude added 2 commits August 3, 2026 21:58
The example wrapped an outer code span in single backticks and used
backslash-escaped backticks for the inner `slast` span. CommonMark does
not process backslash escapes inside code spans, so the escapes rendered
literally on GitHub and garbled the example. Switch the outer span to a
double-backtick delimiter, which lets the inner single backticks be
literal without escaping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015icfsSybE9vMSt8cUrXB6X
Copilot AI review requested due to automatic review settings August 3, 2026 21:59
@github-actions
github-actions Bot removed the request for review from Copilot August 3, 2026 21:59
@claude

claude Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Code review

Reviewed memories/git.md, memories/tools.md, shared/principles/fail-fast.md, and shared/workflow/algorithmatize-checks.md at head 4b30781 (97 additions, 0 deletions). This round's diff versus the previous reviewed head (090be42) is two fix commits: a merge from main, and 4b30781 (Fix broken Markdown code-span in git.md commit-substitution example).

Both prior findings are resolved, independently re-verified.

  1. Round-number citation mismatch -- the shared/principles/fail-fast.md case record no longer cites round numbers; it now describes the sequence by path (the shell-command parser underlying them, then the open path (open_ident), then the self discharge). I independently confirmed open_ident, draft_ident, and request_ident all exist in hooks/no-unreviewed-pr.py, matching the four parallel open/draft/request/self description, and the ordering is consistent with the round 7 to round 13 to post-round-13 sequence already verified against Add a Stop hook catching a PR opened without a reviewer request #1042's actual review comments in the prior round.

  2. Broken Markdown code-span rendering -- memories/git.md now uses a double-backtick delimiter for the span containing the embedded backticks. I re-verified this independently by fetching the actual rendered blob at the current head: it now displays as two clean code spans (one showing the typed command, one showing what actually got committed) with no garbled fragments.

Fresh checks on this round's diff, no new issues found:

  • No banned punctuation (em/en-dash, curly quotes, multiplication sign) on added lines.
  • No multi-sentence-line violations detected.
  • All internal cross-reference links resolve: the fail-fast link target file and section, the ardi link target, and the CLAUDE.md PowerShell CLI Command Safety heading citation.
  • Independently re-ran both shlex reproduction claims in memories/tools.md (newline-drop and punctuation_chars=True defaulting to the default punctuation set) -- both confirmed exactly as described by direct execution.
  • hooks/no-unreviewed-pr.py's actual discharge condition is consistent with the three-clause AND description in the algorithmatize-checks.md addition.

No other CLAUDE.md/lab-manual violations found.

Verdict

Ready for merge.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Claude finished reviewView run


The confirming review posted successfully: #1104 (comment)

Summary

Reviewed PR #1104 at head 4b30781. Both findings from prior review rounds are resolved and I independently re-verified each:

  1. Round-number citation mismatch (round 1 finding) — fixed by describing the fix sequence by path instead of by round number. I independently confirmed the referenced functions (open_ident, draft_ident, request_ident) exist in hooks/no-unreviewed-pr.py and match the description.
  2. Broken Markdown code-span rendering (round 2 finding) — fixed with a double-backtick delimiter. I re-verified via a live fetch of the rendered blob that it now displays as two clean code spans.

Fresh checks on this round's diff (no new issues): no banned punctuation, no multi-sentence-line violations, all cross-reference links resolve, and I independently executed both shlex reproduction snippets in Python to confirm the technical claims.

Verdict

Ready for merge.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

💰 Cost: $7.4508 (review) — run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants