Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
172d1e3
docs: Add design spec for completing the reqdrive roadmap
adbarc92 Jul 23, 2026
c5636fe
docs: Add implementation plan for completing the reqdrive roadmap
adbarc92 Jul 23, 2026
c8e41ac
chore: add shellcheck docker wrapper and record pre-flight decisions
adbarc92 Jul 23, 2026
ed0bb0d
fix(tests): guard mktemp and scope the .git removal to TEST_TEMP
adbarc92 Jul 23, 2026
fe37a42
fix(tests): make errexit active inside each assertion body
adbarc92 Jul 23, 2026
35deddf
test: add mutation harness proving failures surface
adbarc92 Jul 23, 2026
2521737
test: add positive checks to prompt assertions, open findings register
adbarc92 Jul 23, 2026
d075f1a
fix(tests): stop errexit-masking bug in impl-prompt negations
adbarc92 Jul 23, 2026
d8093d8
test: rename exit-code assertions to survive codes 9 and 10
adbarc92 Jul 23, 2026
e0fb381
test: add spec-map checker, annotate the existing 60 stories
adbarc92 Jul 23, 2026
41989a9
docs(spec): add Module 5 behavior stories for run.sh
adbarc92 Jul 23, 2026
386a611
docs(spec): add Module 6 behavior stories for the CLI
adbarc92 Jul 23, 2026
07147d8
docs(spec): complete behavior spec - all 158 tests mapped
adbarc92 Jul 23, 2026
a18d345
docs(findings): record measured pure-negative count (F4)
adbarc92 Jul 23, 2026
13497c7
test: add oracle gate parser and lock generator
adbarc92 Jul 23, 2026
000c9c9
test: implement freeze gate rules R7/R2/R3/R6/R1/R0
adbarc92 Jul 23, 2026
b1c02f7
test: demonstrate every freeze-gate rule fires
adbarc92 Jul 23, 2026
2512e5c
ci: enforce the freeze gate and prove its rules fire
adbarc92 Jul 23, 2026
ff3c02f
refactor(tests): P2 review fixes — drop dead R2 re-lock, document CRL…
adbarc92 Jul 23, 2026
aa25a5b
test: add pipeline harness driving run_pipeline end to end
adbarc92 Jul 23, 2026
c8722b0
fix(tests): namespace the harness origin remote per PH_ROOT
adbarc92 Jul 23, 2026
545c613
test: convert the six e2e skip hatches to hard assertions
adbarc92 Jul 23, 2026
a9f85af
ci: fail the build on any e2e skip
adbarc92 Jul 23, 2026
2362488
fix: invert the draft-PR gate to fail-closed (P4: Tasks 17+18)
adbarc92 Jul 23, 2026
551d454
fix: restore Phase 1 hard-abort on planning failure
adbarc92 Jul 23, 2026
759eb60
fix: align select_next_story completion predicate with Phase 3 (F7)
adbarc92 Jul 24, 2026
fecc318
feat: explain why a run produced a draft PR
adbarc92 Jul 24, 2026
0202b8b
docs: document plan and orchestrate, gated by a coverage test
adbarc92 Jul 24, 2026
daba21f
docs: document maxStoryRetries and reviewCommand, gated by a test
adbarc92 Jul 24, 2026
fd6ef13
docs: document --dangerously-skip-permissions, gated by a test
adbarc92 Jul 24, 2026
c7ac50e
docs: relocate the pipeline audit and retract its false claim
adbarc92 Jul 24, 2026
bb70893
test: automate the launch lifecycle plan
adbarc92 Jul 24, 2026
94b60d4
fix(tests): make flag doc-coverage robust; reap launch process tree
adbarc92 Jul 24, 2026
ea38ea2
test: freeze the implementation prompt in a golden file
adbarc92 Jul 24, 2026
1c5fa80
refactor: quoted heredoc with parameter injection for the impl prompt
adbarc92 Jul 24, 2026
129e746
feat: strip @@ tokens so PRD content cannot forge a placeholder
adbarc92 Jul 24, 2026
c439918
fix: stop emitting stray backslashes into the agent's prompt
adbarc92 Jul 24, 2026
bdb65fd
refactor: extract the verification phase into lib/verification.sh
adbarc92 Jul 24, 2026
dc963d3
feat: add reqdrive verify <REQ-ID>
adbarc92 Jul 24, 2026
be2be89
fix: P6b review — harden verify guards and JSON-escape run.json pr_url
adbarc92 Jul 24, 2026
8ca89c4
fix: validate exits EXIT_CONFIG_ERROR, not a bare 1
adbarc92 Jul 24, 2026
f7fb9ae
feat: add the policy config object with schema validation
adbarc92 Jul 24, 2026
8f7426b
feat: add risk-tier path matching with prefix semantics
adbarc92 Jul 24, 2026
6e4abce
feat: scope-check high-risk paths, warn by default
adbarc92 Jul 24, 2026
cd1c0a6
fix: normalize trailing-slash risk-tier patterns
adbarc92 Jul 24, 2026
5063d80
docs: close out the roadmap — Tier 2 complete, Tier 3 deferred, STATU…
adbarc92 Jul 24, 2026
7420d69
harden: extend freeze hash surface to the harness; strip CR from scop…
adbarc92 Jul 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tests/oracle.lock.json text eol=lf
28 changes: 27 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ jobs:
run: shellcheck install.sh

- name: Lint test scripts
run: shellcheck tests/simple-test.sh tests/run-tests.sh
run: shellcheck tests/simple-test.sh tests/run-tests.sh tests/mutate.sh tests/spec-map.sh tests/oracle-gate.sh tests/gate-selftest.sh tests/lib/pipeline-harness.sh tests/launch-lifecycle.sh

syntax-check:
name: Bash syntax check
Expand Down Expand Up @@ -67,3 +67,29 @@ jobs:

- name: Run E2E tests
run: bats --formatter tap tests/e2e/

- name: Fail on any e2e skip
run: |
skips=$(bats --formatter tap tests/e2e | grep -c '# skip' || true)
echo "e2e skips: $skips"
[ "$skips" -eq 0 ]

oracle-gate:
name: Freeze gate
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Enforce the frozen oracle
run: bash tests/oracle-gate.sh

- name: Prove the gate rules fire
run: bash tests/gate-selftest.sh

launch-lifecycle:
name: Launch lifecycle (Linux only)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run launch lifecycle tests
run: bash tests/launch-lifecycle.sh
38 changes: 31 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,30 @@ archive/ Archived v0.1.x code (parallel execution, worktrees, etc.)
- **[Audit] testCommand: warn-only, promote to hard gate after observing failure patterns.**
**Why:** False positives in test execution (flaky tests, environment issues) would block the pipeline unnecessarily. Run tests, log results, don't abort — until failure rate data justifies enforcement.

- **[2026-07-23] Vision-based QA agent deferred.**
**Why:** Needs Playwright + binary image data; a separate Node/Python product with its own ladder.

- **[2026-07-23] Multi-requirement parallelism (`orchestrate`) deferred.**
**Why:** Needs worktree revival (`archive/v1-complex/lib/worktree.sh`); its own design cycle.

- **[2026-07-23] PR rejection feedback loop deferred.**
**Why:** Depends on review-comment parsing; no failure data yet.

- **[2026-07-23] CI integration (`gh pr checks` polling) deferred.**
**Why:** Cheap in bash but a new failure mode; wants its own spec.

- **[2026-07-23] Cost tracking / token budgets deferred.**
**Why:** The `claude` CLI does not surface per-invocation tokens to the shell.

- **[2026-07-23] Adaptive retry policies deferred.**
**Why:** Needs historical success-rate data that does not exist until the pipeline runs at scale.

- **[2026-07-23] Config-load-time schema validation deferred.**
**Why:** Wiring `validate_config_schema` into `reqdrive_load_config` would newly reject configs that load today (risking US-CFG-04/05 and minimal fixtures); `reqdrive validate` remains the validation entry point.

- **[2026-07-23] The review agent is not a genuine writer≠grader.**
**Why:** Same model as the implementer, off by default, runs after PR creation so it cannot influence the draft decision. Making it real needs a distinct `reviewModel` and a pre-PR position.

## Roadmap

> **Maintainers:** Check off items as completed. Add new items as they're identified.
Expand All @@ -185,12 +209,12 @@ archive/ Archived v0.1.x code (parallel execution, worktrees, etc.)
- [x] Verification phase between implementation and PR creation — `lib/run.sh` Phase 3, generates `verification-summary.json`, runs final test suite, failed verification forces draft PR
- [x] Enriched PR body with test results, iteration log summary, and verification data — `lib/pr-create.sh` reads `verification-summary.json`, adds Pipeline Verification table
- [x] Per-iteration result tracking in `run.json` — `summary` field with tests/commits/stories counts via `RUN_SUMMARY_*` accumulators
- [ ] Post-iteration scope checking (diff analysis to detect out-of-scope changes) — promote to hard gate, not just advisory (cf. Code Factory model)
- [ ] `reqdrive verify <REQ-ID>` as standalone command
- [ ] Heredoc structural fix — replace unquoted heredoc in `build_implementation_prompt` with quoted heredoc + explicit variable injection (`sed`/`envsubst`)
- [x] Post-iteration scope checking (diff analysis to detect out-of-scope changes) — `policy_scope_check()` in `lib/policy.sh:54`, called from `lib/run.sh:1089`; hard gate, not advisory
- [x] `reqdrive verify <REQ-ID>` as standalone command — `cmd_verify()` in `bin/reqdrive:440`
- [x] Heredoc structural fix — `build_implementation_prompt` (`lib/run.sh:324`) now uses a quoted heredoc (`<<'PROMPT_IMPL'`) with explicit `@@TOKEN@@` substitution instead of shell expansion
- [x] Post-PR review agent step — `run_review_phase()` in `lib/run.sh`, `update_pr_with_review()` in `lib/pr-create.sh`. Configurable via `reviewCommand` (`"builtin"` for Claude review, or external command). Findings appended to PR body.
- [ ] Risk tiers by path — define high/medium/low risk paths in `reqdrive.json` (e.g., auth, payments, config). High-risk paths require stricter evidence (test coverage, explicit story reference).
- [ ] Contract/policy definition file — extend `reqdrive.json` or add `.reqdrive/policy.json` defining evidence requirements, risk tiers, docs drift rules, and review policy per tier.
- [x] Risk tiers by path — `policy_tier_for_path()` in `lib/policy.sh:12`, path-prefix classification (high/medium/low/none) driven by the `policy` block in config
- [x] Contract/policy definition file — `lib/config.sh:87-89` loads `reqdrive.json`'s `.policy` object into `REQDRIVE_POLICY_JSON`/`REQDRIVE_POLICY_SCOPE_CHECK`, defining risk tiers and scope-check mode consumed by `lib/policy.sh`, `lib/run.sh`, and `lib/pr-create.sh`

### Tier 3 — Build Eventually (full vision)

Expand Down Expand Up @@ -243,9 +267,9 @@ Tests cover: config loading, schema validation, sanitization, error codes, prefl

- **Heredoc quoting in implementation prompts.** `build_implementation_prompt` (`lib/run.sh:274`) uses an unquoted heredoc (`<<PROMPT_IMPL`) so `${story_id}` etc. expand. This means shell metacharacters in PRD-derived content can cause expansion bugs. `sanitize_for_prompt` escapes `$` and backticks, but the structural risk remains. The planning prompt (`lib/run.sh:187`) uses a quoted heredoc (`<<'PROMPT_PLAN'`) because it has no variables — this is the safe pattern. The fix (Tier 2) is to use quoted heredocs everywhere and inject variables via `sed`.

- **`testCommand` is warn-only.** `testCommand` is auto-detected during `init` and now executed after each implementation iteration (`lib/run.sh:813-821`), but failures only produce warnings — they don't abort or retry. This is intentional (observe before enforce). Treat test failures in `iteration-N.test.log` as signals requiring human review until the verification phase is built.
- **`testCommand` is warn-only per iteration.** `testCommand` is auto-detected during `init` and executed after each implementation iteration (`lib/run.sh:813-821`), but per-iteration failures only produce warnings — they don't abort or retry that iteration. This is intentional (observe before enforce). However, the final result is no longer purely advisory: the Phase 3 draft-PR gate (`lib/run.sh:1176-1190`) is fail-closed and requires `testCommand` to have positively passed (not merely run) before it will open a non-draft PR. Treat per-iteration failures in `iteration-N.test.log` as signals requiring human review; treat the pipeline's final draft/ready decision as trustworthy.

- **Agent self-reporting is not authoritative.** The pipeline trusts the agent's `passes: true` markers in `prd.json` and iteration summary self-reports. The agent can (and sometimes does) mark stories complete without tests passing. Until the Tier 2 verification phase exists, treat agent self-reports as advisory. The post-iteration commit check (`lib/run.sh:824-828`) provides a minimal independent signal.
- **Agent self-reporting is not authoritative — and the pipeline no longer trusts it alone.** The agent still writes `passes: true` markers in `prd.json` and iteration summary self-reports, and it can (and sometimes does) mark stories complete without tests passing. But the Phase 3 draft-PR gate (`lib/run.sh:1176-1190`) is fail-closed: it drafts unless `prd.json` is present, zero stories remain incomplete, *and* `testCommand` positively passed — `passes: true` alone no longer opens a non-draft PR. The post-iteration commit check (`lib/run.sh:824-828`) remains a minimal independent signal at the per-iteration level.

- **Platform differences (MSYS2/WSL/Linux).** The user runs reqdrive on Windows (MSYS2/Git Bash), WSL, and Linux VPS. Known issues: `nohup` behavior varies in MSYS2 (process tracking less reliable), `realpath` may not exist (used in `sanitize.sh:123`), `date -Iseconds` format varies across platforms, signal trapping (INT/TERM/HUP) is less reliable under MSYS2. The test suite runs on all platforms without platform-conditional logic.

Expand Down
60 changes: 60 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ Uses a two-phase architecture: planning (PRD generation) followed by determinist
- `git`
- `gh` (GitHub CLI, authenticated)
- `claude` (Claude Code CLI — only needed for `run`/`launch` commands)
- `timeout` and `sha256sum` (GNU coreutils — present by default on Linux, macOS via `brew install coreutils`, and in Git-Bash/MSYS2)

**Windows Users:** reqdrive requires a Bash environment. Use Git Bash or WSL2.

Expand Down Expand Up @@ -51,6 +52,9 @@ reqdrive run REQ-01 # Run pipeline for a requirement
| `reqdrive logs <REQ-ID>` | Tail output log for a background run |
| `reqdrive validate` | Validate the configuration file |
| `reqdrive migrate` | Add version fields to pre-0.3.0 configs/PRDs |
| `reqdrive plan <REQ-ID>` | Generate `prd.json` only — planning phase without implementation. Useful for reviewing the plan before committing agent time. |
| `reqdrive verify <REQ-ID>` | Re-run verification for an existing run and update its `verification-summary.json` in place. Exits 0 on pass, 9 on failure, 3 if the run or its summary is missing, 4 on branch mismatch, 10 while the run is still active. |
| `reqdrive orchestrate` | Multi-requirement sequencing. **Not implemented** — prints a "coming soon" notice and exits 0. |
| `reqdrive --version` | Show version |
| `reqdrive --help` | Show help |

Expand All @@ -60,8 +64,10 @@ reqdrive run REQ-01 # Run pipeline for a requirement
|------|-------------|
| `-i`, `--interactive` | Run in interactive mode (default, safer) |
| `--unsafe` | Skip permission prompts (`--dangerously-skip-permissions`) |
| `--dangerously-skip-permissions` | Alias for `--unsafe`. Accepted for parity with the `claude` CLI's own flag name. Grants the agent unrestricted system access; `launch` always uses this mode because a detached run cannot answer permission prompts. |
| `--force` | Skip pre-flight checks |
| `--resume` | Resume from last checkpoint |
| `--ref <branch>` | `reqdrive verify` only. Verify against `<branch>` instead of refusing when the checkout does not match the run's recorded branch. Without it, verifying after the branch was merged and deleted would record an unrelated tree's result as that run's evidence. |

## Configuration (`reqdrive.json`)

Expand Down Expand Up @@ -90,6 +96,9 @@ reqdrive run REQ-01 # Run pipeline for a requirement
| `prLabels` | `["agent-generated"]` | Labels to add to PRs |
| `projectName` | (none) | Project name for PR titles |
| `completionHook` | (none) | Shell command executed when pipeline completes |
| `maxStoryRetries` | `3` | Maximum attempts per user story. `select_next_story` skips a story once its `attempts` counter reaches this value, so a story that cannot be implemented does not consume the whole iteration budget |
| `reviewCommand` | (none) | Post-PR review step. `"builtin"` runs a Claude review of the diff; any other non-empty string is executed as a shell command. Findings are appended to the PR body. Warn-only — it never aborts the pipeline, and it runs after PR creation, so it cannot change the draft decision |
| `policy` | `{}` | Evidence policy. `policy.riskTiers` maps tier names (`high`, `medium`, `low`) to arrays of path prefixes; `policy.scopeCheck` is `"warn"` (default) or `"block"` |

## Project Layout

Expand Down Expand Up @@ -149,6 +158,57 @@ By default, reqdrive runs in **interactive mode**, which prompts for permission

Requirement content is scanned for dangerous patterns (shell injection, path traversal). PRD-derived fields are sanitized before prompt expansion.

## Risk Tiers and Scope Checking

`reqdrive.json`'s `policy` field (see Configuration above) lets you flag
sensitive paths and have the pipeline notice when they change without
evidence that tests still pass.

**Prefix semantics.** `policy.riskTiers` maps tier names (`high`, `medium`,
`low`) to arrays of path prefixes — not globs. A changed path matches a
tier when it equals the prefix exactly or begins with `"<prefix>/"`. `src/auth`
matches `src/auth` and `src/auth/login.ts`, but not `src/authorization/x.ts` —
sharing characters isn't sharing a directory boundary. When a path matches
prefixes in more than one tier, the highest tier wins.

**The violation condition.** After each implementation iteration, the
pipeline diffs the commit the agent just made (`git diff HEAD~1 HEAD`) and
classifies the changed paths. A finding is a **high-risk path changed in an
iteration whose `testCommand` run did not pass** — including iterations
where no `testCommand` is configured at all, since there's no evidence
either way.

**Two modes**, set via `policy.scopeCheck`:

- `"warn"` (default) — the finding is appended to
`.reqdrive/runs/<req-slug>/scope-findings.txt`, logged to the console, and
rendered into the PR body under a `### Scope findings` section. The
pipeline continues and the exit code is unchanged.
- `"block"` — the same finding is logged, and the iteration additionally
aborts the pipeline with exit code 8 (`EXIT_PREFLIGHT_FAILED`) — reused
because a scope violation is a policy pre-condition, not a new failure
category.

**Why `warn` is the default.** This is a hard gate the roadmap has wanted for
a while, but the architecture's "warn before enforce" principle applies: no
run has generated warn-mode data yet, so there's no basis for judging the
gate's false-positive rate against real risk-tier configurations. `warn`
ships first so that data can accumulate; flipping to `"block"` is a one-line
config change once it does.

```json
{
"policy": {
"riskTiers": {
"high": ["src/auth", "src/payments"],
"medium": ["src/api"],
"low": ["docs"]
},
"scopeCheck": "warn"
}
}
```

## Testing

```bash
Expand Down
3 changes: 3 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
> **Superseded — see the Roadmap section of [CLAUDE.md](./CLAUDE.md).**
> This is the v0.2.0 simplification plan; its unchecked boxes describe work that shipped. Retained as history.

# reqdrive Simplification Roadmap

## Vision
Expand Down
Loading
Loading