From 52b69a27f8e2db643c5376b4135d453a36ac06c1 Mon Sep 17 00:00:00 2001 From: AGrushikhin Date: Sun, 2 Aug 2026 13:01:11 +0000 Subject: [PATCH] fix(sdd): stop the planning artifact growing with steps x spec-size A real 19-step task planned with default settings produced an 8282-line task file. Measured composition: 6374 lines in Implementation Process, of which 4509 were `#### Verification` blocks against 1865 lines of actual step content. 54% of the artifact was material no reader of the plan needs, and two blocks were byte-identical across all 19 steps (Project Guidelines Alignment 342 lines, Regular Checks 260 lines). This contradicts the marketplace's own stated principle that every token counts. Five changes: - qa-engineer writes the verification spec to a sidecar `.verification.md` with one `### Step N` section per step; the task file gets a one-line `**Verification:**` pointer plus the existing Verification Summary table. A judge scores one step, so it never needed the whole file inline. - qa-engineer emits Regular Checks and Project Guidelines Alignment once, in the sidecar preamble; steps reference them by name and list exceptions only. - implement-task resolves the spec location: sidecar when present, inline `#### Verification` otherwise, so task files written by the previous version keep working unchanged. - Verification level `None` was reachable only for mkdir/delete/config, so every real step got a full spec. It now also covers steps whose outcome is fully decided by a deterministic gate, and Panel is reserved for HIGH criticality with a failure mode no gate can catch. - tech-lead bounds step *count*, not only step size. Only size was bounded, so a ~40-file migration decomposed into 19 steps, each carrying an agent pair. Adds per-effort target bands with a hard ceiling, and flags ordering invariants that exist solely because one slice was split. Also: `--fast` included the `verifications` stage, so it cut judge rounds while leaving the artifact just as large. Removed from the alias; opt in explicitly. Co-Authored-By: Claude Opus 5 --- .claude-plugin/marketplace.json | 4 +- docs/plugins/sdd/implement-task.md | 2 +- docs/plugins/sdd/plan-task.md | 4 +- docs/plugins/sdd/usage-examples.md | 2 +- plugins/sdd/.claude-plugin/plugin.json | 2 +- plugins/sdd/agents/qa-engineer.md | 164 +++++++++++++++------ plugins/sdd/agents/tech-lead.md | 31 +++- plugins/sdd/skills/implement-task/SKILL.md | 46 ++++-- plugins/sdd/skills/plan-task/SKILL.md | 8 +- 9 files changed, 198 insertions(+), 65 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index c92ae3e..79af391 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -1,7 +1,7 @@ { "$schema": "https://anthropic.com/claude-code/marketplace.schema.json", "name": "context-engineering-kit", - "version": "3.5.0", + "version": "3.6.0", "description": "Hand-crafted collection of advanced context engineering techniques and patterns with minimal token footprint focused on improving agent result quality.", "owner": { "name": "NeoLabHQ", @@ -77,7 +77,7 @@ { "name": "sdd", "description": "Specification Driven Development workflow commands and agents, based on Github Spec Kit and OpenSpec. Uses specialized agents for effective context management and quality review.", - "version": "3.1.2", + "version": "3.2.0", "author": { "name": "Vlad Goncharov", "email": "vlad.goncharov@neolab.finance" diff --git a/docs/plugins/sdd/implement-task.md b/docs/plugins/sdd/implement-task.md index 7211f72..1fcd886 100644 --- a/docs/plugins/sdd/implement-task.md +++ b/docs/plugins/sdd/implement-task.md @@ -106,7 +106,7 @@ Reads the task file once and parses the `## Implementation Process` section: - Lists all steps with dependencies - Identifies parallel execution opportunities (`Parallel with:` annotations) -- Classifies verification needs from `#### Verification` sections +- Classifies verification needs from the resolved verification spec — the `.verification.md` sidecar, falling back to inline `#### Verification` sections in task files written by an older `qa-engineer` ### Phase 2: Execute Implementation Steps diff --git a/docs/plugins/sdd/plan-task.md b/docs/plugins/sdd/plan-task.md index 11328f0..1186a50 100644 --- a/docs/plugins/sdd/plan-task.md +++ b/docs/plugins/sdd/plan-task.md @@ -18,7 +18,7 @@ Refine a draft task specification into a fully planned, implementation-ready tas | `--max-iterations` | `--max-iterations N` | `3` | Maximum retry cycles per phase before moving on | | `--included-stages` | `--included-stages s1,s2,...` | All stages | Comma-separated list of stages to include | | `--skip` | `--skip s1,s2,...` | None | Comma-separated list of stages to exclude | -| `--fast` | flag | N/A | Alias for `--target-quality 3.0 --max-iterations 1 --included-stages business analysis,decomposition,verifications` | +| `--fast` | flag | N/A | Alias for `--target-quality 3.0 --max-iterations 1 --included-stages business analysis,decomposition`. Add `verifications` explicitly if `/implement` should score steps with a judge. | | `--one-shot` | flag | N/A | Alias for `--included-stages business analysis,decomposition --skip-judges` | | `--human-in-the-loop` | `--human-in-the-loop p1,p2,...` | None | Phases after which to pause for human review | | `--skip-judges` | flag | `false` | Skip all judge validation checks | @@ -133,7 +133,7 @@ Each sub-phase is validated by a judge agent. All three must pass before proceed ### Phase 6: Define Verifications -`qa-engineer` agent (opus) adds LLM-as-Judge verification sections with custom rubrics, thresholds, and verification levels (None, Single Judge, Panel of 2, or Per-Item) for each implementation step. +`qa-engineer` agent (opus) writes LLM-as-Judge verification specs with custom rubrics, thresholds, and verification levels (None, Single Judge, Panel of 2, or Per-Item) for each implementation step. The specs go into a sidecar file next to the task — `.verification.md`, one `### Step N` section each — and the task file gets a one-line `**Verification:**` pointer per step plus the `## Verification Summary` table. ### Phase 7: Promote Task diff --git a/docs/plugins/sdd/usage-examples.md b/docs/plugins/sdd/usage-examples.md index 7d1f581..7d1aa9a 100644 --- a/docs/plugins/sdd/usage-examples.md +++ b/docs/plugins/sdd/usage-examples.md @@ -64,7 +64,7 @@ Real-world scenarios demonstrating the effective use of the Spec-Driven Developm /implement-task @.specs/tasks/todo/fix-null-pointer-user-service.bug.md --skip-judges ``` -The `--fast` flag sets `--target-quality 3.0 --max-iterations 1 --included-stages "business analysis,decomposition,verifications"`, skipping research, codebase analysis, architecture synthesis, and parallelization. +The `--fast` flag sets `--target-quality 3.0 --max-iterations 1 --included-stages "business analysis,decomposition"`, skipping research, codebase analysis, architecture synthesis, parallelization, and verifications. Append `,verifications` to the stage list when `/implement` should score each step with a judge. --- diff --git a/plugins/sdd/.claude-plugin/plugin.json b/plugins/sdd/.claude-plugin/plugin.json index bf3f7a4..983bdf8 100644 --- a/plugins/sdd/.claude-plugin/plugin.json +++ b/plugins/sdd/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "sdd", - "version": "3.1.2", + "version": "3.2.0", "description": "Specification Driven Development workflow commands and agents, based on Github Spec Kit and OpenSpec. Uses specialized agents for effective context management and quality review.", "author": { "name": "Vlad Goncharov", diff --git a/plugins/sdd/agents/qa-engineer.md b/plugins/sdd/agents/qa-engineer.md index d229253..db40260 100644 --- a/plugins/sdd/agents/qa-engineer.md +++ b/plugins/sdd/agents/qa-engineer.md @@ -23,10 +23,12 @@ You are obsessed perfectionist with evaluation precision. Vague rubrics = UNRELI ## Goal -Produce a complete per-step evaluation specification (rubric dimensions, checklist with default quality items, scoring metadata, testing strategy) for each implementation step in the task file in scratchpad file, then write each specification to the task file as a `#### Verification` sections that a judge agent can apply mechanically to score implementation artifacts per step. -Use a scratchpad-first approach: analyze everything in a scratchpad file, then selectively update the task file with verification sections. +Produce a complete per-step evaluation specification (rubric dimensions, checklist with default quality items, scoring metadata, testing strategy) for each implementation step in the task file in scratchpad file, then write each specification to a **verification sidecar file** as `### Step N` sections that a judge agent can apply mechanically to score implementation artifacts per step. +Use a scratchpad-first approach: analyze everything in a scratchpad file, then write the verification sidecar and add one pointer line per step to the task file. -Each step must have a `#### Verification` section with appropriate verification level, custom rubrics, thresholds, and reference patterns. +Each step must have a `### Step N` section in the sidecar with appropriate verification level, custom rubrics, thresholds, and reference patterns. + +**Why a sidecar and not inline:** the verification spec is read by one consumer at a time — a judge scoring exactly one step — while the task file is read end-to-end by humans and by every other agent in the pipeline. Emitting all of it inline makes the artifact grow with `steps × spec-size`; a real 19-step task measured 4509 lines of verification against 1865 lines of actual step content, i.e. 54% of the file was material no reader of the plan needs. The sidecar keeps the judge's input identical while the plan stays readable. ## Input @@ -351,7 +353,7 @@ checklist: ## Stage 9: Final Verification Sections to Write -[For each step, the final `#### Verification` markdown block that will be inserted into the task file] +[The sidecar's shared preamble, plus for each step the final `### Step N` markdown block that will be written to `.verification.md`] ``` ``` @@ -449,24 +451,37 @@ Use this decision tree to determine verification level for each step: Is artifact type Directory/Deletion/Config? ├── Yes → Level: NONE │ -└── No → Is criticality HIGH? - ├── Yes → Level: Panel of 2 Judges +└── No → Is the step's outcome FULLY decided by a deterministic gate? + │ (build / lint / typecheck / test / grep-returns-zero — a judge would + │ only restate the gate's verdict and add no information) + ├── Yes → Level: NONE — record the gate as the verification instead │ - └── No → Are there multiple similar items? - ├── Yes → Level: Per-Item Judges (one per item) + └── No → Is criticality HIGH **and** is there a failure mode no deterministic + │ gate can catch (silent data loss, security, a green-but-wrong result)? + ├── Yes → Level: Panel of 2 Judges │ - └── No → Level: Single Judge + └── No → Are there multiple similar items? + ├── Yes → Level: Per-Item Judges (one per item) + │ + └── No → Level: Single Judge ``` ##### Verification Levels Reference | Level | When to Use | Configuration | |-------|-------------|---------------| -| ❌ None | Simple operations (mkdir, delete, JSON update) | Skip verification | +| ❌ None | Simple operations (mkdir, delete, JSON update), **or** any step whose success is fully decided by a deterministic gate | Skip verification; cite the gate | | ✅ Single Judge | Non-critical single artifacts | 1 evaluation, threshold 4.0/5.0 | -| ✅ Panel (2) | Critical single artifacts | 2 evaluations, median voting, threshold 4.0/5.0 | +| ✅ Panel (2) | Critical single artifacts with a failure mode a gate cannot catch | 2 evaluations, median voting, threshold 4.0/5.0 | | ✅ Per-Item | Multiple similar items | 1 evaluation per item, parallel, threshold 4.0/5.0 | +**`NONE` is a real answer, not an escape hatch.** A judge run costs a full agent invocation and +~190 lines of spec. If the step is "add a third `tsc` invocation to the typecheck script" and CI +failing is the whole test, `NONE` plus the named gate is the correct and cheaper specification. +Conversely, do not downgrade a step to `NONE` because it *has* a gate — only when the gate is +**sufficient**. The distinguishing question is: could this step pass every gate and still be wrong? +If yes, a judge is warranted. + ```markdown ## Verification Level Determination @@ -1805,16 +1820,68 @@ After self-verification is complete for every step, assemble the final per-step --- -### STAGE 9: Write to Task File +### STAGE 9: Write the Verification Sidecar + +The verification spec goes into a **sidecar file next to the task file**, not inline into it. + +#### 9.0 Sidecar File and Shared Preamble + +**Path:** same directory and basename as the task file, with the extension replaced by +`.verification.md`. For `.specs/tasks/todo/add-auth.feature.md` the sidecar is +`.specs/tasks/todo/add-auth.verification.md`. + +Open the sidecar with a **shared preamble** holding everything that is identical across steps. +Two blocks are always identical and MUST NOT be repeated per step — in the measured 19-step task +they cost 342 and 260 duplicated lines respectively: + +```markdown +# Verification Spec: [Task Title] + +**Task file:** `[path/to/task.feature.md]` +**Steps covered:** [N] + +> One `### Step N` section per implementation step. A judge reads the preamble plus the single +> section for the step it is scoring, and nothing else. + +## Shared: Regular Checks + + + +- [ ] Build passes: `[discovered build command, e.g., npm run build]` +- [ ] Lint passes with zero new errors/warnings: `[discovered lint command, e.g., npm run lint]` +- [ ] Tests pass: `[discovered test command, e.g., npm test]` +- [ ] No code duplication: new code does not duplicate function/logic/concept that already exists elsewhere +- [ ] Boy Scout Rule: scope-appropriate small improvements made to touched code (renames, dead-code removal, missing types) without scope creep +- [ ] Reuse honored: implementation imports/calls existing code specified in the architecture's "Reuses From" / "Reuse:" directives +- [ ] Every `test_matrix` row (main + edge + error) has a corresponding test +- [ ] Every entry in the **Test Cases to Cover** list has an implemented test + +## Shared: Project Guidelines Alignment -Now update the task file with the verification sections produced in Stages 3-8. + -#### 9.1 Verification Section Templates +**Guideline files discovered:** `[CLAUDE.md, CONTRIBUTING.md, .claude/rules/, ...]` + +[Short description paragraph — what this dimension means and covers.] + +[Classification / instruction paragraph — how the judge should classify the artifact.] + +Score Definitions + +- 1: [Condition] +- 2: [Condition (DEFAULT — must justify higher)] +- 3: [Condition (RARE — requires evidence)] +- 4: [Condition (IDEAL — requires evidence that it is impossible to do better)] +- 5: [Condition (OVERLY PERFECT — done much more than what is required)] +``` + +#### 9.1 Per-Step Section Templates ##### Template: No Verification ```markdown -#### Verification +### Step N **Rationale:** [Why verification is unnecessary - e.g., "Simple file operation. Success is binary."] **Level:** NOT NEEDED @@ -1824,12 +1891,13 @@ Now update the task file with the verification sections produced in Stages 3-8. ##### Template: Single Judge ```markdown -#### Verification +### Step N **Level:** ✅ Single Judge **Artifact:** `[path/to/artifact.md]` **Threshold:** 4.0/5.0 +**Regular Checks:** shared set applies. [Exceptions, e.g. "no test command — docs-only step" — or "no exceptions".] **Checklist:** @@ -1837,26 +1905,13 @@ Now update the task file with the verification sections produced in Stages 3-8. |----|----------|----------|------------| | [ID] | [Boolean YES/NO question] | hard_rule \| principle | essential \| important \| optional \| pitfall | -**Regular Checks:** - - - -- [ ] Build passes: `[discovered build command, e.g., npm run build]` -- [ ] Lint passes with zero new errors/warnings: `[discovered lint command, e.g., npm run lint]` -- [ ] Tests pass: `[discovered test command, e.g., npm test]` -- [ ] No code duplication: new code does not duplicate function/logic/concept that already exists elsewhere -- [ ] Boy Scout Rule: scope-appropriate small improvements made to touched code (renames, dead-code removal, missing types) without scope creep -- [ ] Reuse honored: implementation imports/calls existing code specified in the architecture's "Reuses From" / "Reuse:" directives -- [ ] Every `test_matrix` row (main + edge + error) has a corresponding test -- [ ] Every entry in the **Test Cases to Cover** list has an implemented test - **Rubric:** | Criterion | Weight | |-----------|--------| | [Criterion 1] | 0.XX | | | [Criterion 2] | 0.XX | | -| Project Guidelines Alignment | 0.XX | | +| Project Guidelines Alignment | 0.XX | see **Shared: Project Guidelines Alignment** | | ... | ... | ... | **Rubric Score Definitions:** @@ -1918,7 +1973,7 @@ Score Definitions ##### Template: Panel of 2 Judges ```markdown -#### Verification +### Step N **Level:** ✅✅ CRITICAL — Panel of 2 Judges with Aggregated Voting **Artifact:** `[path/to/artifact.md]` @@ -1930,7 +1985,7 @@ Score Definitions ##### Template: Per-Item Judges ```markdown -#### Verification +### Step N **Level:** Per-[Item Type] Judges ([N] separate evaluations in parallel) **Artifacts:** `[path/to/items/{item1,item2,...}.md]` @@ -1939,9 +1994,9 @@ Score Definitions ``` -#### 9.2 Add Verification to Each Step +#### 9.2 Write One Section per Step -For each step, add BOTH a `#### Verification` section AND all sections inside it. The specification (task file) uses **structured markdown** — NOT YAML — for the rubric, checklist, and test strategy. The scratchpad keeps the YAML form as the machine-readable source of truth; this stage transforms it into the human-readable markdown that the developer and judges will read in the task file. +For each step, add a `### Step N` section to the **sidecar file**. The sidecar uses **structured markdown** — NOT YAML — for the rubric, checklist, and test strategy. The scratchpad keeps the YAML form as the machine-readable source of truth; this stage transforms it into the human-readable markdown that the judges will read. 1. Use the appropriate template based on Stage 1's verification level determination 2. Fill in artifact paths from the step's Expected Output @@ -1952,7 +2007,7 @@ For each step, add BOTH a `#### Verification` section AND all sections inside it - Step-specific hard rules and TICK items - Applicable default checklist items — apply per-step conditional adjustments Do NOT emit the checklist as a YAML block in the spec file. -5. Include the Project Guidelines Alignment rubric dimension (if guidelines were discovered in Stage 1), with full score definitions, alongside the other rubric dimensions +5. Reference the Project Guidelines Alignment dimension by **name and weight only**, in the rubric table. Its description and score definitions live once in **Shared: Project Guidelines Alignment**. NEVER restate them per step — that duplication was 342 lines in a measured 19-step task. 6. Include reference pattern if one exists 7. Render the **Test Strategy** as a **structured markdown section** (NOT as a YAML block in the spec file). Order is load-bearing: a. prose metadata as `**Applies:**`, `**Artifact:**`, `**Criticality:**`; @@ -1960,11 +2015,23 @@ For each step, add BOTH a `#### Verification` section AND all sections inside it c. the **`Test Cases to Cover`** bullet list (format `- [type] description (AC-N)` per Stage 5's Case Listing Schema). **Omit the rest of the test strategy block from the spec file**. 8. Verify rubric weights sum to 1.0 -9. Render the regular checks section as a human-readable markdown checkbox list mirroring the default checklist items included in step (4). Substitute the actual discovered build/lint/test commands from Stage 1 (e.g., `just build`, `cargo clippy`, `pnpm test`). Omit any line whose corresponding items was dropped by Stage 3's conditional adjustments. The Regular Checks section is the human-facing CI-gate view; the structured markdown inside Verification is the human-readable specification, and the scratchpad's YAML remains the machine-readable source of truth. +9. Do NOT re-emit the regular checks list. Write the single line `**Regular Checks:** shared set applies.` plus this step's exceptions, if any. The full list lives once in **Shared: Regular Checks**, with the actual discovered build/lint/test commands from Stage 1 substituted there. A step that drops an item via Stage 3's conditional adjustments names only the dropped item as an exception. + +#### 9.2a Add the Pointer to the Task File + +The task file gets **one line per step** and nothing more. Insert it where the `#### Verification` section used to go: + +```markdown +**Verification:** ✅ Single Judge · threshold 4.0/5.0 · see `[task-basename].verification.md` → `### Step N` +``` + +Use the step's actual level and threshold so a reader of the plan sees the verification posture without opening the sidecar. For `NOT NEEDED` steps write `**Verification:** ❌ none — [one-clause rationale]` and no pointer. + +Do NOT copy rubrics, checklists, score definitions or test matrices into the task file. #### 9.3 Add Verification Summary -After all steps, add a summary table before `## Blockers` (or at end if no Blockers): +After all steps, add a summary table to the **task file** before `## Blockers` (or at end if no Blockers). This table stays in the task file — it is the plan-level view of verification posture and is small: ```markdown --- @@ -1978,6 +2045,7 @@ After all steps, add a summary table before `## Blockers` (or at end if no Block | 2b | ✅ Per-Item | N | 4.0/5.0 | [Brief description] | | ... | ... | ... | ... | ... | +**Verification Spec:** `[task-basename].verification.md` (one `### Step N` section per step) **Total Evaluations:** [Calculate total] **Default Checklist Items:** Included in [X] of [Y] steps (build/lint/tests/duplication/boy-scout/reuse — per per-step adjustments) **Project Guidelines Alignment Dimension:** Included in [X] of [Y] step rubrics (omitted only if no guideline files were discovered) @@ -2039,7 +2107,12 @@ Always specify a reference pattern when one exists. Judges use these to calibrat ## Output Format -Your output for each step MUST be a structured-markdown evaluation specification embedded inside a `#### Verification` section in the task file. The specification contains: rubric dimensions (as `####` markdown sections), checklist items (as a markdown table), test strategy (as structured markdown with tables), and scoring metadata. The scratchpad continues to use YAML for these same artifacts as the machine-readable source of truth; Stage 9 transforms scratchpad YAML into spec-file markdown. +Your output is **two files**: + +1. **The verification sidecar** — `.verification.md` next to the task file. A shared preamble (Regular Checks, Project Guidelines Alignment) plus one `### Step N` section per step. Each section is a structured-markdown evaluation specification containing: rubric dimensions (as `####` markdown sections), checklist items (as a markdown table), test strategy (as structured markdown with tables), and scoring metadata. +2. **The task file** — gains exactly one `**Verification:**` pointer line per step, plus the `## Verification Summary` table. Nothing else. + +The scratchpad continues to use YAML for these same artifacts as the machine-readable source of truth; Stage 9 transforms scratchpad YAML into sidecar markdown. --- @@ -2052,12 +2125,13 @@ Your output for each step MUST be a structured-markdown evaluation specification - ALWAYS define explicit score bins (1-5) for every rubric dimension. - NEVER include criteria that reward length, formatting, or style over substance. - ALWAYS ask for clarification when a step's success criteria are ambiguous. -- Every step MUST have a `#### Verification` section in the task file (even if level is NONE). +- Every step MUST have a `### Step N` section in the sidecar file (even if level is NONE) and a one-line `**Verification:**` pointer in the task file. - Rubric weights MUST sum to 1.0 within each step's rubric. - Default checklist items MUST be included by default and dropped only via the per-step conditional adjustments. -- Project Guidelines Alignment dimension MUST be included in every step's rubric when guideline files were discovered in Stage 1. +- Project Guidelines Alignment dimension MUST be included in every step's rubric when guideline files were discovered in Stage 1 — by **reference**, with its definition written once in the sidecar preamble. +- NEVER duplicate the Regular Checks list or the Project Guidelines Alignment score definitions into a per-step section. Both are shared-preamble content; repeating them cost 602 lines in a measured 19-step task. - Do NOT modify content before the first step or after Implementation Process (except adding Verification Summary before Blockers). -- Do NOT change step content, only add Verification sections. +- Do NOT change step content, only add the one-line Verification pointer. - Per-Item count MUST match actual number of items in the step. - Use proper tools (Read, Write) for file operations. - Pass criteria as separate, clearly named items with definitions, not buried in prose. @@ -2081,12 +2155,14 @@ Before completing verification definition, verify: - [ ] Test Strategy designed per applicable step with Decision Gates 0-6 walked (Stage 5) - [ ] Strategy Inputs (Criticality / Artifact surface / Dependencies in scope / Project test frameworks) captured per applicable step in Stage 5 - [ ] Custom rubric assembled per step (Stage 6) -- [ ] Project Guidelines Alignment dimension included in every applicable rubric (Stage 6.6) -- [ ] Test Strategy block (YAML + Test Matrix table + Test Cases to Cover bullet list) emitted in every Verification section where `test_strategy.applies = true` +- [ ] Project Guidelines Alignment dimension included in every applicable rubric (Stage 6.6), defined ONCE in the sidecar preamble and referenced by name and weight per step +- [ ] Test Strategy block (YAML + Test Matrix table + Test Cases to Cover bullet list) emitted in every sidecar step section where `test_strategy.applies = true` - [ ] RRD cycle applied per step (Stage 7) - [ ] Self-verification completed per step with 6 questions answered (Stage 8) - [ ] Rubric weights sum to exactly 1.0 for each step's rubric -- [ ] Verification sections added to ALL steps in the task file +- [ ] Sidecar file written with a shared preamble and one `### Step N` section per step +- [ ] Regular Checks list appears exactly ONCE in the sidecar, not once per step +- [ ] Task file has exactly one `**Verification:**` pointer line per step and no copied rubrics or checklists - [ ] Reference patterns specified where applicable - [ ] Verification Summary table added with correct totals - [ ] All identified gaps from self-verification addressed and task file updated diff --git a/plugins/sdd/agents/tech-lead.md b/plugins/sdd/agents/tech-lead.md index dd6d9bc..7af4f99 100644 --- a/plugins/sdd/agents/tech-lead.md +++ b/plugins/sdd/agents/tech-lead.md @@ -300,6 +300,30 @@ Each step costs at least one impl + one verification agent pair, and steps infla - YOU MUST size each step so it does enough verification-worthy work that the judge's run produces more value than its cost. If the verification would have nothing meaningful to check, the step is too small — merge it. - YOU SHOULD prefer one well-scoped step with multiple subtasks over two thin steps that each carry the full agent-pair overhead. +##### Step Count Upper Bound + +The "no step larger than Large" rule bounds step **size** only. Nothing bounds step **count**, so a +large task decomposes into an arbitrarily long list — a real migration touching ~40 files produced +**19 steps**, and each one carried an agent pair plus a verification spec. Apply an upper bound too: + +| Task effort | Target steps | Hard ceiling | +|-------------|--------------|--------------| +| S / M | 3-5 | 6 | +| L | 5-8 | 10 | +| XL | 8-12 | 14 | + +- YOU MUST report the step count against this band in the Implementation Summary, and justify in one + sentence any count above the target. +- If the count exceeds the **hard ceiling**, YOU MUST merge steps that belong to the same slice + before emitting. Steps that touch the same module, share the same test suites, and have no + external consumer between them are one step with more subtasks — not several. +- Watch for the tell-tale symptom of over-splitting: **ordering invariants that exist only because + of the split** ("repoint the import in step 4 before step 6 deletes the module"). Every such + invariant is a constraint you created and must now defend. Merging the two steps deletes the + invariant for free. +- A count above the ceiling is as much a defect as a step above Large. Both are reported, not + silently emitted. + #### Vertical Slicing Each task should deliver a complete, testable slice of functionality from UI to database. Avoid horizontal layers (all models, then all controllers, then all views). Enable early integration and validation. @@ -500,7 +524,7 @@ Phase 5: Polish | 1 | [Brief goal] | [Key output] | [S/M/L] | | 2 | [Brief goal] | [Key output] | [S/M/L] | -**Total Steps**: N +**Total Steps**: N — target band for [S/M/L/XL] is [X-Y], ceiling [Z]. [If above target: one-sentence justification.] **Critical Path**: Steps [X, Y, Z] are blocking **Parallel Opportunities**: Steps [A, B] can run concurrently @@ -543,6 +567,7 @@ Generate 8 questions based on specifics of your task breakdown. These are exampl | 4 | **TDD Integration**: Does every implementation step include test writing in its Definition of Done or subtasks? Have I placed test infrastructure as foundational tasks? | Scan all steps for test-related subtasks. Tests must not be afterthoughts. | | 5 | **Risk Identification**: Have I identified ALL high-complexity steps? For each, have I either decomposed further OR created preceding spike tasks? | Review Risks & Blockers Summary. All high-impact items need mitigations. | | 6 | **Step Sizing (Upper Bound)**: Is every step completable in 1-2 days? Are there any steps too large that should be broken down? | Review Implementation Summary effort column. No step should be >Large. | +| 6a | **Step Count (Upper Bound)**: Is the total step count within the band for this task's effort, and below the hard ceiling? Are there ordering invariants that exist only because a slice was split? | Compare Total Steps against the Step Count Upper Bound table. Merge same-slice steps until within the ceiling. | | 7 | **No Trivial Standalone Steps**: Does every step do more than a single trivial action (install/delete/copy/move/create-dir)? Are all trivial actions folded into the step that consumes them (or kept separate only under the documented shared-prerequisite exception)? | Scan every step. Flag any whose entire scope is a mechanical action. | | 8 | **Verification-Worthy Granularity**: Does every step do enough work to justify its verification agent's cost? Would the judge have something meaningful to check, or is the step too thin? | Review each step's Success Criteria and Subtasks. Thin steps must be merged. | @@ -564,6 +589,8 @@ For each question, you MUST provide: [ ] Success criteria are specific and testable (not vague) [ ] Subtasks use simple format: - [ ] Description with file path [ ] No step estimated larger than "Large" +[ ] Total step count within the band for this task's effort, below the hard ceiling +[ ] No ordering invariant exists solely because a single slice was split across steps [ ] No step is "Too Small / Trivial" (no standalone install/delete/copy/move/create-dir) [ ] Every step does enough work to justify its verification agent's cost [ ] Phases organized: Setup → Foundational → User Stories → Polish @@ -656,6 +683,8 @@ Before completing decomposition: - [ ] All steps have Goal, Output, Success Criteria, Subtasks, Blockers, Risks - [ ] Steps are ordered by dependency (no step depends on a later step) - [ ] No step estimated larger than "Large" +- [ ] Total step count within the band for this task's effort, below the hard ceiling +- [ ] No ordering invariant exists solely because a single slice was split across steps - [ ] No step is "Too Small / Trivial" — trivial actions folded into consuming steps - [ ] Every step does enough work to justify its verification agent's cost - [ ] Subtasks use simple format: - [ ] Description with file path diff --git a/plugins/sdd/skills/implement-task/SKILL.md b/plugins/sdd/skills/implement-task/SKILL.md index 2640863..8013719 100644 --- a/plugins/sdd/skills/implement-task/SKILL.md +++ b/plugins/sdd/skills/implement-task/SKILL.md @@ -74,7 +74,7 @@ When `--continue` is used: 1. **Step Resolution:** - Parse the task file for `[DONE]` markers on step titles - Identify the last incompleted step - - Launch the `sdd:code-reviewer` agent to verify the last INCOMPLETE step's artifacts (using the step's `#### Verification` specification embedded in the task file) + - Launch the `sdd:code-reviewer` agent to verify the last INCOMPLETE step's artifacts (using the step's verification spec, resolved per the appendix: the sidecar's `### Step N`, or an inline `#### Verification` section in older task files) - If `combined_score >= threshold` (or `>= 3.0` with only Low-priority issues): Mark step as done and resume from the next step - Otherwise: Re-implement the step using the reviewer's issues as feedback and iterate until PASS @@ -117,7 +117,7 @@ When `--refine` is used, it detects changes to **project files** (not the task f - For each changed file, determine which step created/modified it: - Check step's "Expected Output" section for file paths - Check step's subtasks for file references - - Check step's artifacts in `#### Verification` section + - Check step's artifacts in its resolved verification spec - Build a mapping: `{changed_file → step_number}` 3. **Determine Affected Steps:** @@ -541,7 +541,7 @@ Parse all flags from `$ARGUMENTS` and initialize configuration. - For each step, extract the files it creates/modifies from: - "Expected Output" sections - Subtask descriptions mentioning file paths - - `#### Verification` artifact paths + - verification spec artifact paths - Build mapping: `STEP_FILE_MAP = {step_number → [file_paths]}` 3. **Map Changed Files to Steps:** @@ -587,7 +587,7 @@ Parse the `## Implementation Process` section: - List all steps with dependencies - Identify which steps have `Parallel with:` annotations -- Classify each step's verification needs from `#### Verification` sections: +- Classify each step's verification needs from the resolved verification spec (sidecar `### Step N`, or inline `#### Verification`): | Verification Level | Code-Reviewer Dispatch | Threshold | |-----------------------------------|-------------|------------------------|-----------| @@ -615,7 +615,7 @@ Create TodoWrite with all implementation steps, marking verification requirement ## Phase 2: Execute Implementation Steps -For each step in dependency order, select the dispatch pattern by reading the step's `#### Verification` Level: +For each step in dependency order, select the dispatch pattern by reading the step's verification Level — available directly from the task file's one-line `**Verification:**` pointer, or from the resolved spec: | Verification Level | Pattern | |--------------------|---------| @@ -818,7 +818,7 @@ Inputs: **6. Determine Threshold and Apply Gate:** -- Check if step is marked as critical in task file (in `#### Verification` section or step metadata) +- Check if step is marked as critical in its verification spec (Level `Panel of 2 Judges`) or step metadata - If critical: use `THRESHOLD_FOR_CRITICAL_COMPONENTS` - If standard: use `THRESHOLD_FOR_STANDARD_COMPONENTS` @@ -1282,7 +1282,7 @@ After all steps complete and DoD verification passes: │ Phase 1: Load Task │ │ ┌─────────────────────────────────────────────────────────┐ │ │ │ Read $TASK_PATH → Parse steps │ │ -│ │ → Extract #### Verification specs → Create TodoWrite │ │ +│ │ → Resolve verification spec → Create TodoWrite │ │ │ └─────────────────────────────────────────────────────────┘ │ │ │ │ │ ▼ │ @@ -1401,7 +1401,7 @@ Phase 1: Loading task... Task: "Add form validation service" Steps identified: 4 steps -Verification plan (from #### Verification sections): +Verification plan (from the resolved verification spec): - Step 1: No verification (directory creation) - Step 2: Panel of 2 evaluations (ValidationService) - Step 3: Per-item evaluations (3 validators) @@ -1777,9 +1777,33 @@ Before completing implementation: This appendix documents how verification is specified in task files. During Phase 2 (Execute Steps), you will reference these specifications to understand how to verify each artifact. -### How Task Files Define Verification +### Where the Verification Spec Lives -Task files define verification requirements in `#### Verification` sections within each implementation step. These sections specify: +`sdd:qa-engineer` writes the verification spec to a **sidecar file** beside the task file: same +directory and basename, extension replaced by `.verification.md`. For +`.specs/tasks/in-progress/add-auth.feature.md` the sidecar is +`.specs/tasks/in-progress/add-auth.verification.md`. It holds a shared preamble (Regular Checks, +Project Guidelines Alignment) plus one `### Step N` section per step. The task file carries only a +one-line `**Verification:**` pointer per step and the `## Verification Summary` table. + +**Resolution rule — apply this once, before Phase 2:** + +| Condition | Where to read each step's spec | +|-----------|-------------------------------| +| Sidecar `.verification.md` exists | Its `### Step N` section, plus the shared preamble | +| Sidecar absent, task file has inline `#### Verification` sections | Those inline sections (task file written by an older `qa-engineer`) | +| Neither exists | No verification specified — treat every step as level `None` and say so in the summary | + +Both layouts are supported; the sidecar wins when both are present. Everywhere below that refers to +a step's `#### Verification` section means "the step's verification spec, resolved by this rule". + +**When dispatching `sdd:code-reviewer`, pass the resolved location explicitly** — the sidecar path +and the `### Step N` heading, or the task file path for the inline layout — so the reviewer loads one +step's spec and not the whole file. + +### How Verification Is Defined + +The verification spec defines, per step: ### Required Elements @@ -1860,7 +1884,7 @@ When the `sdd:code-reviewer` evaluates artifacts, it uses this 5-point scale for **During Phase 2 (Execute Steps):** 1. After a `sdd:developer` agent completes implementation -2. Read the step's `#### Verification` subsection +2. Read the step's verification spec (sidecar `### Step N`, or inline `#### Verification` in older task files) 3. Extract: Level, Artifact paths, Threshold 5. Launch the appropriate count of `sdd:code-reviewer` agent(s) based on Level 6. Pass exactly the 4 inputs to each reviewer (artifact, step number, specification path, CLAUDE_PLUGIN_ROOT) — **NEVER a threshold** diff --git a/plugins/sdd/skills/plan-task/SKILL.md b/plugins/sdd/skills/plan-task/SKILL.md index 23c9739..2fe8705 100644 --- a/plugins/sdd/skills/plan-task/SKILL.md +++ b/plugins/sdd/skills/plan-task/SKILL.md @@ -45,7 +45,7 @@ Parse the following arguments from `$ARGUMENTS`: | `--max-iterations` | `--max-iterations N` | `3` | Maximum implementation + judge retry cycles per phase before moving to next stage (regardless of pass/fail). | | `--included-stages` | `--included-stages stage1,stage2,...` | All stages | Comma-separated list of stages to include. | | `--skip` | `--skip stage1,stage2,...` | None | Comma-separated list of stages to exclude. | -| `--fast` | `--fast` | N/A | Alias for `--target-quality 3.0 --max-iterations 1 --included-stages business analysis,decomposition,verifications` | +| `--fast` | `--fast` | N/A | Alias for `--target-quality 3.0 --max-iterations 1 --included-stages business analysis,decomposition` - fewest judge rounds AND the smallest artifact. Add `verifications` explicitly if `/implement` should score steps with a judge. | | `--one-shot` | `--one-shot` | N/A | Alias for `--included-stages business analysis,decomposition --skip-judges` - minimal refinement without quality gates. | | `--human-in-the-loop` | `--human-in-the-loop phase1,phase2,...` | None | Phases after which to pause for human verification. | | `--skip-judges` | `--skip-judges` | `false` | Skip all judge validation checks - phases proceed without quality gates. | @@ -76,7 +76,11 @@ TASK_FILE = first argument that is a file path (must exist in .specs/tasks/draft if --fast present: THRESHOLD = 3.0 MAX_ITERATIONS = 1 - INCLUDED_STAGES = ["business analysis", "decomposition", "verifications"] + INCLUDED_STAGES = ["business analysis", "decomposition"] + # `verifications` is deliberately EXCLUDED: it is the single largest contributor to + # artifact size (measured at 54% of an 8282-line task file), so including it here + # made `--fast` cut judge rounds while leaving the output just as large. + # Opt in explicitly when needed: --fast --included-stages "business analysis,decomposition,verifications" if --one-shot present: INCLUDED_STAGES = ["business analysis", "decomposition"]