fix(sdd): stop the planning artifact growing with steps × spec-size - #99
Open
grushikhin wants to merge 1 commit into
Open
fix(sdd): stop the planning artifact growing with steps × spec-size#99grushikhin wants to merge 1 commit into
grushikhin wants to merge 1 commit into
Conversation
A real 19-step task planned with default settings produced an 8282-line task file. Measured composition: 6374 lines in Implementation Process, of which 4509 were `#### Verification` blocks against 1865 lines of actual step content. 54% of the artifact was material no reader of the plan needs, and two blocks were byte-identical across all 19 steps (Project Guidelines Alignment 342 lines, Regular Checks 260 lines). This contradicts the marketplace's own stated principle that every token counts. Five changes: - qa-engineer writes the verification spec to a sidecar `<task-basename>.verification.md` with one `### Step N` section per step; the task file gets a one-line `**Verification:**` pointer plus the existing Verification Summary table. A judge scores one step, so it never needed the whole file inline. - qa-engineer emits Regular Checks and Project Guidelines Alignment once, in the sidecar preamble; steps reference them by name and list exceptions only. - implement-task resolves the spec location: sidecar when present, inline `#### Verification` otherwise, so task files written by the previous version keep working unchanged. - Verification level `None` was reachable only for mkdir/delete/config, so every real step got a full spec. It now also covers steps whose outcome is fully decided by a deterministic gate, and Panel is reserved for HIGH criticality with a failure mode no gate can catch. - tech-lead bounds step *count*, not only step size. Only size was bounded, so a ~40-file migration decomposed into 19 steps, each carrying an agent pair. Adds per-effort target bands with a hard ceiling, and flags ordering invariants that exist solely because one slice was split. Also: `--fast` included the `verifications` stage, so it cut judge rounds while leaving the artifact just as large. Removed from the alias; opt in explicitly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LeoVS09
reviewed
Aug 6, 2026
| ### Phase 6: Define Verifications | ||
|
|
||
| `qa-engineer` agent (opus) adds LLM-as-Judge verification sections with custom rubrics, thresholds, and verification levels (None, Single Judge, Panel of 2, or Per-Item) for each implementation step. | ||
| `qa-engineer` agent (opus) writes LLM-as-Judge verification specs with custom rubrics, thresholds, and verification levels (None, Single Judge, Panel of 2, or Per-Item) for each implementation step. The specs go into a sidecar file next to the task — `<task-basename>.verification.md`, one `### Step N` section each — and the task file gets a one-line `**Verification:**` pointer per step plus the `## Verification Summary` table. |
Contributor
There was a problem hiding this comment.
I suggest another way to make it. I have 2 options in mind:
- Remove from main task file all steps description, and create
.spec/subtasks/*.mdfiles. Each per task. Then main task will contain only checeklist for whole task verification. Majority of tasks "steps" is hard to verify properly, they may not yet contain tests written. So probably verification better to place not steps level, but at overral task level. - Alternatively, I thinking about combining all plan agents to a single one, that will perform majority of planing, and produce much shorter task spec. -> maybe there meaning to make something like that just in SADD plugin and remove SDD at all. Something like
/plan-do-judge
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What prompted this
Running
/sdd:plan-taskwith default settings (plus one--human-in-the-loopgate) on a real migration produced an 8282-line task file. The input spec was 3758 lines; the pipeline added 4524.Measured, not estimated:
## Implementation Process#### Verificationblocks## Acceptance Criteria## ArchitectureStep bodies are fine — 77 lines per step. The multiplier is
steps × per-step spec-size, and two blocks inside every spec were byte-identical across all 19 steps:##### Project Guidelines Alignment**Regular Checks:**That is 602 lines of pure duplication, against
CONTRIBUTING.md's own principle — "Minimal token footprint — every token counts", "Prompts should be concise".Changes
qa-engineerwrites the spec to a sidecar<task-basename>.verification.md, one### Step Nsection per step. Task file gets a one-line**Verification:**pointer per step plus the existing Verification Summary table.agents/qa-engineer.mdagents/qa-engineer.mdimplement-taskresolves the spec location — sidecar when present, inline#### Verificationotherwise.skills/implement-task/SKILL.mdNonemade reachable;Panelnarrowed.agents/qa-engineer.mdtech-leadbounds step count, not only step size.agents/tech-lead.md--fastno longer includesverifications.skills/plan-task/SKILL.mdWhy a sidecar
The verification spec has exactly one consumer, reading exactly one step: a judge scoring that step. The task file is read end-to-end by humans and by every other agent in the pipeline. Inlining made the plan carry 4509 lines that no reader of the plan needs. The judge's input is unchanged — it is told the sidecar path and the
### Step Nheading, so it now loads ~190 lines instead of scanning 8282.Why
NoneandPanelwere recalibratedNonewas reachable only for "mkdir, delete, JSON update", so effectively every real step received a full spec plus an agent pair. It now also applies when the step's outcome is fully decided by a deterministic gate — if the step is "add a thirdtscinvocation to the typecheck script" and CI failing is the whole test, a judge only restates the gate's verdict. The guard against over-using it is stated explicitly: could this step pass every gate and still be wrong? If yes, a judge is warranted.Panelnow additionally requires a failure mode no deterministic gate can catch.Why step count needed a bound
tech-leadalready bounds step size ("no step larger than Large") and forbids trivial steps — but nothing bounded count, so a ~40-file migration decomposed into 19 steps, each carrying an impl+judge pair. Adds per-effort target bands with a hard ceiling, and names the diagnostic symptom of over-splitting: ordering invariants that exist only because a slice was split ("repoint the import in step 4 before step 6 deletes the module"). The run that prompted this generated five such invariants plus two pages defending them; merging the steps deletes them for free.Why
--fastwas a trapIts alias included the
verificationsstage, so it reduced judge rounds while leaving output size untouched — the opposite of what a user reaching for--fastexpects. Removed from the alias, opt in explicitly.--one-shotalready excluded it and was the only flag that actually shrank the artifact.Backward compatibility
No migration needed.
implement-taskapplies a resolution rule before Phase 2:### Step Nsection + shared preamble#### VerificationNone, stated in the summaryTask files already written by
qa-engineer3.1.2 keep working untouched; the sidecar wins when both are present.Expected effect
On the task that prompted this: 8282 → ~3800 lines in the task file, with the verification spec intact in a sidecar of comparable size to what it replaced minus the 602 duplicated lines. Combined with the step-count bound, that same task would plan as ~8–12 steps rather than 19, cutting agent pairs proportionally.
Not included
Trimming the per-step spec itself — 5 score anchors to 3, and dropping
Test Cases to Coverin favour oftest_matrix+coverage_map. Both are defensible cuts (~another 1600 lines on the measured task) but they change what a judge evaluates, not merely where it lives, so they belong in a separate change with its own review.Verification
just set-version sdd 3.2.0andjust set-marketplace-version 3.6.0perCLAUDE.md(not edited by hand).docs/plugins/sdd/{plan-task,implement-task,usage-examples}.md.just sync-*not run — those targets copy onlyREADME.md, which needed no change.#### Verificationre-checked; remaining hits are theVerification Level Determinationheading and the intentional backward-compatibility references.plugins/customaize-agent:test-prompt— worth a run before merge, since changes 1–3 form a contract between two skills.🤖 Generated with Claude Code