doe-designer Claude Skill + minimum moment aberration

[kgdunn]

Summary

Opens a second distribution channel for the designed-experiments tooling: a Claude Skill (plus a plugin marketplace catalog) that users install into their own Claude account, alongside the existing hosted web app. The library is the computation; the skill carries the methodology.

• skills/doe-designer/ - a Claude Skill wrapping the ten @tool_spec DOE tools with the workflow knowledge an LLM does not reliably have on its own.
• .claude-plugin/marketplace.json - /plugin marketplace add kgdunn/process-improve works, and the skill can also be dropped into ~/.claude/skills/ or zipped for claude.ai.
• Minimum moment aberration (Xu, 2003) - a new moment_aberration public function and evaluate_design metric.

Why now

Vazquez, Rother and Charles-Gonzalez (arXiv:2512.17113v2, March 2026) evaluated GPT-5.1 and Gemini 2.5 Flash on 36 two-level fractional factorial construction tasks (8/16/32 runs, 4 to 26 factors, 10 replicates each). The models produce optimal designs reliably up to about eight factors and then degrade, and the degradation is the dangerous kind: compliant-looking tables that are actually resolution 1 or 2, non-regular arrays presented as regular fractions, and tables with missing cells.

That is the argument for the skill existing. An LLM asked to do DOE should be routed to a catalogue-backed generator and a verifier, not left to emit a design matrix token by token. SKILL.md leads with exactly that rule, and the new metric is what lets the verifier check a matrix it did not generate.

Minimum moment aberration

The existing minimum_aberration metric reads the word-length pattern off the defining relation, so it needs a DesignResult carrying generators. Handed a bare matrix it has nothing to work from, which rules out the case that matters most. Worth noting how wide that gap is in practice: generate_design does not populate DesignResult.generators for fractional factorials, so minimum_aberration returns "Not a fractional factorial design" even for designs this library generated itself.

Moment aberration is computed from the pairwise similarities between runs, so it needs only the matrix. It is equivalent to minimum aberration for regular designs, extends to non-regular ones, costs O(n^2 m^2) rather than O(n 2^m), and yields the design's strength and hence a resolution for any two-level matrix.

The lower bound is derived by writing delta = (m + h)/2 and expanding the u-th moment of h as a sum of squares over column tuples, minimised at n^2 E_u(m) where E_u(m) counts tuples whose columns all appear an even number of times. Every power sum is formed in exact Python integers via a similarity histogram and compared as Fractions, so strength detection is exact rather than tolerance-based. Bounds are reported only up to the first order the design misses; past that the bound is valid but so slack it can go negative, and reporting it would invite a meaningless comparison.

The skill

Three CLI entry points, all lint-clean and exercised end to end:

• doe_tool.py - generic dispatcher over the whole @tool_spec registry, JSON in and JSON out. Generic on purpose, so it cannot go stale as tools are added.
• verify_design.py - the gate. Reports the resolution a matrix actually has, with --require-resolution to fail outright (exit 3), and --compare to rank candidates by minimum moment aberration.
• render_plot.py - turns a visualize_doe chart spec into PNG/SVG/PDF via kaleido, or a self-contained interactive HTML page that needs nothing installed to view.

Plus four references (choosing a design, verification, analysis workflow, worked example) and runnable examples.

The worked example is a real run, not an illustration. Screening 7 factors in a 16-run resolution IV design recovers the two active factors, and the three aliased two-factor interactions come back with identical estimates of 1.773 - which makes what resolution IV costs visible rather than abstract, and shows up on the half-normal plot as a single point wearing three labels.

Test plan

• moment_aberration reproduces the published pattern for the 2^(7-3) design printed in full in the paper's Table 1: (3.27, 11.67, 42.47, 157.27, 591.27, 2251.67, 8666.47), resolution IV
• Reproduces the published patterns for the 8-, 16- and 32-run minimum aberration designs (Tables 3, 5, 7, 9, 11)
• Moment-derived resolution agrees with the shortest word in the defining relation across eight regular designs
• Catches the failure modes it exists for: duplicated column (res II), unbalanced column (res I), and a non-regular design the word-length route cannot evaluate at all
• E_u(m) closed form brute-force-checked against direct enumeration
• 55 new tests; full suite 2223 passed, 4 skipped
• ruff check . and mypy src/process_improve clean
• All three skill scripts run end to end (design -> verify -> analyse -> plot), including the non-zero exit path on a failing design

Checklist

• Version bumped in pyproject.toml (1.62.2 -> 1.63.0, MINOR: new public API and a new distribution channel; CITATION.cff kept in sync)
• Tests added or updated where relevant
• ruff check . passes
• CHANGELOG.md updated

Note

uv.lock is deliberately not committed, per CLAUDE.md.

[codecov]

Codecov Report

❌ Patch coverage is 95.32710% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/process_improve/experiments/evaluate.py	78.57%	3 Missing ⚠️
.../process_improve/experiments/_moment_aberration.py	97.82%	1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!