feat(skills): reorganize skills/ into domain suites - #189
Open
chr6192 wants to merge 2 commits into
Open
Conversation
Regroup the 9 flat non-eval_pipeline skills into three folder-based domain suites, matching the paradigm eval_pipeline already established (three-layer structure, directory-as-grouping, every <NN-name>/SKILL.md independently installable, no code-level registry): - academic-eval/ <- paper-review, bib-verify, + a copy of ref-hallucination-arena, plus a new 00-academic-router - arena-eval/ <- auto-arena, + a second independent copy of ref-hallucination-arena, plus a new 00-arena-router - openjudge-core/ <- openjudge, rl-reward (no router: sub-skills are mutually exclusive by description, see openjudge-core/README.md) ref-hallucination-arena is intentionally duplicated (not shared/symlinked) per the Agent Skill protocol's independent-installability constraint; the two copies are free to diverge. claude-authenticity/mmx-cli/find-skills-combo stay as standalone skills (no other skill references them). Also adds actor+judge functional-test coverage for the two new suites, reusing eval_pipeline's runner (parametrized via --skill-root/--cases/ --report-prefix, not duplicated) rather than forking it per suite. Initial audit: 21/21 test cases pass, including 3x-repeat checks on the trickiest routing-ambiguity cases (see each suite's tests/*_skill_audit.md). See docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md for the full design rationale and migration plan (Option D, section 7). Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
These 6 files under skills/eval_pipeline/ fail pre-commit's --all-files run in CI on plain upstream/main (pre-existing, unrelated to this PR's reorg work) — this branch was cut from main, not from cl_agentic_grader, so it doesn't carry that branch's earlier pre-commit fix for the same files. Reformats with black/isort and adds the same pylint-disable/f-string fixes already applied on cl_agentic_grader (PR #188) so this PR's CI passes too. Verified locally: pre-commit run --files <these 6 files> now passes black/isort/flake8/pylint/pyroma. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Regroups the 9 flat non-
eval_pipelineskills into three folder-based domain suites, applying the paradigmeval_pipeline(#187) already established: three-layer structure, directory-as-grouping, every<NN-name>/SKILL.mdindependently installable (Anthropic Agent Skill protocol), no code-level registry.skills/academic-eval/←paper-review,bib-verify, + a copy ofref-hallucination-arena, plus a new00-academic-routerentry pointskills/arena-eval/←auto-arena, + a second independent copy ofref-hallucination-arena, plus a new00-arena-routerentry pointskills/openjudge-core/←openjudge,rl-reward(no router — sub-skills are mutually exclusive bydescription, see the suite'sREADME.md)skills/README.md— new top-level suite index + the breaking-change path migration tableclaude-authenticity/mmx-cli/find-skills-combostay standalone (no other skill references them)ref-hallucination-arenais intentionally duplicated (not shared or symlinked) rather than "fixed" — the independent-installability constraint means a skill can't rely on content living outside its own folder, so the two copies are free to diverge over time.This is a breaking change for 6 previously published skill paths (see the migration table in
skills/README.md); migrations usedgit mvto preserve file history.Full design rationale:
docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md(Option D, §7).Testing
Added actor+judge functional-test coverage for the two new suites, reusing
eval_pipeline's runner (parametrized via--skill-root/--cases/--report-prefix) instead of forking it per suite — the runner isn't subject to the installability constraint (that applies toSKILL.mdcontent, not internal test tooling), and the change is backward-compatible (verifiedeval_pipeline's own default invocation produces byte-identical output filenames/report title).academic-eval: 12/12 test cases pass (skills/academic-eval/tests/)arena-eval: 9/9 test cases pass (skills/arena-eval/tests/)See each suite's
tests/*_skill_audit.mdfor full results and a caveat on why a clean first-run pass rate shouldn't be over-read (same author wrote both the router content and the test cases).Test plan
git mv-based renames preserve file history (git log --follow)paper-review,bib-verify,ref-hallucination-arena,auto-arena,openjudge,rl-reward)SKILL.mdfrontmatter has onlyname+description../../→../../../where a skill moved one level deeper)eval_pipeline's existing functional-test invocation still produces identical default output (regression-checked)academic-eval(12 cases) andarena-eval(9 cases), plus--repeat 3on ambiguous routing casesMade with Cursor