Skip to content

feat(skills): reorganize skills/ into domain suites - #189

Open
chr6192 wants to merge 2 commits into
mainfrom
cl_skills_domain_suite
Open

feat(skills): reorganize skills/ into domain suites#189
chr6192 wants to merge 2 commits into
mainfrom
cl_skills_domain_suite

Conversation

@chr6192

@chr6192 chr6192 commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Regroups the 9 flat non-eval_pipeline skills into three folder-based domain suites, applying the paradigm eval_pipeline (#187) already established: three-layer structure, directory-as-grouping, every <NN-name>/SKILL.md independently installable (Anthropic Agent Skill protocol), no code-level registry.

  • skills/academic-eval/paper-review, bib-verify, + a copy of ref-hallucination-arena, plus a new 00-academic-router entry point
  • skills/arena-eval/auto-arena, + a second independent copy of ref-hallucination-arena, plus a new 00-arena-router entry point
  • skills/openjudge-core/openjudge, rl-reward (no router — sub-skills are mutually exclusive by description, see the suite's README.md)
  • skills/README.md — new top-level suite index + the breaking-change path migration table
  • claude-authenticity / mmx-cli / find-skills-combo stay standalone (no other skill references them)

ref-hallucination-arena is intentionally duplicated (not shared or symlinked) rather than "fixed" — the independent-installability constraint means a skill can't rely on content living outside its own folder, so the two copies are free to diverge over time.

This is a breaking change for 6 previously published skill paths (see the migration table in skills/README.md); migrations used git mv to preserve file history.

Full design rationale: docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md (Option D, §7).

Testing

Added actor+judge functional-test coverage for the two new suites, reusing eval_pipeline's runner (parametrized via --skill-root/--cases/--report-prefix) instead of forking it per suite — the runner isn't subject to the installability constraint (that applies to SKILL.md content, not internal test tooling), and the change is backward-compatible (verified eval_pipeline's own default invocation produces byte-identical output filenames/report title).

  • academic-eval: 12/12 test cases pass (skills/academic-eval/tests/)
  • arena-eval: 9/9 test cases pass (skills/arena-eval/tests/)
  • 3x-repeat variance check on the 5 trickiest routing-ambiguity cases across both suites: all pass, no variance observed

See each suite's tests/*_skill_audit.md for full results and a caveat on why a clean first-run pass rate shouldn't be over-read (same author wrote both the router content and the test cases).

Test plan

  • git mv-based renames preserve file history (git log --follow)
  • No repo file still links to a pre-migration skill path (paper-review, bib-verify, ref-hallucination-arena, auto-arena, openjudge, rl-reward)
  • Every migrated/new SKILL.md frontmatter has only name + description
  • Relative links depth-adjusted for the new nesting (../../../../../ where a skill moved one level deeper)
  • eval_pipeline's existing functional-test invocation still produces identical default output (regression-checked)
  • Full functional-test suite run for academic-eval (12 cases) and arena-eval (9 cases), plus --repeat 3 on ambiguous routing cases

Made with Cursor

Regroup the 9 flat non-eval_pipeline skills into three folder-based domain
suites, matching the paradigm eval_pipeline already established (three-layer
structure, directory-as-grouping, every <NN-name>/SKILL.md independently
installable, no code-level registry):

- academic-eval/  <- paper-review, bib-verify, + a copy of
  ref-hallucination-arena, plus a new 00-academic-router
- arena-eval/     <- auto-arena, + a second independent copy of
  ref-hallucination-arena, plus a new 00-arena-router
- openjudge-core/ <- openjudge, rl-reward (no router: sub-skills are
  mutually exclusive by description, see openjudge-core/README.md)

ref-hallucination-arena is intentionally duplicated (not shared/symlinked)
per the Agent Skill protocol's independent-installability constraint; the
two copies are free to diverge. claude-authenticity/mmx-cli/find-skills-combo
stay as standalone skills (no other skill references them).

Also adds actor+judge functional-test coverage for the two new suites,
reusing eval_pipeline's runner (parametrized via --skill-root/--cases/
--report-prefix, not duplicated) rather than forking it per suite. Initial
audit: 21/21 test cases pass, including 3x-repeat checks on the trickiest
routing-ambiguity cases (see each suite's tests/*_skill_audit.md).

See docs/superpowers/specs/2026-07-09-skills-domain-suite-proposal.md for
the full design rationale and migration plan (Option D, section 7).

Co-authored-by: Cursor <cursoragent@cursor.com>
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@chr6192 chr6192 changed the title feat(skills): reorganize skills/ into domain suites (Option D) feat(skills): reorganize skills/ into domain suites Aug 3, 2026
These 6 files under skills/eval_pipeline/ fail pre-commit's --all-files run
in CI on plain upstream/main (pre-existing, unrelated to this PR's reorg
work) — this branch was cut from main, not from cl_agentic_grader, so it
doesn't carry that branch's earlier pre-commit fix for the same files.
Reformats with black/isort and adds the same pylint-disable/f-string fixes
already applied on cl_agentic_grader (PR #188) so this PR's CI passes too.

Verified locally: pre-commit run --files <these 6 files> now passes
black/isort/flake8/pylint/pyroma.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant