Merged
Conversation
Removes the learn and extract-skills commands, the learners/ module, learning_service.py, and the continuous-learning CI workflow. The feature's premise is flawed: scores measure file presence not quality, metrics are fabricated, the extractor is hardcoded to 5 of 25 assessors, and the CI workflow was producing unenriched skill proposals on every release with LLM enrichment silently disabled. Also removes orphaned test files left over from the prior removal of eval_harness and benchmark commands. Closes #395 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Warning
|
| Layer / File(s) | Summary |
|---|---|
Prompt / Constants src/agentready/learners/prompt_templates.py |
Removed PATTERN_EXTRACTION_PROMPT and CODE_SAMPLING_GUIDANCE constants. |
Pattern Extraction src/agentready/learners/pattern_extractor.py |
PatternExtractor implemented to convert passing findings into DiscoveredSkill objects with confidence/impact/reusability scores and up to three code examples. |
Code Sampling src/agentready/learners/code_sampler.py |
CodeSampler implemented to select files/directories per attribute patterns, build directory trees, and format code samples with truncation. |
LLM Enrichment src/agentready/learners/llm_enricher.py |
LLMEnricher implemented: builds prompts, calls Anthropic Claude, parses JSON (handles fenced code), supports caching, retries on rate limits, merges enrichment into DiscoveredSkill. |
Skill Output Generation src/agentready/learners/skill_generator.py |
SkillGenerator implemented to write per-skill SKILL.md, GitHub issue markdown, and detailed reports; supports batch generation and all formats. |
Service Orchestration src/agentready/services/learning_service.py |
LearningService implemented to load assessment JSON, reconstruct domain objects, extract patterns (with min_confidence), optionally enrich via Anthropic, and generate outputs (json/skill_md/github_issues/markdown/all). |
CLI Command src/agentready/cli/extract_skills.py |
Adds extract-skills Click command: validates repo/assessment discovery, configures LearningService, optionally enables LLM enrichment when ANTHROPIC_API_KEY present, runs full workflow, prints results and next steps. |
CLI Wiring src/agentready/cli/main.py |
Updated LazyGroup lazy-subcommands mapping and comments to remove extract-skills and learn from heavy-command list; overall command registration adjusted. |
Project Metadata pyproject.toml |
project.description updated to remove “continuous learning” phrase. |
Tests — Added/Modified tests/unit/learners/test_llm_enricher.py, tests/unit/test_cli_extract_skills.py, tests/unit/test_code_sampler.py, tests/unit/learners/test_pattern_extractor.py, ... |
Extensive unit tests added/expanded for LLMEnricher, CodeSampler, PatternExtractor, SkillGenerator, and CLI extract-skills behavior. |
Tests — Removed tests/unit/test_learning_service.py, tests/integration/test_eval_harness_e2e.py, tests/unit/test_cli_learn.py, and several learner test modules indicated as deleted |
Several test modules covering the prior learn command and eval-harness integration were deleted. |
Sequence Diagram(s)
sequenceDiagram
autonumber
participant CLI as CLI (extract-skills)
participant LS as LearningService
participant PE as PatternExtractor
participant CS as CodeSampler
participant LLM as Anthropic Claude
participant SG as SkillGenerator
CLI->>LS: run_full_workflow(assessment, options)
LS->>LS: load_assessment -> build domain objects
LS->>PE: extract_all_patterns(assessment)
PE-->>LS: list[DiscoveredSkill]
LS->>CS: initialize per-repo (when enriching)
LS->>LLM: for up to budget -> request enrichment (prompt + samples)
LLM-->>LS: JSON enrichment (or error / retry)
LS->>PE: _merge_enrichment (merge into DiscoveredSkill)
LS->>SG: generate_skills(skills, output_format)
SG-->>LS: paths of generated files
LS-->>CLI: summary (counts, files, next steps)
Estimated code review effort
🎯 4 (Complex) | ⏱️ ~45 minutes
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
| Check name | Status | Explanation |
|---|---|---|
| Title check | ✅ Passed | The title 'Remove learn and extract-skills commands' directly and clearly summarizes the primary change—removal of two CLI commands. |
| Description check | ✅ Passed | The description is directly related to the changeset. It explains why the commands were removed (core premise invalid, implementation issues, CI problems) and references the linked issue #395. |
| Linked Issues check | ✅ Passed | The PR fully implements the removal objectives from issue #395: removes the learn and extract-skills commands, the learners module, the continuous-learning workflow, and cleans up orphaned test files. |
| Out of Scope Changes check | ✅ Passed | All changes are in scope. The PR removes only the specified commands, modules, workflows, and associated tests—no unrelated functionality or additions are present. |
| Docstring Coverage | ✅ Passed | Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. |
✏️ Tip: You can configure your own custom pre-merge checks in the settings.
✨ Finishing Touches
📝 Generate docstrings
- Create stacked PR
- Commit on current branch
🧪 Generate unit tests (beta)
- Create PR with unit tests
- Commit unit tests in branch
remove-learn-extract-skills
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands and usage tips.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/agentready/cli/main.py`:
- Around line 95-99: The lazy_subcommands dict has a mis-indented entry
("submit": ("submit", "submit")) causing a syntax error at import; fix by
aligning the "submit" key/value with the other entries inside the
lazy_subcommands mapping (ensure it is indented to the same column as
"assess-batch" and "experiment" and that the dict commas/braces remain correct)
so the dict definition in main.py (symbol: lazy_subcommands) parses properly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: e9b3b981-b933-4d08-8cbe-122389a6dc82
📒 Files selected for processing (21)
.github/workflows/continuous-learning.ymlpyproject.tomlsrc/agentready/cli/extract_skills.pysrc/agentready/cli/learn.pysrc/agentready/cli/main.pysrc/agentready/learners/__init__.pysrc/agentready/learners/code_sampler.pysrc/agentready/learners/llm_enricher.pysrc/agentready/learners/pattern_extractor.pysrc/agentready/learners/prompt_templates.pysrc/agentready/learners/skill_generator.pysrc/agentready/services/learning_service.pytests/integration/test_eval_harness_e2e.pytests/unit/learners/test_llm_enricher.pytests/unit/learners/test_pattern_extractor.pytests/unit/learners/test_skill_generator.pytests/unit/test_cli_extract_skills.pytests/unit/test_cli_learn.pytests/unit/test_code_sampler.pytests/unit/test_eval_harness_models.pytests/unit/test_learning_service.py
💤 Files with no reviewable changes (19)
- tests/unit/learners/test_skill_generator.py
- tests/unit/test_cli_learn.py
- .github/workflows/continuous-learning.yml
- src/agentready/cli/extract_skills.py
- tests/unit/test_learning_service.py
- tests/unit/test_eval_harness_models.py
- src/agentready/learners/init.py
- tests/integration/test_eval_harness_e2e.py
- src/agentready/learners/prompt_templates.py
- tests/unit/learners/test_llm_enricher.py
- src/agentready/learners/pattern_extractor.py
- tests/unit/learners/test_pattern_extractor.py
- src/agentready/learners/skill_generator.py
- src/agentready/learners/llm_enricher.py
- tests/unit/test_cli_extract_skills.py
- src/agentready/learners/code_sampler.py
- src/agentready/services/learning_service.py
- tests/unit/test_code_sampler.py
- src/agentready/cli/learn.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
📉 Test Coverage Report
Coverage calculated from unit tests only |
Contributor
|
🎉 This PR is included in version 2.35.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
learnandextract-skillsCLI commands and their sharedLearningServicelearners/module (code_sampler, llm_enricher, pattern_extractor, prompt_templates, skill_generator).github/workflows/continuous-learning.yml, which was running on every release and weekly with LLM enrichment silently disabledCloses #395
Why
As described in #395, the feature's core premise doesn't hold: scores measure file presence, not implementation quality. A 100 on
claude_md_fileonly means the file exists and exceeds 50 bytes. Skills derived from such signals are not reliable examples worth replicating.Additional implementation problems made it worse: metrics were fabricated, the extractor was hardcoded to 5 of 25 assessors (silently skipping all others), and the CI workflow produced unenriched proposals on a recurring schedule.
Test plan
agentready --helpno longer listslearnorextract-skillspytest tests/passes (one pre-existing e2e failure unrelated to this change, tracked in Fix failing e2e test: repomix_config not registered in create_all_assessors #400)🤖 Generated with Claude Code under the supervision of Bill Murdock