Retire the vestigial length-penalty in fitness scoring - #144
Merged
Conversation
The length_penalty in LLMJudge.score was dead code: no caller passes the artifact_size/max_size it needs, so it was always 0.0 and docked nothing. Length pressure is handled deliberately by the proposer's length budget and two hard deploy ceilings (_check_size/max_skill_size and the baseline-scaled effective_absolute_char_ceiling), and the tool path already rejects those size kwargs by design. Remove the field from FitnessScore, the ratio/penalty computation and unused artifact_size/max_size params from LLMJudge.score, the matching kwarg in ToolJudge, and the now-invalid kwarg/assertion in the tests. Behavior- preserving: composite values are unchanged since the penalty was always 0.0. Closes upstream triage #134 (investigated -> not applicable): the "pre-cap length-penalty cliff" premise doesn't hold on this fork, so the graduated-cap build would only re-introduce a deliberately-removed mechanism.
jramos
enabled auto-merge (squash)
June 28, 2026 20:45
jramos
added a commit
that referenced
this pull request
Jul 6, 2026
…ne new item (#150) Incremental upstream review. Backlog steady at 72 open; six new PRs #142-#147. Five already covered -> SKIP; one genuinely-new latent bug -> new action item. - NousResearch#142: 5 of 6 fixes are covered clusters (GEPA/DSPy-3.2 compat x3, validate-full). The 6th is real and new here: find_skill via Path.rglob('SKILL.md') skips symlinked skill dirs on Python <3.13 (skill_sources.py:63/67/80). Promoted to a CHERRY-PICK action item (recommended). - #143: our #102 — state.db read-only importer already shipped; only cross-platform discovery extras. SKIP. - #144/#145: Sunwo0u HSE sanitized evidence packets, report-only, no mechanism. SKIP. - #146: extraction cluster — our skill_text is a @Property over signature.instructions, so its 'dead input field' premise is upstream-only. SKIP. - #147: compat + material_diff reporting subsumed by our behavioral deploy gate. SKIP. Review-log entry + new action-item row + 2026-07-06 snapshot delta.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
length_penaltyinLLMJudge.scorewas dead code: no caller ever passed theartifact_size/max_sizeit needs, so it was always0.0and docked nothing. Lengthpressure is handled deliberately elsewhere — the proposer's length budget plus two hard
deploy ceilings (
_check_size/max_skill_sizeand the baseline-scaledeffective_absolute_char_ceiling) — and the tool path already rejects those size kwargsby design.
This removes the vestige:
FitnessScore: drop thelength_penaltyfield;compositeis now just theprofile-weighted sum, floored at 0.
LLMJudge.score: drop the unusedartifact_size/max_sizeparams and the ratio/penaltycomputation.
ToolJudge: drop thelength_penalty=0.0kwarg and refresh the stale comment.length_penalty=0.0kwarg and theresult.length_penaltyassertion (the
ToolJudge-rejects-size-kwargs test is kept — still valid).Behavior-preserving: the penalty was always
0.0, socompositevalues are unchanged.Triage close-out
Closes upstream triage #134 (graduated / class-aware skill-size cap) as
investigated → not applicable. The proposal's premise — a "pre-cap length-penalty cliff"
that docks under-cap skills — doesn't hold on this fork, since that penalty never fires.
Building the graduated cap would only re-introduce a deliberately-removed mechanism.
docs/upstream_pr_triage.mdis updated in this PR (row + review log + snapshot).Test Plan
uv run pytest tests/core/test_fitness.py tests/core/test_fitness_closed_loop.py tests/core/test_behavioral_metric.py tests/tools/ -q— 275 passeduv run pytest tests/ -q -m 'not slow'— 1699 passed, 4 deselectedgrep -rn -E "length_penalty|artifact_size|max_size"— only the intentionalToolJudge-rejects-size-kwargs test remains