fix(engine): make debtScore a measurement; scope rehydration candidates (OX-M6, OX-M7) - #38
Merged
Conversation
Audit OX-M7 and OX-M6. DECISIONS §64. Measured over a corpus frozen at 8b447ce -- 289 files, 578 rows, both CLI routes, ratio 0.3. M7: computeDebtBreakdown added metadata.originalBytes to elidedBytes for any item flagged `elided`. But originalBytes is the item's ENTIRE pre-transform length and `elided` is a boolean on the whole item, so an item that lost 5% of its bytes contributed 100% of its size to the numerator. On the CLI a single file is a single-item bundle, making elidedBytes === totalBytes whenever anything was elided at all. Baseline: 317 of the 317 rows carrying any debt scored debtScore exactly 35.00 -- the weightElisionRatio * 100 ceiling -- whether the file lost 4.7% or 66.8%. Math.min(1.0, ...) in calculateDebt was clamping a ratio with no business exceeding 1, which is why nothing looked wrong. The number was a constant wearing the name of a measurement. After (bytes actually removed): 0 of 317 at the ceiling, distribution 1.31 -> 34.99, and over the 101 reducing rows the implied ratio tracks the measured byte cut with correlation 1.0000. Output did not move: per-row across 578 rows the only field that changed is debtScore. outputSha, byteIdentical, tokenBefore, tokenAfter, reduction, fallbackUsed, driftScore, planMode and stageCount identical. The audit described a denominator mixing pre- and post-transform sizes. That is not what happened -- every stage setting `elided` also sets originalBytes, so the denominator was already clean. The numerator was the defect, and saturated rather than skewed. M6: attemptAutomatedRehydration guarded with `candidates && candidates.size > 0 && !candidates.has(item.id)`. The size > 0 clause is for the MISSING ledger case; it also swallowed a ledger reporting zero items below threshold, turning "nothing needs restoring" into "restore every elision in the bundle". No bundled entry point reaches it: the CLI passes neither hasher nor ledger, MCP and bench pass a hasher but no ledger, and the Gateway passes a ledger but no hasher and plans only session-dedup. It IS reachable through the exported `optimize` API. Reproduced there: a 1,481-byte item came back at exactly 1,481 bytes, every elision undone, with debtScore recomputed to 0 on the restored bundle so the trace showed no debt. Its corpus arm differs on 0 of 578 rows. That is what the structure predicts and is NOT evidence of correctness -- the harness runs CLI routes, which supply no ledger, so it cannot see the shape. §56's lesson with the sign reversed. The M6 test carries two controls because the first version passed against the unfixed code: the default maxDebtThreshold of 75 is unreachable on turn 1, so the branch it claimed to exercise never ran. Verified: typecheck, lint and build clean, 86 files / 791 tests. Both fixes confirmed failing against the unfixed tree first. Comparison engine built with an src-only tsconfig; dist hash moved 32c40f61c05e -> 3a7284583d99 -> d959d9b93e76 across the three arms. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Audit OX-M7 and OX-M6, both in
src/core/engine/index.ts. DECISIONS §64.Measured over a corpus frozen at
8b447ce— 289 files, 578 rows, both CLI routes, ratio 0.3.Three arms, comparison engines built with an src-only tsconfig;
disthash moved32c40f61c05e→3a7284583d99→d959d9b93e76, so no arm was compared against itself.M7 —
debtScorewas a constantcomputeDebtBreakdownaddedmetadata.originalBytestoelidedBytesfor any item flaggedelided. ButoriginalBytesis the item's entire pre-transform length andelidedis aboolean on the whole item — so an item that lost 5% of its bytes contributed 100% of its size to
the numerator. On the CLI a single file is a single-item bundle, making
elidedBytes === totalByteswhenever anything was elided at all.Whether the file lost 4.7% or 66.8%, it scored 35.00.
Math.min(1.0, …)incalculateDebtwasclamping a ratio with no business exceeding 1, which is why nothing ever looked wrong. Over the 101
reducing rows the implied ratio now tracks the measured byte cut with correlation 1.0000 —
0.047 against 0.047, 0.198 against 0.198, 0.429 against 0.429.
No output moved. Per-row across 578 rows the only field that changed is
debtScore;outputSha,byteIdentical,tokenBefore,tokenAfter,reduction,fallbackUsed,driftScore,planModeandstageCountare identical.The audit's stated mechanism was wrong and the finding was right. It described a denominator
mixing pre- and post-transform sizes. That does not happen — every stage setting
elidedalso setsoriginalBytes, so the denominator was already a clean sum of original sizes. The numerator wasthe defect, and it was saturated, not skewed.
M6 — an empty candidate set meant "restore everything"
The guard read
candidates && candidates.size > 0 && !candidates.has(item.id). Thesize > 0clause exists for the missing ledger case; it also swallowed the case where a ledger exists and
reports zero items below the confidence threshold, turning "nothing needs restoring" into
"restore every elision in the bundle".
Reachability was checked, not assumed. No bundled entry point hits it — the CLI passes neither
hasher nor ledger, MCP and
benchpass a hasher but no ledger, and the Gateway passes a ledger butno hasher and plans only
cleanup:session-dedup. It is reachable through the exportedoptimizeAPI. Reproduced there: a 1,481-byte item came back at exactly 1,481 bytes, every elisionundone, with
debtScorerecomputed to 0 on the restored bundle so the trace reported no debteither.
Its corpus arm differs on 0 of 578 rows — and that is not evidence of correctness. The harness
runs CLI routes, which supply no ledger, so it cannot see the shape at all. This is §56's lesson
with the sign reversed: byte-identical is not inert, and here it is not even informative.
The test carries two controls because the first version passed against the unfixed code. The
default
maxDebtThresholdof 75 is unreachable on turn 1 — confidence penalty 0, turn age 0,elision term capped at 35 — so the branch it claimed to exercise never ran. One control proves the
setup elides at all; the other proves the branch is entered under
maxDebtThreshold: 1, via theno-ledger path where a full restore is the intended behaviour rather than the bug.
What this does not establish
there.
correction is larger on multi-item bundles, but that is reasoned, not measured.
8b447ce; therecipe expected 62 TypeScript and 17 prose and selected 63 and 18, because the repo has grown.
Only the per-row A/B over this one frozen corpus means anything.
--max-debtstill gates nothing on the CLI. Debt is a real number now, not yet a live gate:the default threshold of 75 stays unreachable without a ledger. Whether the threshold or weights
should change is a separate question and was not touched.
Verification
npm run typecheck,npm run lint,npm run buildandnpx vitest runall pass: 86 files /791 tests. Both fixes confirmed failing against the unfixed tree first.
🤖 Generated with Claude Code