fix(compression): price a region without registering it (OX-M5) - #42
Merged
Conversation
Audit OX-M5, DECISIONS §68. trimRegionsToCeiling renders a marker for EVERY candidate span in order to price it -- correctly, since the marker is variable-length and self-describing and a saving estimated without it would overstate every region. But the renderer it was handed, markerFor, also called hasher.registerBlock(), so every span the ceiling considered and then discarded was written into the store anyway. Measured on a 12-region file: target 0.10 12 blocks registered, 5 markers emitted target 0.05 12 blocks registered, 3 markers emitted The store held one block per candidate rather than per elision, so memory grew with how much the scanner found rather than with how much was removed, and hasHash/expandBlockHash answered for placeholders present in no output. Invisible on the CLI, which supplies no hasher. It matters on MCP, where the server is long-lived and the hasher IS the reversibility store. priceMarker renders the same bytes without registering; registration now happens only on the two paths that actually elide. The binding constraint was that pricing and emission must render IDENTICAL bytes, or the ceiling is computed against a different string from the one written and --target-reduction-ratio adherence shifts. That holds by construction: renderElisionMarker is a pure function of the text, its noun and its hash, and hashContent is pure -- registration never contributed to the output. Corpus A/B, frozen at 48ac6c8, 289 files / 578 rows, both CLI routes, ratio 0.3. Comparison engines built with an src-only tsconfig; dist hashes 7a3bad0f515f and 83e544289dc4, so neither arm was compared against itself. ZERO rows differ on any of ten compared fields -- and 101 of them genuinely reduced, so the corpus contains the shape and the result is not vacuous. The A/B proves output-neutrality and nothing about the leak: CLI routes supply no hasher, so markerFor's registration branch was already a no-op there. The leak is measured by the new unit test, which supplies one. Verified: typecheck, lint and build clean, 90 files / 822 tests. Both count cases confirmed failing against the unfixed tree first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Audit OX-M5. DECISIONS §68. Fourth Lane B item, and the last one that doesn't need a decision
from you.
The defect
trimRegionsToCeilingrenders a marker for every candidate span in order to price it — which iscorrect, since the marker is variable-length and self-describing and a saving estimated without it
would overstate every region. But the renderer it was handed,
markerFor, also calledhasher.registerBlock(...). So every span the ceiling considered and discarded was written intothe store anyway.
The store held one block per candidate, not per elision. Memory grew with how much the scanner
found rather than with how much was removed, and
hasHash/expandBlockHashanswered forplaceholders present in no output anywhere.
Invisible on the CLI, which supplies no hasher at all. It matters on MCP, where the server
instance is long-lived and the hasher is the reversibility store.
The constraint that governed the fix
Pricing and emission must render identical bytes, or the ceiling is computed against a different
string from the one written and
--target-reduction-ratioadherence shifts.That holds by construction rather than by care:
renderElisionMarkeris a pure function of thetext, its noun and its hash, and
hashContentis pure. Registration never contributed to theoutput.
priceMarkerrenders the same bytes without it; registration now happens only on the twopaths that actually elide.
Corpus A/B — 578 of 578 rows byte-identical
Corpus frozen at
48ac6c8, 289 files, 578 rows, both CLI routes, ratio 0.3. Comparison enginesbuilt with an src-only tsconfig;
disthashes7a3bad0f515f(baseline) and83e544289dc4(candidate), so neither arm was compared against itself.
Zero rows differ, on any of the ten compared fields —
outputSha,byteIdentical,tokenBefore,tokenAfter,reduction,fallbackUsed,driftScore,debtScore,planMode,stageCount.And it is not a vacuous zero: 101 of those rows actually reduced, so markers were genuinely
priced and emitted. The corpus contains the shape the change touches — the check §56 exists to
demand.
What this does not establish
TokenHasher, so the registration branch was already a no-op there. The A/B provesoutput-neutrality — the constraint — and nothing about the leak. The leak is measured by
token-hashing-store-pollution.test.ts, which supplies a hasher directly.a real long-lived session accumulated was not measured — only that growth tracked candidates
rather than elisions.
bench. It supplies a hasher too and was equally affected, but its runs areshort-lived, so the accumulation had nowhere to build up.
Verification
npm run typecheck,npm run lint,npm run buildandnpx vitest runall pass: 90 files /822 tests. Both count cases were confirmed failing against the unfixed tree first, with the
byte-identity guard passing throughout — which is what isolates the leak from the constraint.
🤖 Generated with Claude Code