fix(audit): close ten oxaudit findings across CLI, config and docs - #35
Merged
Conversation
Audit OX-H3. There was no vitest config, so collection used the default `include` and walked the whole tree. Agent worktrees live under `.claude/worktrees/<name>/` and each one is a full checkout with its own `test/`, so every stale worktree contributed a second, older copy of the entire suite. Measured: the canonical suite is 78 files / 723 tests. The audit's run, in a tree holding one stale worktree, reported 155 / 1410 -- almost exactly double, the extra half two commits behind main. The cost is not wall-clock. A stale copy passes against its own frozen source, so a green run says nothing about the tree being edited. That is invariant 10 arriving through the test runner. CI never saw it, because a fresh checkout has no `.claude/`, so "the suite" meant two different things locally and remotely. `exclude` restates node_modules and dist because specifying it replaces vitest's default instead of extending it, and dist is not hypothetical -- `tsc -p tsconfig.json` compiles test/ too, so a build leaves a second copy of every suite in dist/test/. `globals: true` makes tsconfig's existing `types: ["node", "vitest/globals"]` true rather than decorative; no test relies on it today. The guard pins both halves: that the include stays anchored, and that every suite really does live under test/ -- an anchor that quietly stopped collecting a directory would be this same defect with the sign reversed. It was mutation-checked, not assumed: widening the include, dropping .claude from exclude, deleting the config, and planting a suite outside test/ each fail it. Verified: typecheck clean, lint clean, 79 files / 727 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-H1. `runExecCommand` always resolved the real code; `runCli` threw it away -- the exec branch fired the promise, attached a .catch, and returned a synchronous 0 that had already been assigned to process.exitCode before the child finished. `tokendamper exec -- aider ... && next-step` ran next-step after a failed tool. `runCli` now returns `number | Promise<number>`. A union rather than a blanket async: optimize, bench and mcp know their code immediately and every existing caller reads the result directly. `await` handles both. The larger half was not in exec. package.json's bin points at dist/src/cli/main.js, so the shipped command runs the `require.main === module` block, not the exported main() that tests call. The two had drifted into separate copies of the same assignment, and the copy at the foot carried a `typeof exitCode === 'number'` guard that was harmless dead code while runCli always returned a number. Once exec returned a promise it became a silent drop: the installed binary would have kept exiting 0 with the whole suite green behind it. The block now delegates to main(), and the test pins the delegation as well as the codes, because the defect is duplication rather than any one line. Verified end to end on the built binary, not only through runCli: requested 0/3/42 give exit 0/3/42, an unresolvable command gives 1, and optimize still exits 0. The new test was checked against the unfixed tree first -- 3 of its cases fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M2, OX-M3, OX-M14. --diff is rendered on a multi-file run. It was honored on the single-file path and silently ignored here, so asking for a terminal diff over a directory produced nothing. renderTerminalDiff already takes a whole ContextBundle -- the capability was there, the call was not. --language is refused when a directory or more than one path is ingested, rather than stamped onto every file. It exists for input with no filename to classify by; a walk has one filename per file, so a blanket declaration can only overwrite a correct answer with a single wrong one. The harm is bigger than a mislabel. Measured on a three-file tree, `--language python` moved languageSupport from "1 unsupported (json)" to "3 supported, 0 unsupported", left astCoverage reading `unchecked: 0` because the Python validator genuinely did look at the JSON, and fell the whole run back. A coverage report that lies, with a guaranteed 0% behind it. Single-file and stdin are unchanged. The dropped-files warning no longer advises in both directions at once: "Raise --target-reduction-ratio's budget (a lower ratio prunes less)" had the verb and the parenthesis disagreeing. --input-name was reported with these and is NOT a defect. parseArguments already throws for any non-`-` input path, so it never no-oped on a directory. Checked rather than assumed, and pinned so it stays true. tools/corpus-harness spawns only single-file and stdin routes, so the --language rejection does not reach it. Verified: typecheck clean, lint clean, 80 files / 739 tests. The three new behavioral cases were confirmed failing against the unfixed tree first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M4 and OX-M16, with OX-L4 recorded at its site. Both are selection defects, and selection is load-bearing: the cache-aware prefix lock pins the first ~1,024 tokens and pinned items bypass the knapsack, so which files appear and in what order decides which survive (invariants 6, 7). SKIP_DIRECTORIES enumerated .git, .next and .venv by name, so .claude was missing -- and .claude/worktrees/<name>/ holds entire duplicate checkouts. Observed: `tokendamper optimize .` ingested this repo's own source twice. Skipping is now by shape, any dot-directory, because enumerating names never kept up: .agents, .cursor, .idea, .cache each arrive as a silent duplicate rather than an error. Naming a file inside one still takes it. Sorting now runs on separator-normalized keys. path.join builds native separators, so a bare sort compared `\` (0x5C) on Windows where `/` (0x2F) is compared elsewhere -- every digit and capital letter sorts between them. `src/a.ts` precedes `srcZ/a.ts` on POSIX because `/` < `Z` and follows it on Windows because `Z` < `\`: the same directory, two bundles, two sets of pinned files. Emitted paths stay native, since they are what the envelope prints and what the caller has to open. OX-L4 (symlinks skipped in a walk, followed when named) is documented rather than fixed. The fix is small but could not be exercised where it was written -- creating a symlink needs elevation or Developer Mode on Windows and returned EPERM -- and shipping an unverified change to the code deciding which files reach the pipeline is the trade this project keeps declining. Skipping omits a file, it never corrupts one. Verified: typecheck clean, lint clean, 81 files / 745 tests. All three behavioral cases confirmed failing against the unfixed tree first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M10, OX-M11, OX-M12. minimumConfidence reached TokenDamperConfig through a bare finite-number check, so TOKENDAMPER_MINIMUM_CONFIDENCE=1.5 forced a fallback on every run -- full output, 0% reduction, permanently, with no diagnostic naming the cause, because no confidence the pipeline computes reaches 1.5. Negatives were meaningless. Now rejected in the message style v1.6.0 gave the TOKENDAMPER_* enums. The audit named only the environment variable. The CLI flag carried the identical check and the config file was type-checked as `number`, so all three doors validate -- enumerate the doors, not the one reported. ARCHITECTURE.md's diagram described a pipeline that does not exist: it listed "Session Deduplication (TokenHasher) -> Delta Compression -> Workspace Topology Pruning". That is not the knapsack order, constraint-preservation was missing from the front, session-dedup is not in that plan at all, and it uses the session store, not TokenHasher. The file declares itself the canonical frozen reference, so contributors and agents were validating new stages against a fiction. It now documents all three real plan shapes, pass_through included -- the zero-stage case users keep mistaking for a bug. The new test derives the documented lists from the planner rather than trusting prose; all four of its cases fail against the previous text. bench/runner.ts no longer documents token-hashing's removed `new TokenHasher()` default. A comment asserting removed behavior invites someone to simplify the runner back onto it and silently recreate the fabricated-store defect. Verified: typecheck clean, lint clean, 82 files / 755 tests. The five minimumConfidence cases and all four doc cases were confirmed failing against the unfixed tree first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The changelog cites OX-* IDs throughout, so the audit itself belongs in the repo the way max_audit.md already is. Committed unmodified: like max_audit.md it records what was true when it was written, and editing it to match today would falsify that. Which creates the hazard this repo has hit twice in the other direction. A fresh audit sitting at the root, written in the present tense, reads as live -- and ten of its findings are already closed. So the disposition goes where the disposition is kept: docs/audit-remediation-status.md, the doc CLAUDE.md names as the current one. It records the Lane A/B split, what is closed, what is awaiting a decision rather than implementation, what needs the measurement loop, and that Lane B is untouched. It also records the two findings that did not survive checking -- M3(a), which is not a defect, and M10, which was scoped to one door and had three. Those are the reason the section exists rather than a footnote to it: an audit file is a set of claims, not a set of facts. The status doc's "Last updated" moves to 2026-08-23. No source change; typecheck, lint and 83 files / 769 tests unaffected and re-run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lane A of the
oxaudit.mdsplit (2026-08-23, ox-alpha). Ten findings closed across five commits.Lane B — the Gateway half (H2, H4, M1, M5, M8, M9 and five lows) — is unstarted and untouched here;
oxaudit-split.mdcarries the partition and the reason for it.The cut is by file ownership, not severity: nothing in this branch touches
src/gateway/{proxy,server,session-store,types}.ts,stages/cleanup/session-dedup.ts,stages/compression/token-hashing.ts,adapters/mcp/server.tsorbench/fixtures/loader.ts.What's in it
ca104d476f5398execdiscarded the child exit code84bdf5eb3f02f924712b9Three things worth reading the diff for
H1 was half a fix until the last moment.
package.json'sbinpoints atdist/src/cli/main.js,so the shipped command runs the
require.main === moduleblock at the foot ofmain.ts— not theexported
main()that every test calls. The two had drifted into separate copies of the sameassignment, and the shipped copy carried a
typeof exitCode === 'number'guard that was harmlessdead code while
runClialways returned a number. The momentexecreturned a promise it became asilent drop: the installed binary would have gone on exiting 0 with the entire suite green. The
block now delegates to
main(), and the test pins the delegation as well as the codes. Verified onthe built binary: 0→0, 3→3, 42→42, unresolvable command→1.
M3 is worse than the audit describes, and its part (a) is not a defect.
--input-namedoes notsilently no-op on a directory —
parseArgumentsalready throws for any non--input path (pinned soit stays true). But blanket
--languageover a mixed tree movedlanguageSupportfrom "1 unsupported(json)" to "3 supported, 0 unsupported", left
astCoveragereadingunchecked: 0because thePython validator genuinely did look at the JSON, and fell the whole run back. A coverage report
that lies with a guaranteed 0% behind it, so it is refused rather than documented.
M10 had three doors. The audit named
TOKENDAMPER_MINIMUM_CONFIDENCE; the CLI flag carried theidentical
Number.isFinite-and-nothing-else check and the config file was type-checked as barenumber. All three validate now.Verification
Every behavioral fix was confirmed failing against the unfixed tree first. The two config-only
fixes were mutation-checked rather than assumed:
.claudefrom exclude, deleting the config, andplanting a suite outside
test/each fail it.previous text.
Rebased onto
mainafter #34 landed.npm run typecheck,npm run lint,npm run buildandnpx vitest runall pass: 83 files / 769 tests.Deliberately not fixed
L4 — symlinks are skipped in a directory walk but followed when named directly. Recorded at its
site with the reason: creating a symlink in the environment this was written in returns
EPERMwithout elevation, and shipping an unverified change to the code that decides which files reach the
pipeline is the trade this project keeps declining. Skipping omits a file; it never corrupts one.
Two notes for whoever takes Lane B
tools/corpus-harness/measure.jspasses--trace-output stderr, so withdrawing that flag underH5 breaks the harness. Worth knowing before the implement-vs-withdraw call.
OX-*IDs throughout, butoxaudit.mdis not committed (unlikemax_audit.md), so those references currently dangle.🤖 Generated with Claude Code