Skip to content

fix(audit): close ten oxaudit findings across CLI, config and docs - #35

Merged
ojassug merged 6 commits into
mainfrom
audit/lane-a
Aug 23, 2026
Merged

fix(audit): close ten oxaudit findings across CLI, config and docs#35
ojassug merged 6 commits into
mainfrom
audit/lane-a

Conversation

@ojassug

@ojassug ojassug commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Lane A of the oxaudit.md split (2026-08-23, ox-alpha). Ten findings closed across five commits.
Lane B — the Gateway half (H2, H4, M1, M5, M8, M9 and five lows) — is unstarted and untouched here;
oxaudit-split.md carries the partition and the reason for it.

The cut is by file ownership, not severity: nothing in this branch touches
src/gateway/{proxy,server,session-store,types}.ts, stages/cleanup/session-dedup.ts,
stages/compression/token-hashing.ts, adapters/mcp/server.ts or bench/fixtures/loader.ts.

What's in it

Commit Findings
ca104d4 H3 — vitest collected suites from other checkouts
76f5398 H1exec discarded the child exit code
84bdf5e M2, M3, M14 — multi-file route accepted flags it dropped
b3f02f9 M4, M16 — directory ingestion (+L4 recorded, not fixed)
24712b9 M10, M11, M12 — config validation, ARCHITECTURE drift, stale comment

Three things worth reading the diff for

H1 was half a fix until the last moment. package.json's bin points at dist/src/cli/main.js,
so the shipped command runs the require.main === module block at the foot of main.ts — not the
exported main() that every test calls. The two had drifted into separate copies of the same
assignment, and the shipped copy carried a typeof exitCode === 'number' guard that was harmless
dead code while runCli always returned a number. The moment exec returned a promise it became a
silent drop: the installed binary would have gone on exiting 0 with the entire suite green. The
block now delegates to main(), and the test pins the delegation as well as the codes. Verified on
the built binary: 0→0, 3→3, 42→42, unresolvable command→1.

M3 is worse than the audit describes, and its part (a) is not a defect. --input-name does not
silently no-op on a directory — parseArguments already throws for any non-- input path (pinned so
it stays true). But blanket --language over a mixed tree moved languageSupport from "1 unsupported
(json)" to "3 supported, 0 unsupported", left astCoverage reading unchecked: 0 because the
Python validator genuinely did look at the JSON, and fell the whole run back. A coverage report
that lies with a guaranteed 0% behind it, so it is refused rather than documented.

M10 had three doors. The audit named TOKENDAMPER_MINIMUM_CONFIDENCE; the CLI flag carried the
identical Number.isFinite-and-nothing-else check and the config file was type-checked as bare
number. All three validate now.

Verification

Every behavioral fix was confirmed failing against the unfixed tree first. The two config-only
fixes were mutation-checked rather than assumed:

  • H3's guard: widening the include, dropping .claude from exclude, deleting the config, and
    planting a suite outside test/ each fail it.
  • M11's guard derives the documented stage lists from the planner; all four cases fail against the
    previous text.

Rebased onto main after #34 landed. npm run typecheck, npm run lint, npm run build and
npx vitest run all pass: 83 files / 769 tests.

Deliberately not fixed

L4 — symlinks are skipped in a directory walk but followed when named directly. Recorded at its
site with the reason: creating a symlink in the environment this was written in returns EPERM
without elevation, and shipping an unverified change to the code that decides which files reach the
pipeline is the trade this project keeps declining. Skipping omits a file; it never corrupts one.

Two notes for whoever takes Lane B

  • tools/corpus-harness/measure.js passes --trace-output stderr, so withdrawing that flag under
    H5 breaks the harness.
    Worth knowing before the implement-vs-withdraw call.
  • The changelog now cites OX-* IDs throughout, but oxaudit.md is not committed (unlike
    max_audit.md), so those references currently dangle.

🤖 Generated with Claude Code

ojassug and others added 6 commits August 23, 2026 13:21
Audit OX-H3. There was no vitest config, so collection used the default
`include` and walked the whole tree. Agent worktrees live under
`.claude/worktrees/<name>/` and each one is a full checkout with its own
`test/`, so every stale worktree contributed a second, older copy of the
entire suite.

Measured: the canonical suite is 78 files / 723 tests. The audit's run,
in a tree holding one stale worktree, reported 155 / 1410 -- almost
exactly double, the extra half two commits behind main.

The cost is not wall-clock. A stale copy passes against its own frozen
source, so a green run says nothing about the tree being edited. That is
invariant 10 arriving through the test runner. CI never saw it, because a
fresh checkout has no `.claude/`, so "the suite" meant two different
things locally and remotely.

`exclude` restates node_modules and dist because specifying it replaces
vitest's default instead of extending it, and dist is not hypothetical --
`tsc -p tsconfig.json` compiles test/ too, so a build leaves a second copy
of every suite in dist/test/. `globals: true` makes tsconfig's existing
`types: ["node", "vitest/globals"]` true rather than decorative; no test
relies on it today.

The guard pins both halves: that the include stays anchored, and that
every suite really does live under test/ -- an anchor that quietly stopped
collecting a directory would be this same defect with the sign reversed.
It was mutation-checked, not assumed: widening the include, dropping
.claude from exclude, deleting the config, and planting a suite outside
test/ each fail it.

Verified: typecheck clean, lint clean, 79 files / 727 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-H1. `runExecCommand` always resolved the real code; `runCli`
threw it away -- the exec branch fired the promise, attached a .catch,
and returned a synchronous 0 that had already been assigned to
process.exitCode before the child finished. `tokendamper exec -- aider
... && next-step` ran next-step after a failed tool.

`runCli` now returns `number | Promise<number>`. A union rather than a
blanket async: optimize, bench and mcp know their code immediately and
every existing caller reads the result directly. `await` handles both.

The larger half was not in exec. package.json's bin points at
dist/src/cli/main.js, so the shipped command runs the
`require.main === module` block, not the exported main() that tests call.
The two had drifted into separate copies of the same assignment, and the
copy at the foot carried a `typeof exitCode === 'number'` guard that was
harmless dead code while runCli always returned a number. Once exec
returned a promise it became a silent drop: the installed binary would
have kept exiting 0 with the whole suite green behind it. The block now
delegates to main(), and the test pins the delegation as well as the
codes, because the defect is duplication rather than any one line.

Verified end to end on the built binary, not only through runCli:
requested 0/3/42 give exit 0/3/42, an unresolvable command gives 1, and
optimize still exits 0. The new test was checked against the unfixed
tree first -- 3 of its cases fail there.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M2, OX-M3, OX-M14.

--diff is rendered on a multi-file run. It was honored on the single-file
path and silently ignored here, so asking for a terminal diff over a
directory produced nothing. renderTerminalDiff already takes a whole
ContextBundle -- the capability was there, the call was not.

--language is refused when a directory or more than one path is ingested,
rather than stamped onto every file. It exists for input with no filename
to classify by; a walk has one filename per file, so a blanket declaration
can only overwrite a correct answer with a single wrong one.

The harm is bigger than a mislabel. Measured on a three-file tree,
`--language python` moved languageSupport from "1 unsupported (json)" to
"3 supported, 0 unsupported", left astCoverage reading `unchecked: 0`
because the Python validator genuinely did look at the JSON, and fell the
whole run back. A coverage report that lies, with a guaranteed 0% behind
it. Single-file and stdin are unchanged.

The dropped-files warning no longer advises in both directions at once:
"Raise --target-reduction-ratio's budget (a lower ratio prunes less)" had
the verb and the parenthesis disagreeing.

--input-name was reported with these and is NOT a defect. parseArguments
already throws for any non-`-` input path, so it never no-oped on a
directory. Checked rather than assumed, and pinned so it stays true.

tools/corpus-harness spawns only single-file and stdin routes, so the
--language rejection does not reach it.

Verified: typecheck clean, lint clean, 80 files / 739 tests. The three new
behavioral cases were confirmed failing against the unfixed tree first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M4 and OX-M16, with OX-L4 recorded at its site.

Both are selection defects, and selection is load-bearing: the cache-aware
prefix lock pins the first ~1,024 tokens and pinned items bypass the
knapsack, so which files appear and in what order decides which survive
(invariants 6, 7).

SKIP_DIRECTORIES enumerated .git, .next and .venv by name, so .claude was
missing -- and .claude/worktrees/<name>/ holds entire duplicate checkouts.
Observed: `tokendamper optimize .` ingested this repo's own source twice.
Skipping is now by shape, any dot-directory, because enumerating names
never kept up: .agents, .cursor, .idea, .cache each arrive as a silent
duplicate rather than an error. Naming a file inside one still takes it.

Sorting now runs on separator-normalized keys. path.join builds native
separators, so a bare sort compared `\` (0x5C) on Windows where `/`
(0x2F) is compared elsewhere -- every digit and capital letter sorts
between them. `src/a.ts` precedes `srcZ/a.ts` on POSIX because `/` < `Z`
and follows it on Windows because `Z` < `\`: the same directory, two
bundles, two sets of pinned files. Emitted paths stay native, since they
are what the envelope prints and what the caller has to open.

OX-L4 (symlinks skipped in a walk, followed when named) is documented
rather than fixed. The fix is small but could not be exercised where it
was written -- creating a symlink needs elevation or Developer Mode on
Windows and returned EPERM -- and shipping an unverified change to the
code deciding which files reach the pipeline is the trade this project
keeps declining. Skipping omits a file, it never corrupts one.

Verified: typecheck clean, lint clean, 81 files / 745 tests. All three
behavioral cases confirmed failing against the unfixed tree first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Audit OX-M10, OX-M11, OX-M12.

minimumConfidence reached TokenDamperConfig through a bare finite-number
check, so TOKENDAMPER_MINIMUM_CONFIDENCE=1.5 forced a fallback on every
run -- full output, 0% reduction, permanently, with no diagnostic naming
the cause, because no confidence the pipeline computes reaches 1.5.
Negatives were meaningless. Now rejected in the message style v1.6.0 gave
the TOKENDAMPER_* enums.

The audit named only the environment variable. The CLI flag carried the
identical check and the config file was type-checked as `number`, so all
three doors validate -- enumerate the doors, not the one reported.

ARCHITECTURE.md's diagram described a pipeline that does not exist: it
listed "Session Deduplication (TokenHasher) -> Delta Compression ->
Workspace Topology Pruning". That is not the knapsack order,
constraint-preservation was missing from the front, session-dedup is not
in that plan at all, and it uses the session store, not TokenHasher. The
file declares itself the canonical frozen reference, so contributors and
agents were validating new stages against a fiction.

It now documents all three real plan shapes, pass_through included --
the zero-stage case users keep mistaking for a bug. The new test derives
the documented lists from the planner rather than trusting prose; all
four of its cases fail against the previous text.

bench/runner.ts no longer documents token-hashing's removed
`new TokenHasher()` default. A comment asserting removed behavior invites
someone to simplify the runner back onto it and silently recreate the
fabricated-store defect.

Verified: typecheck clean, lint clean, 82 files / 755 tests. The five
minimumConfidence cases and all four doc cases were confirmed failing
against the unfixed tree first.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The changelog cites OX-* IDs throughout, so the audit itself belongs in
the repo the way max_audit.md already is. Committed unmodified: like
max_audit.md it records what was true when it was written, and editing it
to match today would falsify that.

Which creates the hazard this repo has hit twice in the other direction.
A fresh audit sitting at the root, written in the present tense, reads as
live -- and ten of its findings are already closed. So the disposition
goes where the disposition is kept: docs/audit-remediation-status.md, the
doc CLAUDE.md names as the current one. It records the Lane A/B split,
what is closed, what is awaiting a decision rather than implementation,
what needs the measurement loop, and that Lane B is untouched.

It also records the two findings that did not survive checking -- M3(a),
which is not a defect, and M10, which was scoped to one door and had
three. Those are the reason the section exists rather than a footnote to
it: an audit file is a set of claims, not a set of facts.

The status doc's "Last updated" moves to 2026-08-23. No source change;
typecheck, lint and 83 files / 769 tests unaffected and re-run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ojassug
ojassug merged commit a79d39f into main Aug 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant