fix(audit): close oxaudit.md — the last four findings - #45
Merged
Conversation
…documented
Closes the last two Lane A items of oxaudit.md.
OX-M15 -- quality evaluation is opt-in, not opt-out.
`BenchmarkRunner.run` read `config.evaluateQuality !== false`, so it
defaulted on, and `BenchmarkEvaluator.evaluateFixture` executes each
fixture's code and its dataset checks through `python -c`. A harness
ARCHITECTURE.md calls offline and deterministic reached for an
interpreter because someone typed `bench`, and the only way out was
TOKENDAMPER_BENCH_DISABLE_PYTHON, documented nowhere.
The default is now `=== true`, and `--evaluate-quality` is the way to ask
for it. Command-scoped per DECISIONS §30: accepted on `bench`, a parse
error naming `bench` anywhere else.
Verified on the built artifact rather than in-process: plain
`bench humaneval --report-json` writes a report with **0** occurrences of
`python-subprocess`, and the same command with `--evaluate-quality`
writes **5**.
Two regression suites asserted on the execution-derived pass rate and now
ask for it by name, which is the half of the decision that keeps this
from being a silent loss of coverage -- bench.test.ts Test 5 and Test 6
both read 0.6 instead of 1.0 without it, because `syntaxPassRate` falls
back to validation outcomes. Same field name, different quantity; the
comment at each site says so.
OX-M13 -- `--minimum-confidence` and `--max-debt`, documented not fixed.
Both are parsed, range-validated and threaded into `optimize()`, and
neither can change what the CLI emits:
- validation confidence is binary (`passed ? 1 : 0`), so
`validation.confidence < minimum` is `1 < x` on a passing run -- false
for everything the schema admits -- and `0 < x` on a failing one, where
`!validation.passed` has already decided the same line. The ledger arm
defaults to a literal 1.0 when no ledger is supplied, and the CLI
supplies none.
- `--max-debt` can flip `shouldRehydrate` and enter the rehydration
branch, but `attemptAutomatedRehydration` returns on its first line
without a hasher or a ledger, and the CLI supplies neither.
That second reason is stronger than §64's ("the elision term caps at 35,
the threshold is 75"): `--max-debt` is precisely the flag that lowers the
threshold, so the default is not what makes it inert. §64's phrasing
happened to be right about the outcome for the wrong reason.
Documented rather than made live because the machinery is real and
reachable through the exported `optimize()`; only the CLI supplies
neither input.
`test/unit/cli/inert-dials.test.ts` pins it as a characterization test,
in the shape of validator-guarantee.test.ts -- it passes against this
tree on purpose, and fails if either dial becomes live, forcing the
README section to be rewritten in the same commit. Its control needed a
second attempt: `--max-drift 0` vs `--max-drift 1` emit identical bytes
here, because drift on the fixture is already 0.0000 and the gate asks
whether drift exceeds the threshold rather than reaches it. A budget flag
replaced it, since with none the planner returns `pass_through`.
The corpus was not run. It cannot see either change -- bench and the
flag-parse loop are both off the optimize path -- so a 578/578 identical
result would have been vacuous rather than reassuring.
Verified: typecheck, lint and build clean; 93 files / 838 tests passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rigin and Host
Closes oxaudit.md. DECISIONS §70.
OX-M8 -- an exposed bind must be authenticated.
The token gate read `if (this.config.gatewayToken && !isLoopbackPeer(req))`
-- enforced only *if one was configured*. `host: '0.0.0.0'` with no token
was an unauthenticated relay forwarding arbitrary bodies to upstream
providers, and nothing warned. README:154 already stated the intended
rule; the code implemented "enforced only if provided".
`start()` now throws, naming the host and the ways out.
`allowUnauthenticatedNonLoopback` is the explicit opt-in -- a separate
field rather than a magic token value, so the intent is legible in a
config file and greppable in a deployment.
Refusing rather than warning, because the configuration this protects is
a server nobody is watching: stderr reaches whoever starts it in a
terminal and no one who starts it from a unit file. Auto-generating a
token was the other candidate and is worse in a specific way -- startup
succeeds and every existing client begins failing 401, a subtler break
than a refusal that names itself.
Checked in `start()`, not the constructor, so constructing stays free of
side effects; the exposure begins at `listen`. `isLoopbackHost` treats
`0.0.0.0` and `::` as NOT loopback -- they include the loopback
interface, which is what makes them easy to mistake for it, and every
other interface besides. Loopback trust (C3) and the constant-time
compare are untouched and asserted, so a later change cannot quietly buy
this guarantee by revoking C3. `exec` is unaffected on two counts: it
binds the default loopback host and generates a token.
OX-M9 -- Origin and Host validation, not token-on-loopback.
Token-on-loopback splits browsers from local clients more cleanly, since
browsers cannot set custom headers on a simple request, but it taxes
every existing local client to close a browser-only hole. Declined.
Two corrections, both from measuring before fixing:
- The audit proposed answering OPTIONS with a restrictive CORS policy.
Measured first: the server ALREADY answers OPTIONS `405` with no
`Access-Control-*` headers, which is that policy. No handler added.
Preflight was never the gap -- the threat is a simple `text/plain`
POST, which skips it, so the check has to be on requests that never
preflight. Third OX reachability claim this project has corrected by
measuring.
- The recorded decision said non-browser clients "send neither header".
True of `Origin`; false of `Host`, which every HTTP/1.1 client must
send. The local client contract is preserved by what is ACCEPTED --
`localhost`, any `127.x`, `::1`, the configured bind -- not by absence.
`Origin` present and foreign -> 403 on every bind. `Host` naming
somewhere else -> 403 on a loopback bind only, where DNS rebinding is the
threat; on an exposed bind hostnames are legitimately varied and the
token M8 now requires is the real control. The policy runs BEFORE
`/health`: a check that endpoint sat in front of would be a check with a
documented way around it.
OX-L13 folded in as intended -- `/health` returns `{"status":"ok"}` and
no longer reports `sessionCount`. server.ts already carried a comment
deferring this here so the two answers could not drift; this is that
answer.
Verified on the built artifact, not only in vitest: exposed+tokenless
refuses and the message names both the host and the opt-in; loopback
tokenless still starts and serves; /health is `{"status":"ok"}`; foreign
Origin 403; foreign Host 403; OPTIONS 405.
Both suites drive real sockets rather than `mockUpstream`, because both
defects live in header handling a short-circuited upstream never
exercises. Ten M8 cases and eleven M9 cases were confirmed failing
against the unfixed tree first -- except the two OPTIONS cases, which
pass before and after and are labelled as guards rather than proofs,
since that behaviour was already correct.
The corpus was not run: every Gateway path is off the optimize route, so
byte-identical would have been vacuous rather than reassuring.
Verified: typecheck, lint and build clean; 95 files / 859 tests passing.
Typecheck caught a bad `Parameters<...>` in the new suite that vitest ran
green -- the reason the rule is typecheck AND build, not just tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes
oxaudit.mdin full. DECISIONS §70. Three of the four were decisions rather thandefects; one was a paragraph.
OX-M15 —
benchstops executing dataset codeBenchmarkRunner.runreadconfig.evaluateQuality !== false, so quality evaluation defaultedon, and the evaluator runs each fixture's code and its dataset checks through
python -c. Aharness
ARCHITECTURE.mdcalls offline and deterministic reached for an interpreter becausesomeone typed
bench, escapable only via the undocumentedTOKENDAMPER_BENCH_DISABLE_PYTHON.Decided: default off, opt-in by name.
--evaluate-qualityasks for it, command-scoped perDECISIONS §30.
Verified on the built artifact rather than in-process: plain
bench humaneval --report-jsonwrites a report with 0 occurrences of
python-subprocess; with the flag, 5.OX-M8 — an exposed bind must be authenticated
The gate read
if (this.config.gatewayToken && !isLoopbackPeer(req))— enforced only if one wasconfigured.
0.0.0.0with no token served an unauthenticated relay forwarding arbitrary bodiesto upstream providers.
Decided: refuse to start, with
allowUnauthenticatedNonLoopbackas the explicit opt-in.Refusing rather than warning because the configuration this protects is a server nobody is
watching — stderr reaches whoever starts it in a terminal, and no one who starts it from a unit
file.
OX-M9 (+ OX-L13) — Origin and Host validation
A foreign
Origin→403on every bind. A foreignHost→403on a loopback bind, whereDNS rebinding is the threat.
localhost, any127.x,::1and the configured bind are accepted,so local clients are unchanged. The policy runs before
/health.GET /healthnow returns{"status":"ok"}only —activeSessionstold an unauthenticated callerhow much traffic flows through the machine.
Two corrections to the finding, both from verifying before fixing:
server already answers OPTIONS
405with noAccess-Control-*headers, which is thatpolicy. No handler added. Preflight was never the gap: the threat is a simple
text/plainPOST, which skips it.
Origin, falseof
Host— every HTTP/1.1 client sends it. The local contract is preserved by what isaccepted, not by absence.
OX-M13 — documented, not fixed
--minimum-confidenceand--max-debtare parsed, range-validated, threaded intooptimize(),and cannot change what the CLI emits. Validation confidence is binary (
passed ? 1 : 0); theledger arm is a literal
1.0with no ledger, and the CLI supplies none.--max-debtcan enterthe rehydration branch, but
attemptAutomatedRehydrationreturns on its first line without ahasher or ledger.
That reason is stronger than the one DECISIONS §64 gave — §64 explained the default 75
threshold, and
--max-debtis precisely the flag that lowers it. Right outcome, argument thatdoes not carry.
test/unit/cli/inert-dials.test.tspins it as a characterization test in thevalidator-guarantee.test.tsshape: it passes against this tree on purpose, and fails if eitherdial becomes live, forcing the README section to be rewritten in the same commit.
Verification
npm run typecheck,npm run lint,npm run buildclean; 95 files / 859 tests passingcases — labelled guards rather than proofs, since that behaviour was already correct
names host and opt-in), loopback+tokenless still starts and serves,
/healthis{"status":"ok"}, foreign Origin403, foreign Host403, OPTIONS405mockUpstream, because both defects livein header handling a short-circuited upstream never exercises
Parameters<...>in a new suite that vitest ran green — the reason therule is typecheck and build, not just tests
The corpus was deliberately not run. Bench, the flag-parse loop and every Gateway path are off
the optimize route, so a byte-identical result would have been vacuous rather than reassuring —
§56's caution in the other direction.
🤖 Generated with Claude Code