Count the span-density cost instead of timing it - #146
Open
wildthink wants to merge 1 commit into
Open
Conversation
The wall-clock ratio these assertions used was not portable, and gating CI on it turned main red: the same parser reads 5.3x on an idle laptop and 10.9x on a contended runner, which is above the 11.3x pre-rewrite floor, so no threshold separates the two parsers. 350b2d3 made them opt-in, which left the rewrite with no CI-visible cost guard. InlineParser.parse can now report an InlineParseCost — claimed-range probes and containment tests — and the assertions use that. Both quantities are pure functions of the input, so they read identically everywhere, and the separation is decisive rather than marginal: 6.0x for 6x the spans against 33.9x with the pre-rewrite pairwise containment restored, verified by reintroducing it. The bound stays at 8, now inside a deterministic gap. The counters are plain Ints beside comparisons the loops already do, threaded through rather than kept in a global, because swift-testing runs suites in parallel and a global would be raced by every other suite that parses. ClaimedIndex owns its own probe count; the scans take it inout so the caller can read it back. The timed assertions stay, still opt-in, for absolute numbers. What the counted form does not catch is a new scan bypassing ClaimedIndex and buildTree entirely — it holds the existing structures to linear rather than proving nothing quadratic exists anywhere. Corpus fingerprint unchanged, so the instrumentation changed no parse. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wildthink
force-pushed
the
perf/deterministic-span-cost-test
branch
from
August 10, 2026 13:14
ae54c04 to
36c98d2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #140 and to 350b2d3, and an answer to "happy to hear an argument for a different number": I don't think there is a better number, because the thing being measured is wrong. This replaces the timing with a count.
Sorry for the red CI — the bound was mine and the reasoning behind it was wrong in a way your commit message pins exactly.
Why no threshold worked
I justified those bounds with "minimum of several runs; noise only ever adds time, so the floor is stable." That holds for an absolute measurement on an idle machine. It does not hold for the ratio I actually asserted: under sustained contention there is no quiet run to find a floor in, and the smaller measurement inflates proportionally more, so the ratio drifts up with load rather than staying put. 5.3x here, 10.9x on the runner, same parser. Your point that the post-rewrite CI reading sits above the pre-rewrite floor is the end of the argument — any bound green on CI would have passed the parser the test exists to catch.
Counting instead
InlineParser.parsecan now report anInlineParseCost: claimed-range probes and containment tests. Both are pure functions of the input, so they read the same on your laptop, mine, and a loaded runner — and the separation is decisive instead of marginal:Exactly 6.0x for every construct — code, links, emphasis, highlight, mixed, and link labels with nested code spans. I got the 33.9x by actually restoring the old pairwise containment and re-running, not by extrapolating. The bound stays at 8, which now sits inside a gap that doesn't move.
These run on CI. The timed assertions stay exactly as you left them — opt-in, useful for absolute numbers — and I corrected the
msPerParsecomment, which still asserted the floor reasoning I'd used to justify the ratio.On the instrumentation
Counters are plain
Ints bumped beside comparisons the loops already perform, threaded through rather than parked in a global. The global would have been a smaller diff and wrong: swift-testing runs suites in parallel, so every other suite that parses markdown would land in the counter mid-measurement.ClaimedIndexowns its own probe count and the scans take itinoutso the caller reads it back;buildTreetakes the costinout.That is a real cost in the hot path — two integer increments — which I judged worth it against a global that races or a
#if DEBUGcounter that leaves release untested. Say the word if you'd rather have it behind a compile flag and I'll move it.Two things worth being explicit about, since both are limitations rather than features:
claimedProbesis zero for a paragraph of links. Nothing is claimed before the link pass, and there are no*or\characters to ask about, so the query never fires.containmentTestscarries the signal there andclaimedProbescarries it for code spans, which is why the assertion sums them — and why it also asserts the count is non-zero, so it can't pass by measuring nothing.ClaimedIndexandbuildTreeentirely. It holds the existing structures to linear; it doesn't prove nothing quadratic exists anywhere. The timed version had the opposite trade — measured everything, meant nothing off this machine.corpusFingerprintis untouched and stillb74649ffbbbe237a, which is also the check that the instrumentation changed no parse. 324 tests green.Thanks for rebasing #140 rather than bouncing it — and for catching this on CI instead of letting it rot as a flake.