Skip to content

Count the span-density cost instead of timing it - #146

Open
wildthink wants to merge 1 commit into
nodes-app:mainfrom
wildthink:perf/deterministic-span-cost-test
Open

Count the span-density cost instead of timing it#146
wildthink wants to merge 1 commit into
nodes-app:mainfrom
wildthink:perf/deterministic-span-cost-test

Conversation

@wildthink

Copy link
Copy Markdown
Contributor

Follow-up to #140 and to 350b2d3, and an answer to "happy to hear an argument for a different number": I don't think there is a better number, because the thing being measured is wrong. This replaces the timing with a count.

Sorry for the red CI — the bound was mine and the reasoning behind it was wrong in a way your commit message pins exactly.

Why no threshold worked

I justified those bounds with "minimum of several runs; noise only ever adds time, so the floor is stable." That holds for an absolute measurement on an idle machine. It does not hold for the ratio I actually asserted: under sustained contention there is no quiet run to find a floor in, and the smaller measurement inflates proportionally more, so the ratio drifts up with load rather than staying put. 5.3x here, 10.9x on the runner, same parser. Your point that the post-rewrite CI reading sits above the pre-rewrite floor is the end of the argument — any bound green on CI would have passed the parser the test exists to catch.

Counting instead

InlineParser.parse can now report an InlineParseCost: claimed-range probes and containment tests. Both are pure functions of the input, so they read the same on your laptop, mine, and a loaded runner — and the separation is decisive instead of marginal:

40 spans 240 spans growth
current parser 508 3248 6.0x
pre-rewrite pairwise containment 1840 58320 33.9x

Exactly 6.0x for every construct — code, links, emphasis, highlight, mixed, and link labels with nested code spans. I got the 33.9x by actually restoring the old pairwise containment and re-running, not by extrapolating. The bound stays at 8, which now sits inside a gap that doesn't move.

These run on CI. The timed assertions stay exactly as you left them — opt-in, useful for absolute numbers — and I corrected the msPerParse comment, which still asserted the floor reasoning I'd used to justify the ratio.

On the instrumentation

Counters are plain Ints bumped beside comparisons the loops already perform, threaded through rather than parked in a global. The global would have been a smaller diff and wrong: swift-testing runs suites in parallel, so every other suite that parses markdown would land in the counter mid-measurement. ClaimedIndex owns its own probe count and the scans take it inout so the caller reads it back; buildTree takes the cost inout.

That is a real cost in the hot path — two integer increments — which I judged worth it against a global that races or a #if DEBUG counter that leaves release untested. Say the word if you'd rather have it behind a compile flag and I'll move it.

Two things worth being explicit about, since both are limitations rather than features:

  • claimedProbes is zero for a paragraph of links. Nothing is claimed before the link pass, and there are no * or \ characters to ask about, so the query never fires. containmentTests carries the signal there and claimedProbes carries it for code spans, which is why the assertion sums them — and why it also asserts the count is non-zero, so it can't pass by measuring nothing.
  • It won't catch a brand-new scan that bypasses ClaimedIndex and buildTree entirely. It holds the existing structures to linear; it doesn't prove nothing quadratic exists anywhere. The timed version had the opposite trade — measured everything, meant nothing off this machine.

corpusFingerprint is untouched and still b74649ffbbbe237a, which is also the check that the instrumentation changed no parse. 324 tests green.

Thanks for rebasing #140 rather than bouncing it — and for catching this on CI instead of letting it rot as a flake.

The wall-clock ratio these assertions used was not portable, and gating CI
on it turned main red: the same parser reads 5.3x on an idle laptop and
10.9x on a contended runner, which is above the 11.3x pre-rewrite floor,
so no threshold separates the two parsers. 350b2d3 made them opt-in, which
left the rewrite with no CI-visible cost guard.

InlineParser.parse can now report an InlineParseCost — claimed-range probes
and containment tests — and the assertions use that. Both quantities are
pure functions of the input, so they read identically everywhere, and the
separation is decisive rather than marginal: 6.0x for 6x the spans against
33.9x with the pre-rewrite pairwise containment restored, verified by
reintroducing it. The bound stays at 8, now inside a deterministic gap.

The counters are plain Ints beside comparisons the loops already do,
threaded through rather than kept in a global, because swift-testing runs
suites in parallel and a global would be raced by every other suite that
parses. ClaimedIndex owns its own probe count; the scans take it inout so
the caller can read it back.

The timed assertions stay, still opt-in, for absolute numbers. What the
counted form does not catch is a new scan bypassing ClaimedIndex and
buildTree entirely — it holds the existing structures to linear rather
than proving nothing quadratic exists anywhere.

Corpus fingerprint unchanged, so the instrumentation changed no parse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wildthink
wildthink force-pushed the perf/deterministic-span-cost-test branch from ae54c04 to 36c98d2 Compare August 10, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants