Skip to content

bench: typecheck_scaling — schema-size × query-size scaling of the type checker - #93

Draft
Felipe705x wants to merge 1 commit into
mainfrom
bench/typecheck-scaling
Draft

bench: typecheck_scaling — schema-size × query-size scaling of the type checker#93
Felipe705x wants to merge 1 commit into
mainfrom
bench/typecheck-scaling

Conversation

@Felipe705x

@Felipe705x Felipe705x commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Do not merge — pinned-baseline benchmark. This branch deliberately sits on the v0.2.0 release (6cba43e) so the numbers track a fixed released baseline of the checker; the bin uses the crate's lib name at that point (gqlrust), so merging into main (lib renamed to frogql) would break the build. Run the bench from this branch; cite it as bench/typecheck-scaling @ HEAD.

Runtime-independent microbenchmark of Typechecker::check_query: cost as a function of schema size (a homogeneous synthetic schema family — the controlled variable, so a fixed real schema can't serve) and query size (chain length k). No runtime, storage, or data involved.

  • Cases: from internal_bench.rsv_chain_knows, e_bad_edge_deep, e_label_only — byte-identical after renaming labels into the synthetic universe (Person→L0, knows→e0, Wagumi→NoSuchLabel; projected property names exist verbatim). The chain is parameterized in k by inserting intermediate nodes only; k=1 is the original query.
  • Schema family: max(S/3,2) node types (two properties each) + undirected edge types, 1:2 ratio matching the measured LDBC inferred schema — 11 node + 25 edge entries = 36, identical for SF0.1 and SF0.3 (verified on both datasets; the inferred schema is scale-invariant, which is precisely why varying schema size requires a synthetic family). Suggested ladder: 16–1024 (powers of two) plus 36 as the LDBC-size anchor.
  • Protocol: parse/elaborate outside the timed loop; median of ≤4000 iterations under a 200 ms/case budget; verdicts asserted every run. Aggregate ≥3 interleaved sweeps by medians (session-to-session wall-clock varies on consumer hardware).

🤖 Generated with Claude Code

…pe checker

Runtime-independent microbenchmark of Typechecker::check_query over a
homogeneous synthetic schema family (schema size is the controlled
variable; 1:2 node:edge ratio matching the measured LDBC SF0.1 inferred
schema, 11+25=36). Cases come from internal_bench.rs — v_chain_knows,
e_bad_edge_deep, e_label_only, byte-identical after renaming labels
into the synthetic universe — with the chain parameterized in k by
inserting intermediate nodes. Median timing under an adaptive per-case
budget; verdicts asserted every run. Based on the v0.2.0 release so the
numbers track a fixed released baseline of the checker.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant