feat(science): corpus sizing by seeded power analysis — Phase A6 - #62
Merged
pixelstrade-dev merged 2 commits intoAug 23, 2026
Merged
Conversation
Replaces the roadmap's round-number corpus target with numbers DERIVED from Run 001's committed parameters, reproducible bit-for-bit (fixed seed 20260823, 20k sims): - lib/statistics/simulation.ts: mulberry32 + Box-Muller + H1 bound-detection power + bootstrap CI half-widths. Anchored against the analytic answer (sigma 5, delta 5, n 5 -> Phi(sqrt(5)) = 0.987, matched by simulation) with monotonicity and determinism tests. - cli/power-analysis.ts reads summary.json (median cell SD 2.05, max 7.60, the 11 observed |judge diffs|) and prints tables A/B/C. - docs/power-analysis-a6.md records the real outputs and the derived recommendation: corpus v1 ~200-300 items (25-30% adversarial — the stratum that dominates precision cost: alpha still +/-0.15 at 100 items there), n=5 samples/cell suffices for >=5-point violations even at adversarial-worst SD, ~100 all-judge items pin the inter-judge difference to +/-1.7 pts. The 2000-5000 full-benchmark scale is re-sized after Run 002 (3-rater sizing from 2-rater data would require inventing a judge — refused). - Assumptions stated in code, CLI output AND doc: normality for table A; representativeness of the 11 items for B/C (the binding limitation — which is the argument for the corpus); 2-rater only. app 374/374, core 187/187, cold tsc 0 both sides, sync no drift Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7wiMwFa2D4P8zD9RhCgsv
…veats - power-analysis CLI: even-count median is now interpolated (1.92, was upper-middle 2.05); simulations run at the unrounded observed sigmas and rows are labeled with sigma.toFixed(2) — the doc table now states exactly what was simulated - bit-for-bit reproducibility claim scoped to Node (Box-Muller uses Math.log/sin/cos, not bit-specified by IEEE 754) - doc: 'no parameter is invented' replaces the false 'nothing is assumed that the run did not measure' (normality IS an assumption, and is listed); table A includes all rows incl. saturated 1.000 ones; table C half-widths labeled optimistic lower bounds (bootstrap from 5-6 observed values); controls median alpha ~0.159 cross-referenced to the A5 disaggregated analysis - sync-core: statistics/simulation mirrored into packages/core Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y7wiMwFa2D4P8zD9RhCgsv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase A6 of the validity program — corpus sizing derived, not decreed
The growth plan inherited a "2 000–5 000 items" corpus target chosen by round number. This PR replaces it with numbers derived from committed Run 001 parameters, via seeded simulations that anyone can reproduce.
New module:
lib/statistics/simulation.ts(mirrored to@caims/core)mulberry32(seed)— deterministic PRNG;makeNormalSampler— Box–Muller with spare caching.boundDetectionPower({sigma, delta, n, sims, seed})— P(mean of n samples flags a true bound violation of Δ). Anchored against the analytic answer Φ(√5) ≈ 0.987.bootstrapHalfWidth({observed, itemCount, stat, sims, seed})— 95 % CI half-width of a statistic under item resampling; returnsnull(never a fake number) when the statistic is mostly undefined.New CLI + report:
cli/power-analysis.ts,docs/power-analysis-a6.mdAll parameters from
run-001/results/summary.json(seed 20260823, 20 000 sims):Orchestrator review — applied fixes
Adversarial review returned APPROVED WITH REQUIRED FIXES; all applied:
toFixed(2)— the table now states exactly what was simulated.Math.log/sin/cosare not bit-specified by IEEE 754).Validation
tscclean on core and on all touched files (the only web errors are the pre-existing Prisma-generated-client absence in this container — CI regenerates it).sync-coredrift check clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01Y7wiMwFa2D4P8zD9RhCgsv
Generated by Claude Code