feat(science): Phase B — corpus v1 (250 items), run-002 ready, PIGA annotation kit

[pixelstrade-dev]

Phase B kickoff — everything buildable in-repo, built

Phase A ended stating that only new empirical data moves the validity score. This PR delivers every Phase B piece that doesn't require the owner's credentials or human annotators — and leaves those two gates one step from ready.

B1 — Corpus v1: 250 authored items, preregistered design, CI-enforced

• Design preregistered in research/corpus/v1/README.md, sized by the A6 power analysis: 6 strata (4 positive × 45, 2 adversarial × 35), 28 % adversarial, bounds fixed at authoring time (meaningful violation ≥ 5 pts), no-citation rule for positives, invented DOIs locked to the reserved 10.5555/ prefix.
• 36 plain-correct items (9 per positive stratum) instrument the fluency confound: short, correct, unadorned responses at minComposite 45 — if they systematically undershoot eloquent-correct, the instrument rewards style, and that gets published.
• 70 adversarial controls across 14 declared control types (7 fluent-failure, 7 epistemic-failure), each with the designed flaw stated exactly in its rationale.
• validate-corpus.mjs (zero-dep, wired into CI next to the constructs validator) enforces strata sizes, the adversarial fraction, bounds policy, type counts, global id and (question, response) uniqueness, and the citation/DOI rules.
• This is an instrument-validation corpus: authored responses with known properties measure the judges, not any subject model — scoped in the README and carried into every claim.

B3 — Run 002: ready to run

• Experiment runner gained bounded item-level concurrency (config field, default = the old sequential behavior). Call set and all aggregates are identical at any setting — proven by tests including a max-in-flight assertion, a deterministic flaky-adapter equivalence run, and an order-independent mock (the old mock was call-order-dependent; keyed per prompt now, with the uniqueness precondition enforced by the corpus validator).
• Workflow timeout raised to the hosted cap; full run ≈ 1–2 h at concurrency 4.
• research/experiments/run-002/: config + owner runbook. Validated end-to-end by a local mock run: 3 750/3 750 calls, 750 cells, 3 judges, zero pipeline failures. Owner needs only: the two OPENWEIGHT_* Secrets, the exact endpoint model id, one workflow click.

B2 — PIGA human annotation kit (needs ≥ 3 humans)

• Protocol with preregistered analysis: intent-space audit, expectation labels, and behavior classification of 12 pre-written responses; supermajority threshold and tie rules fixed in advance; machine-readable sheet with the judge's decision procedure extracted verbatim programmatically (no drift); Task-3 answer key segregated under facilitator/ with an explicit withholding rule.
• PIGA protocol bumped to 0.3.0-alpha: review showed the stated/silent class boundary was contestable ("describing what the delivered work does" vs "stating an assumption") — tightened in the judge prompt before any run exists.

Review process (4 adversarial reviewers, all findings applied)

• S1+S4 accuracy (expert pass on 90 items): 1 BLOCKER (Bloom-filter bits-per-element figure), 2 MAJORs (CORS preflight trigger; jet-lag light advice wrong for ≥5-zone eastward jumps) + minors — all fixed.
• S2+S3 accuracy: 1 MAJOR (stellar life-cycle order) + minors (violet-absorption clause, near-citation phrasing) — all fixed.
• S5+S6 adversarial: 70/70 controls individually valid, 0 blockers; 4 cross-strata duplicate question pairs resolved by retargeting (each control's declared flaw preserved); misattribution safety rail verified (all false attributees fictional, no real person miscredited).
• Infra: counter-race, sparse-results, and poisoned-queue attacks all probed empirically (aggregates byte-identical seq vs concurrent, incl. failure paths); annotation-kit arithmetic and threshold gaps fixed.

Validation

Corpus validator: 250 items, 28.0 %, 0 warnings. Registry 8/8. apps/web 403/403, packages/core 207/207, cold tsc clean, sync-core no drift.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y7wiMwFa2D4P8zD9RhCgsv

---

Generated by Claude Code