Establish the current Phase 1 human benchmark gate

[NeoLorenzo]

Audit Priority: P1
Audit Type: VALIDATE

Problem / Opportunity

PolyVision now explicitly defers current-interface multi-seed Phase 1 training until the model, on average, outperforms the canonical human benchmark, but the benchmark needed to apply that gate does not yet exist in a usable form. Only 1 of 17 canonical human first attempts is complete, and the documented full-pool PPO comparison on the human-benchmark split is still the superseded v3 Seed3 model rather than the active v5 PARITY002 reference.

Why This Matters

The README and human-benchmark documentation now make human-relative performance a decision gate for the next expensive research stage. Without a completed human sample and a same-protocol evaluation of the active model, that gate remains undefined and cannot be applied reproducibly. Continuing optimization without resolving the benchmark also increases the chance that the human pool becomes another informal development target rather than a clearly specified comparison.

Evidence

• README.md now states that current-interface multi-seed Phase 1 training is deferred until the canonical human-benchmark gate is established and the relevant candidate passes it.
• docs/human-benchmark.md repeats that sequencing rule, reports only 1/17 completed canonical human first attempts, and identifies this issue as owning completion of the benchmark and definition of the machine-readable gate.
• The completed human attempt reached Turn-10 SPT 27 on map_004393.csv.
• The same document's full 17-map PPO baseline is for the superseded Phase 1 v3 Seed3 16M model (mean 16.71 SPT), while README.md and docs/evaluation.md identify v5 PARITY002 Seed3 16M as the active frozen reference.
• tools/human_benchmark.py already provides immutable first-attempt recording and a persistent result registry, so the missing work is benchmark completion and current-model comparison rather than new benchmark infrastructure.
• tools/evaluate_phase1.py supports --pool human_benchmark, but its canonical classification is restricted to full validation/test runs. A complete 17-map human-benchmark evaluation is therefore currently labeled partial-smoke-noncanonical, which is incompatible with using that artifact as the canonical model side of this gate without an explicit protocol distinction.

Proposed Outcome

Complete the canonical first-attempt human benchmark and evaluate the active Phase 1 reference policy on exactly the same 17 maps under the same interface/protocol. Define the human-relative gate in machine-readable terms so future runs can determine whether the threshold for multi-seed training has actually been met without reinterpretation. Ensure the full 17-map model comparison is recorded as an explicitly recognized human-benchmark protocol artifact rather than being conflated with partial/smoke evaluation output.

Definition of Done

• All 17 human-benchmark maps have one completed immutable canonical first attempt.
• The active frozen Phase 1 reference checkpoint is evaluated on all 17 maps under the benchmark's policy-visible interface and recorded episode configuration.
• A full 17-map model run used for the gate is identified in machine-readable provenance as the canonical human-benchmark comparison protocol (or an equivalently explicit designation), not partial-smoke-noncanonical.
• A paired per-map human-vs-model result artifact is produced with mean/median SPT, paired deltas, win/tie/loss counts, and uncertainty appropriate for the 17-map sample.
• The repository records an explicit machine-readable rule for what "model on average outperforms the human benchmark" means.
• Documentation reports the active model comparison rather than relying on the superseded v3 baseline, while preserving historical comparisons.
• The gate can be evaluated from saved artifacts without replaying or modifying canonical human attempts.