feat(benchmarks): add GSM8K, HumanEval, and BBH end-to-end (#99)

[minion1227]

Fixes #99

Problem

The leaderboard schema, mock data, and web types already carry gsm8k / humaneval / bbh metric fields, but nothing in the training/eval core ever produces those scores — they are permanent nulls. This wires all three end-to-end so a miner can actually train/eval against them and see real numbers.

Approach — reuse existing grading paths, no core-checker changes

Benchmark	Dataset	Grading
GSM8K	openai/gsm8k (main)	Loader extracts the canonical #### <answer> number → graded on the existing MATH_BENCHMARKS path
HumanEval	openai/openai_humaneval	Loader shapes each row into the existing run_pass_at_1 assert contract ("<test>\ncheck(<entry_point>)") → no sandbox changes
BBH (27 subtasks)	lukaemon/bbh	New bbh checker: MCQ-letter path for "(X)" targets, else normalized exact-match → OOD suite (relevant to R5)

Changes

• dataset.py: gsm8k/humaneval/bbh added to SUPPORTED_BENCHMARKS + _HF_LOADERS, with HF loaders (all 27 BBH subtask configs) + offline toy fallbacks, following the existing math500/mmlu/livecodebench pattern (same lazy-import guard + seeded-shuffle determinism).
• reward.py: register gsm8k in MATH_BENCHMARKS, humaneval in CODE_BENCHMARKS, add BBH_BENCHMARKS + _check_bbh. BBH multiple-choice subtasks go up to (G) (5–7 options), so a permissive A–Z _bbh_extract_letter is used for BBH (extract_choice_letter caps at A–D) — otherwise those subtasks silently under-count.
• benchmarks/: gsm8k.py, humaneval.py, bbh.py facades mirroring livecodebench.py.
• configs/benchmarks.yaml: gsm8k/humaneval in-distribution (train/eval), bbh in ood.
• tests: tests/test_benchmarks_gsm8k_humaneval_bbh.py — offline loader (toy determinism + monkeypatched HF-row parsing) and grader coverage for all three (incl. the (E)/(G) BBH cases and HumanEval correct/wrong).
• docs: JOURNAL.md entry (AGENTS.md §6 protocol) + SPEC.md §6.6 [OUR CHOICE] note.

Scope / safety

Additive only — no changes to the coordinator/CMA-ES loop, the sandbox/executor, or the frontend. scripts/results_table.py and the validator leaderboard need no changes (both read benchmark names generically), so real gsm8k/humaneval/bbh scores flow through once an eval reports them.

Full root suite: 204 passed; ruff clean on the changed lines. All loaders verified offline via toy fallbacks; graders verified with correct/wrong cases.

[tmimmanuel]

1. HumanEval is added to both train and eval, but both entries use the same only available test split. That makes the eval score non-disjoint from the optimization set and invalidates the held-out comparison. If the intent is a real eval benchmark, HumanEval should not be in train as wired here. See configs/benchmarks.yaml.

2. BBH has a scorer, but the shared has_answer() helper does not know about bbh. That helper is used by _committed_answer() and the training format_bonus path, so BBH trajectories will never be recognized as having a valid parseable answer even when they do. Add a BBH_BENCHMARKS branch there, aligned with _bbh_extract_letter / _check_bbh. See src/trinity/orchestration/reward.py.

[minion1227]

Thanks — both addressed in f758b22:

1. HumanEval eval-only. It ships a single (test) split, so training + evaluating on the same split was non-disjoint. Removed it from train; it is now eval-only. GSM8K keeps its proper disjoint train/test splits.

2. has_answer() now handles bbh. Added a BBH_BENCHMARKS branch aligned with _bbh_extract_letter / _check_bbh (multiple-choice letter or non-empty free-form target), so _committed_answer() and the training format_bonus path recognize BBH trajectories.

Added tests for both (has_answer bbh MCQ/free-form/empty; a config check that HumanEval is eval-only while GSM8K stays in both). Also rebased onto latest main (resolved the BFCL benchmark overlap, keeping both). Full suite: 244 passed.

[minion1227]

CI fix pushed in 41f361c — test-router is green now.

The failure was environment-specific: CI has datasets + network, so load_tasks("humaneval") loaded the real HumanEval set and [0] was the specialFilter task, not the add toy — my grading assertion then failed. Made the toy tests deterministic (grade against _toy_tasks(...) directly, and stub _try_load_hf to force the offline path), so they no longer depend on dataset/network availability. Full suite: 244 passed.