feat: make a run measurable, affordable, and honest about failure - #6

Merged

wimaan3 merged 23 commits into

mainfrom

feat/local-scoring

Aug 27, 2026

Commits on Aug 26, 2026

feat: grade a run locally against the GAIA reference answers
wimaan3
committed
bb18879
fix: never memoise a failed GAIA dataset listing
wimaan3
committed
9331e7e
feat: record how many delegation rounds a task actually took
wimaan3
committed
29ef028
fix: trim the middle of tool output, not the end
wimaan3
committed
a1b0486
perf: memoise read-only tool results within a task
wimaan3
committed
644421c
feat: cap what a run may spend, and what the router may generate
wimaan3
committed
b288ce6
feat: label each run, and make reasoning effort tunable per role
wimaan3
committed
db70036
refactor: structure the prompts with XML tags, and stop truncating documents
wimaan3
committed
e50344f
feat: ground the finalizer and router prompts in real reference answers
wimaan3
committed
5525eb3
feat: run specialists at medium reasoning effort
wimaan3
committed
1a441c0
fix: four regressions the first five-task run exposed
wimaan3
committed
4cd86c7

Commits on Aug 27, 2026