From d0198903e754ad2606f346c22e6210a0749ebfe5 Mon Sep 17 00:00:00 2001
From: agentforce314 <273884145+agentforce314@users.noreply.github.com>
Date: Sun, 16 Aug 2026 08:27:54 -0700
Subject: [PATCH] =?UTF-8?q?docs:=20full-suite=20TB=202.1=20result=20?=
 =?UTF-8?q?=E2=80=94=20nano=2064/89=20($1.31)=20vs=20pi=2063/89=20($2.01)?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The pi-matched full rerun (tb21-nano-flash-max-2: vision+websearch,
deepseek-v4-flash max thinking, k=1) lands nano at 64/89 (71.9%) for
$1.31 against pi's 63/89 (70.8%) for $2.01 and the pre-fix nano
baseline's 60/89. Six of the eleven gained tasks are directly
attributable to the shipped fixes (vision, websearch, care guideline,
timeout-race mechanics); seven v1 passes flipped on k=1 build-time
variance, so the honest read is parity-to-slight-edge on score with a
durable ~35% cost advantage. Runbook carries the full table and the
variance caveat; README banner and docs/nano.md updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 README.md                    |  2 +-
 docs/nano.md                 |  5 ++++-
 eval/harbor/RUN_NANO_TB21.md | 19 +++++++++++++++++++
 3 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/README.md b/README.md
index 690cec47..be8b0c42 100644
--- a/README.md
+++ b/README.md
@@ -79,7 +79,7 @@ ClawCodex keeps your request prefix **byte-stable**, so DeepSeek's prompt cache
 
 # The pi-style minimal harness, built in
 
-### Terminal-Bench 2.1 A/B (`fix-git` + `pypi-server`), same model, same wheel, one flag apart: **2/2 solved in both modes** — nano at **$0.0042 vs $0.0186 total (4.4× cheaper)**, up to 5.2× less input, faster on both.
+### Full Terminal-Bench 2.1, same model, head-to-head with the [pi harness](https://pi.dev) (its own TB setup, vision+websearch matched): **nano 64/89 (71.9%) at $1.31 vs pi 63/89 (70.8%) at $2.01** — equal-or-better score, **35% cheaper**, k=1.
 
 Six tools, a **~2K-token** fixed payload (vs ~16K default), zero per-turn injections, /eco on.
 `clawcodex --nano -p "<task>"` ports the pi harness's edit ladder (multi-edit + fuzzy match),
diff --git a/docs/nano.md b/docs/nano.md
index bc05a7ad..7a7f2b19 100644
--- a/docs/nano.md
+++ b/docs/nano.md
@@ -36,7 +36,10 @@ Measured on terminal-bench 2.1 (`fix-git` + `pypi-server`,
 deepseek-v4-flash, identical wheel, k=1): **both tasks solved in both
 modes (reward 1.0)** — nano at **$0.0042 total vs $0.0186 (4.4×
 cheaper)**, e.g. fix-git: 54.8K vs 282.6K input tokens, $0.00165 vs
-$0.01015, 75 s vs 128 s. Full tables: `eval/harbor/RUN_NANO_TB21.md`.
+$0.01015, 75 s vs 128 s. On the full 89-task suite head-to-head with
+the pi harness (both with vision+websearch, deepseek-v4-flash at max
+thinking, k=1): **nano 64/89 ($1.31) vs pi 63/89 ($2.01)**. Full
+tables: `eval/harbor/RUN_NANO_TB21.md`.
 
 ## What nano does differently
 
diff --git a/eval/harbor/RUN_NANO_TB21.md b/eval/harbor/RUN_NANO_TB21.md
index 27f0b5a0..9c2d96e6 100644
--- a/eval/harbor/RUN_NANO_TB21.md
+++ b/eval/harbor/RUN_NANO_TB21.md
@@ -63,6 +63,25 @@ Aggregate: **2/2 pass in both modes; nano $0.0042 vs default $0.0186 —
 4.4× cheaper** at equal quality. Nano's fix-git trajectory was 10 focused
 Bash calls, zero tool errors.
 
+## Full-suite nano vs pi (89 tasks, deepseek-v4-flash, max thinking, k=1)
+
+Same wheel lineage, pi ran its TB extension (vision+websearch); the nano
+arm matched it (`NANO_VISION=openai:gpt-5.6-luna NANO_WEBSEARCH=1`):
+
+| | nano `tb21-nano-flash-max-2` | pi `tb21-pi-flash-max-2` |
+|---|---|---|
+| solved | **64/89 (71.9%)** | 63/89 (70.8%) |
+| cost | **$1.31** | $2.01 |
+| runtime | 5h38m | 5h54m |
+
+The pre-fix nano baseline (`tb21-nano-flash-max-1`, no vision/websearch,
+before the care/verification guidelines and staleness soft-refresh) was
+60/89 at $1.02. Six of the eleven tasks v2 gained are directly
+attributable to shipped fixes (vision ×1, websearch ×1, care ×1,
+timeout-race mechanics ×3); seven v1 passes flipped red on k=1
+variance (build/training-time jitter). At this k, treat 64-vs-63 as
+parity-to-slight-edge on score with a durable ~35% cost advantage.
+
 Nano sends six tools and a ~2K-token fixed payload (vs ~16K default), no
 per-turn injections, /eco on. A trivial live A/B outside Harbor
 (deepseek-v4-pro, write-and-verify, both solved in 4 turns) showed the