Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .dev/memory-bank/lessons-learned.md
Original file line number Diff line number Diff line change
Expand Up @@ -1132,3 +1132,106 @@ local, and false.
`pharn/pharn-contracts/eval-format.md:52`, `pharn/pharn-contracts/finding-shape.md`, and the seven
grillers
- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved).

## L34 — "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission

type: floor · concepts: [vacuous-truth, empty-set, fail-closed, eval-design]

**Lesson.** A checker assembled from per-item assertions passes **vacuously** over an empty domain, and
a vacuous pass is indistinguishable from a real one at the verdict. `pharn/floor/check-structural.mjs`
is the instance: its `field_equals`, `file_resolves` and `needle_absent_from_enum_gated` kinds all
iterate `findings`, so an eval that wrote per-finding assertions but omitted `finding_count` certified a
skill that emitted **nothing** — every assertion true, zero findings examined, GREEN. The eval a
Capability ships as its specification (P1) was satisfied by the total absence of the behaviour it
specifies. Remedy, now at `pharn/floor/check-structural.mjs:208-224`: guard the zero-item case
explicitly — membership over the assertion kinds present, plus an integer length test, no judgment (P5).
The legitimate "I expect nothing" intent MUST stay expressible, and that is exactly what distinguishes
the two cases: an eval that genuinely expects an empty result says so with `finding_count == 0` and
stays GREEN. Silence and asserted-silence are different claims; only the second is a specification.

**The shape is widely RECOGNISED and still nothing ranges over it — recognition is not coverage.** Four
other production sites defend the identical shape, each written by hand, sharing no helper (all five
import Node stdlib only; zero cross-imports between them): `pharn/floor/check-verify.mjs:99` and
`pharn/floor/check-regress.mjs:376` carry the byte-identical string `is empty — no gates captured`,
written separately; `.dev/floor/check-variance.mjs:171` dies INCONCLUSIVE on `valid.length === 0`;
`.dev/floor/check-contributing-gates.mjs:108` REDs `EMPTY_CHAIN` because "a set with no members is not a
chain". Six further test files carry hand-written comments defending their **own** tests against passing
vacuously — `check-ship-briefing`, `check-provenance`, `check-contributing-gates`, `command-hygiene`,
`entry-point-guard`, `lessons-index-core`. So the failure mode is understood repo-wide, re-derived
locally at each site, and enforced nowhere as a class — and the one place the hand-rolled guard was
missing is `check-structural`, the checker that executes the eval contract itself.

**Why it matters.** A vacuous pass is the **fail-OPEN** direction of a checker whose entire purpose is to
fail closed, and it is invisible in the direction people look: a suppressed emission and a clean
codebase produce the same GREEN. That makes it the highest-value silent regression an enforcement
pipeline can carry — a lens that stops emitting is certified by its own eval, and P1's "evals are the
spec" degrades to "evals are the spec, unless the output is empty." [[L20]] says a discipline-only
remedy will recur and the second occurrence earns a floor check; here the count reached **five**
hand-written guards before the sixth site slipped, which is that trigger long overdue rather than newly
fired. [[L29]] says a remedy quantified over a set owes the **enumeration** as its deliverable — the
five sites above are that enumeration, materialized in one place for the first time, and the honest
residual is that nothing yet iterates it. Distinct from [[L23]], which also turned on an empty array
hiding a defect: there the emptiness was the **happy path** concealing a self-referential gate conflict
between a stage and its own artifact; here the emptiness is the **domain of quantification** itself, and
the defect is that a universally-quantified claim over it is true for free. Same symptom, different
mechanism — L23's remedy is to exempt an artifact from a gate, this one's is to assert the domain is
non-empty.

**Provenance.**

- feature: `floor-hardening`
- commit: `02c2d17e54f49b4b16395b98799726e5e77c5a01`
- source: `.dev/features/floor-hardening/PLAN.md` (its L7 rows in `## Files` + the `## Guarantee audit`
line "L7 'per-finding assertions cannot pass vacuously' → **floor: enum-regex**") + the CHANGELOG
2.7.13 entry, with the five-site recurrence and the six recognition sites re-derived live at HEAD
`a0cfd0d`
- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved).

## L35 — When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync

type: process · concepts: [redundant-identity, remedy-design, version-discipline, sync-cost]

**Lesson.** When the same fact is stored in two places, the first question is whether the second copy
should exist at all — **not** how to keep them agreeing. A sync check is itself a third artifact with its
own wiring, its own invoker and its own failure mode, and it makes the redundancy **permanent by making
it maintained**. Three identities carried the product version: `package.json`'s `version`,
`SKILLS_VERSION`, and the README shields badge. The first read `1.0.0` as a "foundation tag" and sat
stale through the entire 2.x line while every gate stayed green, because nothing bound it and nothing
could have bound it without becoming the fourth thing to maintain. The remedy chosen in `#164` was to
**neutralise the identity, not bind it**: `package.json:4` is now `"version": "0.0.0"` and
`package.json:3` is a `_version_comment` stating the inertness **in the file itself**, so the fact is
self-documenting rather than documented somewhere a reader must find.
`.dev/floor/check-version-badge.mjs` then deliberately does **not** read `package.json` — it names the
field only in its own honest-bounds header (`:26-40`) — and pins the two identities that genuinely
remain: `SKILLS_VERSION` 2.7.15 == the README badge 2.7.15.

**The precise remedy is "retire", not "delete" — and that distinction is the transferable part.** npm
requires a `version` key, so the redundant copy could not be removed; it was drained of authority
instead, which is the general move when a mandatory slot holds a fact that belongs elsewhere. Give the
slot a value that cannot be mistaken for the truth (`0.0.0`, never a plausible-looking number), and put
the reason **at the slot** rather than in a doc. The rejected alternative — pinning `package.json` to
`SKILLS_VERSION` — is recorded, not silently dropped, in `.dev/features/apparatus-batch/PLAN.md` under
"Open questions (HALT)": option (b) "creates a third identity to keep in sync, which is the defect being
fixed."

**Why it matters.** [[L20]] says a discipline-only invariant has earned a floor check on its second
occurrence, and read alone it will send you to build a checker every time. This is the qualifier: **a
sync check is the right remedy only once you have established the second copy must exist.** Reach for it
first and you have converted a deletable redundancy into a maintained one, plus a checker whose own
wiring must now be pinned by a test — the cost L20 never charges. Distinct from [[L31]], and exactly
opposite in prescription: L31 governs copies that must **both** exist (the deliberate `check-provenance`
and `lessons-index-core` product/dev pairs, which cannot be merged because a user's install ships
`pharn/floor/` without `.dev/`) and asks what **ranges over** them; this governs a copy that should
**not** exist and says draining beats binding. Both readings are live in this repo, and the question
that separates them is the same one in both directions: _must the second copy exist?_ — asked before
choosing a remedy, never after.

**Provenance.**

- feature: `apparatus-batch`
- commit: `a0916b8b7ce84f51b97b3dbe053eda7d96d9e804`
- source: `.dev/features/apparatus-batch/PLAN.md` "Open questions (HALT)" (the L11 option-(a)-over-(b)
record) + the CHANGELOG entry for `#164`, with `package.json:3-4`,
`.dev/floor/check-version-badge.mjs:26-40` and the live `SKILLS_VERSION`/badge agreement re-derived at
HEAD `a0cfd0d`
- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved).
4 changes: 3 additions & 1 deletion docs/lessons-index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ lessons to fetch; canon stays the source of truth and the floor's verification t
applied, without fetching its full `## L<n>` entry from canon, is the P0 disease. "The index was
consulted" never means "the relevant lessons were read".

33 lessons · 33 tagged · 0 malformed · 0 untagged · ~21257 tokens total
35 lessons · 35 tagged · 0 malformed · 0 untagged · ~23261 tokens total

Columns: `id | type | concepts | title | promoted | ~tokens`. Every canon entry carries a tag line, so
BOTH absence markers are unexpected: `-` = no tag line, i.e. an entry that reached canon without
Expand Down Expand Up @@ -53,4 +53,6 @@ L30 | process | style-gates,command-prescription,lesson-recurrence,floor-escala
L31 | process | lesson-recurrence,dev-product-boundary,branch-coverage,floor-escalation | A deliberate copy-pair creates an obligation set nothing ranges over — the second copy is where the obligation is dropped | 2026-08-20 | ~668
L32 | process | verification-fidelity,doc-drift,command-prescription,false-green | A verification method that consults a mutable ALIAS proves reachability, not identity | 2026-08-23 | ~781
L33 | process | doc-drift,shipped-surface,false-green,enumeration | A "not yet built" claim expires the moment the work lands — nothing reads shipped prose, and the repair pass misses the variant spellings | 2026-08-23 | ~1083
L34 | floor | vacuous-truth,empty-set,fail-closed,eval-design | "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission | 2026-08-23 | ~1083
L35 | process | redundant-identity,remedy-design,version-discipline,sync-cost | When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync | 2026-08-23 | ~921
```