diff --git a/.dev/memory-bank/lessons-learned.md b/.dev/memory-bank/lessons-learned.md index c3b6638..b2f92f6 100644 --- a/.dev/memory-bank/lessons-learned.md +++ b/.dev/memory-bank/lessons-learned.md @@ -1132,3 +1132,106 @@ local, and false. `pharn/pharn-contracts/eval-format.md:52`, `pharn/pharn-contracts/finding-shape.md`, and the seven grillers - promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). + +## L34 — "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission + +type: floor · concepts: [vacuous-truth, empty-set, fail-closed, eval-design] + +**Lesson.** A checker assembled from per-item assertions passes **vacuously** over an empty domain, and +a vacuous pass is indistinguishable from a real one at the verdict. `pharn/floor/check-structural.mjs` +is the instance: its `field_equals`, `file_resolves` and `needle_absent_from_enum_gated` kinds all +iterate `findings`, so an eval that wrote per-finding assertions but omitted `finding_count` certified a +skill that emitted **nothing** — every assertion true, zero findings examined, GREEN. The eval a +Capability ships as its specification (P1) was satisfied by the total absence of the behaviour it +specifies. Remedy, now at `pharn/floor/check-structural.mjs:208-224`: guard the zero-item case +explicitly — membership over the assertion kinds present, plus an integer length test, no judgment (P5). +The legitimate "I expect nothing" intent MUST stay expressible, and that is exactly what distinguishes +the two cases: an eval that genuinely expects an empty result says so with `finding_count == 0` and +stays GREEN. Silence and asserted-silence are different claims; only the second is a specification. + +**The shape is widely RECOGNISED and still nothing ranges over it — recognition is not coverage.** Four +other production sites defend the identical shape, each written by hand, sharing no helper (all five +import Node stdlib only; zero cross-imports between them): `pharn/floor/check-verify.mjs:99` and +`pharn/floor/check-regress.mjs:376` carry the byte-identical string `is empty — no gates captured`, +written separately; `.dev/floor/check-variance.mjs:171` dies INCONCLUSIVE on `valid.length === 0`; +`.dev/floor/check-contributing-gates.mjs:108` REDs `EMPTY_CHAIN` because "a set with no members is not a +chain". Six further test files carry hand-written comments defending their **own** tests against passing +vacuously — `check-ship-briefing`, `check-provenance`, `check-contributing-gates`, `command-hygiene`, +`entry-point-guard`, `lessons-index-core`. So the failure mode is understood repo-wide, re-derived +locally at each site, and enforced nowhere as a class — and the one place the hand-rolled guard was +missing is `check-structural`, the checker that executes the eval contract itself. + +**Why it matters.** A vacuous pass is the **fail-OPEN** direction of a checker whose entire purpose is to +fail closed, and it is invisible in the direction people look: a suppressed emission and a clean +codebase produce the same GREEN. That makes it the highest-value silent regression an enforcement +pipeline can carry — a lens that stops emitting is certified by its own eval, and P1's "evals are the +spec" degrades to "evals are the spec, unless the output is empty." [[L20]] says a discipline-only +remedy will recur and the second occurrence earns a floor check; here the count reached **five** +hand-written guards before the sixth site slipped, which is that trigger long overdue rather than newly +fired. [[L29]] says a remedy quantified over a set owes the **enumeration** as its deliverable — the +five sites above are that enumeration, materialized in one place for the first time, and the honest +residual is that nothing yet iterates it. Distinct from [[L23]], which also turned on an empty array +hiding a defect: there the emptiness was the **happy path** concealing a self-referential gate conflict +between a stage and its own artifact; here the emptiness is the **domain of quantification** itself, and +the defect is that a universally-quantified claim over it is true for free. Same symptom, different +mechanism — L23's remedy is to exempt an artifact from a gate, this one's is to assert the domain is +non-empty. + +**Provenance.** + +- feature: `floor-hardening` +- commit: `02c2d17e54f49b4b16395b98799726e5e77c5a01` +- source: `.dev/features/floor-hardening/PLAN.md` (its L7 rows in `## Files` + the `## Guarantee audit` + line "L7 'per-finding assertions cannot pass vacuously' → **floor: enum-regex**") + the CHANGELOG + 2.7.13 entry, with the five-site recurrence and the six recognition sites re-derived live at HEAD + `a0cfd0d` +- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). + +## L35 — When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync + +type: process · concepts: [redundant-identity, remedy-design, version-discipline, sync-cost] + +**Lesson.** When the same fact is stored in two places, the first question is whether the second copy +should exist at all — **not** how to keep them agreeing. A sync check is itself a third artifact with its +own wiring, its own invoker and its own failure mode, and it makes the redundancy **permanent by making +it maintained**. Three identities carried the product version: `package.json`'s `version`, +`SKILLS_VERSION`, and the README shields badge. The first read `1.0.0` as a "foundation tag" and sat +stale through the entire 2.x line while every gate stayed green, because nothing bound it and nothing +could have bound it without becoming the fourth thing to maintain. The remedy chosen in `#164` was to +**neutralise the identity, not bind it**: `package.json:4` is now `"version": "0.0.0"` and +`package.json:3` is a `_version_comment` stating the inertness **in the file itself**, so the fact is +self-documenting rather than documented somewhere a reader must find. +`.dev/floor/check-version-badge.mjs` then deliberately does **not** read `package.json` — it names the +field only in its own honest-bounds header (`:26-40`) — and pins the two identities that genuinely +remain: `SKILLS_VERSION` 2.7.15 == the README badge 2.7.15. + +**The precise remedy is "retire", not "delete" — and that distinction is the transferable part.** npm +requires a `version` key, so the redundant copy could not be removed; it was drained of authority +instead, which is the general move when a mandatory slot holds a fact that belongs elsewhere. Give the +slot a value that cannot be mistaken for the truth (`0.0.0`, never a plausible-looking number), and put +the reason **at the slot** rather than in a doc. The rejected alternative — pinning `package.json` to +`SKILLS_VERSION` — is recorded, not silently dropped, in `.dev/features/apparatus-batch/PLAN.md` under +"Open questions (HALT)": option (b) "creates a third identity to keep in sync, which is the defect being +fixed." + +**Why it matters.** [[L20]] says a discipline-only invariant has earned a floor check on its second +occurrence, and read alone it will send you to build a checker every time. This is the qualifier: **a +sync check is the right remedy only once you have established the second copy must exist.** Reach for it +first and you have converted a deletable redundancy into a maintained one, plus a checker whose own +wiring must now be pinned by a test — the cost L20 never charges. Distinct from [[L31]], and exactly +opposite in prescription: L31 governs copies that must **both** exist (the deliberate `check-provenance` +and `lessons-index-core` product/dev pairs, which cannot be merged because a user's install ships +`pharn/floor/` without `.dev/`) and asks what **ranges over** them; this governs a copy that should +**not** exist and says draining beats binding. Both readings are live in this repo, and the question +that separates them is the same one in both directions: _must the second copy exist?_ — asked before +choosing a remedy, never after. + +**Provenance.** + +- feature: `apparatus-batch` +- commit: `a0916b8b7ce84f51b97b3dbe053eda7d96d9e804` +- source: `.dev/features/apparatus-batch/PLAN.md` "Open questions (HALT)" (the L11 option-(a)-over-(b) + record) + the CHANGELOG entry for `#164`, with `package.json:3-4`, + `.dev/floor/check-version-badge.mjs:26-40` and the live `SKILLS_VERSION`/badge agreement re-derived at + HEAD `a0cfd0d` +- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). diff --git a/docs/lessons-index.md b/docs/lessons-index.md index d2f92dd..1f4818a 100644 --- a/docs/lessons-index.md +++ b/docs/lessons-index.md @@ -10,7 +10,7 @@ lessons to fetch; canon stays the source of truth and the floor's verification t applied, without fetching its full `## L` entry from canon, is the P0 disease. "The index was consulted" never means "the relevant lessons were read". -33 lessons · 33 tagged · 0 malformed · 0 untagged · ~21257 tokens total +35 lessons · 35 tagged · 0 malformed · 0 untagged · ~23261 tokens total Columns: `id | type | concepts | title | promoted | ~tokens`. Every canon entry carries a tag line, so BOTH absence markers are unexpected: `-` = no tag line, i.e. an entry that reached canon without @@ -53,4 +53,6 @@ L30 | process | style-gates,command-prescription,lesson-recurrence,floor-escala L31 | process | lesson-recurrence,dev-product-boundary,branch-coverage,floor-escalation | A deliberate copy-pair creates an obligation set nothing ranges over — the second copy is where the obligation is dropped | 2026-08-20 | ~668 L32 | process | verification-fidelity,doc-drift,command-prescription,false-green | A verification method that consults a mutable ALIAS proves reachability, not identity | 2026-08-23 | ~781 L33 | process | doc-drift,shipped-surface,false-green,enumeration | A "not yet built" claim expires the moment the work lands — nothing reads shipped prose, and the repair pass misses the variant spellings | 2026-08-23 | ~1083 +L34 | floor | vacuous-truth,empty-set,fail-closed,eval-design | "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission | 2026-08-23 | ~1083 +L35 | process | redundant-identity,remedy-design,version-discipline,sync-cost | When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync | 2026-08-23 | ~921 ```