From 3817d29a1cc128667bffa1a7c60eca7c1950f707 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Przemys=C5=82aw=20Galarowicz?= Date: Sun, 23 Aug 2026 18:45:01 +0200 Subject: [PATCH 1/3] =?UTF-8?q?docs(memory-bank):=20promote=20L34=20?= =?UTF-8?q?=E2=80=94=20a=20per-item=20assertion=20set=20is=20vacuously=20t?= =?UTF-8?q?rue=20over=20zero=20items?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Promoted via gated /pharn-dev-memory-promote (human-approved at the Step-5 accept/deny halt). The originating defect, fixed in 02c2d17 (#163): pharn/floor/check-structural.mjs's field_equals, file_resolves and needle_absent_from_enum_gated kinds all iterate `findings`, so an eval that wrote per-finding assertions but omitted `finding_count` certified a skill that emitted nothing. Guard now at check-structural.mjs:208-224. What makes it canon rather than a bug report: four other production sites defend the identical shape by hand with zero cross-imports (check-verify.mjs:99, check-regress.mjs:376, check-variance.mjs:171, check-contributing-gates.mjs:108), and six further test files carry hand-written comments defending their own tests against vacuity. The shape is recognised repo-wide and enforced nowhere as a class. Apparatus-only (.dev/**): no SKILLS_VERSION bump, no CHANGELOG entry. docs/lessons-index.md regenerated with the narrow .dev/floor/gen-lessons-index.mjs. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_019LozSBkwtGWc1Rz5Ufv8yj --- .dev/memory-bank/lessons-learned.md | 54 +++++++++++++++++++++++++++++ docs/lessons-index.md | 3 +- 2 files changed, 56 insertions(+), 1 deletion(-) diff --git a/.dev/memory-bank/lessons-learned.md b/.dev/memory-bank/lessons-learned.md index c3b6638..de14376 100644 --- a/.dev/memory-bank/lessons-learned.md +++ b/.dev/memory-bank/lessons-learned.md @@ -1132,3 +1132,57 @@ local, and false. `pharn/pharn-contracts/eval-format.md:52`, `pharn/pharn-contracts/finding-shape.md`, and the seven grillers - promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). + +## L34 — "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission + +type: floor · concepts: [vacuous-truth, empty-set, fail-closed, eval-design] + +**Lesson.** A checker assembled from per-item assertions passes **vacuously** over an empty domain, and +a vacuous pass is indistinguishable from a real one at the verdict. `pharn/floor/check-structural.mjs` +is the instance: its `field_equals`, `file_resolves` and `needle_absent_from_enum_gated` kinds all +iterate `findings`, so an eval that wrote per-finding assertions but omitted `finding_count` certified a +skill that emitted **nothing** — every assertion true, zero findings examined, GREEN. The eval a +Capability ships as its specification (P1) was satisfied by the total absence of the behaviour it +specifies. Remedy, now at `pharn/floor/check-structural.mjs:208-224`: guard the zero-item case +explicitly — membership over the assertion kinds present, plus an integer length test, no judgment (P5). +The legitimate "I expect nothing" intent MUST stay expressible, and that is exactly what distinguishes +the two cases: an eval that genuinely expects an empty result says so with `finding_count == 0` and +stays GREEN. Silence and asserted-silence are different claims; only the second is a specification. + +**The shape is widely RECOGNISED and still nothing ranges over it — recognition is not coverage.** Four +other production sites defend the identical shape, each written by hand, sharing no helper (all five +import Node stdlib only; zero cross-imports between them): `pharn/floor/check-verify.mjs:99` and +`pharn/floor/check-regress.mjs:376` carry the byte-identical string `is empty — no gates captured`, +written separately; `.dev/floor/check-variance.mjs:171` dies INCONCLUSIVE on `valid.length === 0`; +`.dev/floor/check-contributing-gates.mjs:108` REDs `EMPTY_CHAIN` because "a set with no members is not a +chain". Six further test files carry hand-written comments defending their **own** tests against passing +vacuously — `check-ship-briefing`, `check-provenance`, `check-contributing-gates`, `command-hygiene`, +`entry-point-guard`, `lessons-index-core`. So the failure mode is understood repo-wide, re-derived +locally at each site, and enforced nowhere as a class — and the one place the hand-rolled guard was +missing is `check-structural`, the checker that executes the eval contract itself. + +**Why it matters.** A vacuous pass is the **fail-OPEN** direction of a checker whose entire purpose is to +fail closed, and it is invisible in the direction people look: a suppressed emission and a clean +codebase produce the same GREEN. That makes it the highest-value silent regression an enforcement +pipeline can carry — a lens that stops emitting is certified by its own eval, and P1's "evals are the +spec" degrades to "evals are the spec, unless the output is empty." [[L20]] says a discipline-only +remedy will recur and the second occurrence earns a floor check; here the count reached **five** +hand-written guards before the sixth site slipped, which is that trigger long overdue rather than newly +fired. [[L29]] says a remedy quantified over a set owes the **enumeration** as its deliverable — the +five sites above are that enumeration, materialized in one place for the first time, and the honest +residual is that nothing yet iterates it. Distinct from [[L23]], which also turned on an empty array +hiding a defect: there the emptiness was the **happy path** concealing a self-referential gate conflict +between a stage and its own artifact; here the emptiness is the **domain of quantification** itself, and +the defect is that a universally-quantified claim over it is true for free. Same symptom, different +mechanism — L23's remedy is to exempt an artifact from a gate, this one's is to assert the domain is +non-empty. + +**Provenance.** + +- feature: `floor-hardening` +- commit: `02c2d17e54f49b4b16395b98799726e5e77c5a01` +- source: `.dev/features/floor-hardening/PLAN.md` (its L7 rows in `## Files` + the `## Guarantee audit` + line "L7 'per-finding assertions cannot pass vacuously' → **floor: enum-regex**") + the CHANGELOG + 2.7.13 entry, with the five-site recurrence and the six recognition sites re-derived live at HEAD + `a0cfd0d` +- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). diff --git a/docs/lessons-index.md b/docs/lessons-index.md index d2f92dd..1810241 100644 --- a/docs/lessons-index.md +++ b/docs/lessons-index.md @@ -10,7 +10,7 @@ lessons to fetch; canon stays the source of truth and the floor's verification t applied, without fetching its full `## L` entry from canon, is the P0 disease. "The index was consulted" never means "the relevant lessons were read". -33 lessons · 33 tagged · 0 malformed · 0 untagged · ~21257 tokens total +34 lessons · 34 tagged · 0 malformed · 0 untagged · ~22340 tokens total Columns: `id | type | concepts | title | promoted | ~tokens`. Every canon entry carries a tag line, so BOTH absence markers are unexpected: `-` = no tag line, i.e. an entry that reached canon without @@ -53,4 +53,5 @@ L30 | process | style-gates,command-prescription,lesson-recurrence,floor-escala L31 | process | lesson-recurrence,dev-product-boundary,branch-coverage,floor-escalation | A deliberate copy-pair creates an obligation set nothing ranges over — the second copy is where the obligation is dropped | 2026-08-20 | ~668 L32 | process | verification-fidelity,doc-drift,command-prescription,false-green | A verification method that consults a mutable ALIAS proves reachability, not identity | 2026-08-23 | ~781 L33 | process | doc-drift,shipped-surface,false-green,enumeration | A "not yet built" claim expires the moment the work lands — nothing reads shipped prose, and the repair pass misses the variant spellings | 2026-08-23 | ~1083 +L34 | floor | vacuous-truth,empty-set,fail-closed,eval-design | "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission | 2026-08-23 | ~1083 ``` From 02cc2cba3fd82edca99c6f9c6060ce574972ec71 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Przemys=C5=82aw=20Galarowicz?= Date: Sun, 23 Aug 2026 18:52:43 +0200 Subject: [PATCH 2/3] =?UTF-8?q?docs(memory-bank):=20promote=20L35=20?= =?UTF-8?q?=E2=80=94=20retire=20the=20redundant=20identity,=20do=20not=20a?= =?UTF-8?q?dd=20a=20sync=20check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Promoted via gated /pharn-dev-memory-promote (human-approved at the Step-5 accept/deny halt). Qualifies L20. L20 says a discipline-only invariant has earned a floor check on its second occurrence; read alone it sends you to build a checker every time. L35 names the case where a checker is the wrong answer: a sync check is a third artifact with its own wiring, invoker and failure mode, so reaching for it before asking whether the second copy must exist converts a deletable redundancy into a maintained one. Evidence, re-derived live at HEAD a0cfd0d: three identities carried the product version (package.json version, SKILLS_VERSION, README badge); the first read 1.0.0 as a "foundation tag" and sat stale through the whole 2.x line. #164 (a0916b8) neutralised it — package.json:4 is now 0.0.0 with a self-documenting _version_comment at :3 — rather than binding it. The rejected alternative is recorded in .dev/features/apparatus-batch/PLAN.md "Open questions (HALT)". Two corrections to the source brief: SKILLS_VERSION is 2.7.15 (the brief's 2.7.14 was stale as of 71e71ee), and the remedy is stated as "retire", not "delete" — npm requires a version key, so the copy was drained of authority rather than removed. Apparatus-only (.dev/**): no SKILLS_VERSION bump, no CHANGELOG entry. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_019LozSBkwtGWc1Rz5Ufv8yj --- .dev/memory-bank/lessons-learned.md | 49 +++++++++++++++++++++++++++++ docs/lessons-index.md | 3 +- 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/.dev/memory-bank/lessons-learned.md b/.dev/memory-bank/lessons-learned.md index de14376..b2f92f6 100644 --- a/.dev/memory-bank/lessons-learned.md +++ b/.dev/memory-bank/lessons-learned.md @@ -1186,3 +1186,52 @@ non-empty. 2.7.13 entry, with the five-site recurrence and the six recognition sites re-derived live at HEAD `a0cfd0d` - promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). + +## L35 — When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync + +type: process · concepts: [redundant-identity, remedy-design, version-discipline, sync-cost] + +**Lesson.** When the same fact is stored in two places, the first question is whether the second copy +should exist at all — **not** how to keep them agreeing. A sync check is itself a third artifact with its +own wiring, its own invoker and its own failure mode, and it makes the redundancy **permanent by making +it maintained**. Three identities carried the product version: `package.json`'s `version`, +`SKILLS_VERSION`, and the README shields badge. The first read `1.0.0` as a "foundation tag" and sat +stale through the entire 2.x line while every gate stayed green, because nothing bound it and nothing +could have bound it without becoming the fourth thing to maintain. The remedy chosen in `#164` was to +**neutralise the identity, not bind it**: `package.json:4` is now `"version": "0.0.0"` and +`package.json:3` is a `_version_comment` stating the inertness **in the file itself**, so the fact is +self-documenting rather than documented somewhere a reader must find. +`.dev/floor/check-version-badge.mjs` then deliberately does **not** read `package.json` — it names the +field only in its own honest-bounds header (`:26-40`) — and pins the two identities that genuinely +remain: `SKILLS_VERSION` 2.7.15 == the README badge 2.7.15. + +**The precise remedy is "retire", not "delete" — and that distinction is the transferable part.** npm +requires a `version` key, so the redundant copy could not be removed; it was drained of authority +instead, which is the general move when a mandatory slot holds a fact that belongs elsewhere. Give the +slot a value that cannot be mistaken for the truth (`0.0.0`, never a plausible-looking number), and put +the reason **at the slot** rather than in a doc. The rejected alternative — pinning `package.json` to +`SKILLS_VERSION` — is recorded, not silently dropped, in `.dev/features/apparatus-batch/PLAN.md` under +"Open questions (HALT)": option (b) "creates a third identity to keep in sync, which is the defect being +fixed." + +**Why it matters.** [[L20]] says a discipline-only invariant has earned a floor check on its second +occurrence, and read alone it will send you to build a checker every time. This is the qualifier: **a +sync check is the right remedy only once you have established the second copy must exist.** Reach for it +first and you have converted a deletable redundancy into a maintained one, plus a checker whose own +wiring must now be pinned by a test — the cost L20 never charges. Distinct from [[L31]], and exactly +opposite in prescription: L31 governs copies that must **both** exist (the deliberate `check-provenance` +and `lessons-index-core` product/dev pairs, which cannot be merged because a user's install ships +`pharn/floor/` without `.dev/`) and asks what **ranges over** them; this governs a copy that should +**not** exist and says draining beats binding. Both readings are live in this repo, and the question +that separates them is the same one in both directions: _must the second copy exist?_ — asked before +choosing a remedy, never after. + +**Provenance.** + +- feature: `apparatus-batch` +- commit: `a0916b8b7ce84f51b97b3dbe053eda7d96d9e804` +- source: `.dev/features/apparatus-batch/PLAN.md` "Open questions (HALT)" (the L11 option-(a)-over-(b) + record) + the CHANGELOG entry for `#164`, with `package.json:3-4`, + `.dev/floor/check-version-badge.mjs:26-40` and the live `SKILLS_VERSION`/badge agreement re-derived at + HEAD `a0cfd0d` +- promoted: 2026-08-23 via gated `/pharn-dev-memory-promote` (human-approved). diff --git a/docs/lessons-index.md b/docs/lessons-index.md index 1810241..1f4818a 100644 --- a/docs/lessons-index.md +++ b/docs/lessons-index.md @@ -10,7 +10,7 @@ lessons to fetch; canon stays the source of truth and the floor's verification t applied, without fetching its full `## L` entry from canon, is the P0 disease. "The index was consulted" never means "the relevant lessons were read". -34 lessons · 34 tagged · 0 malformed · 0 untagged · ~22340 tokens total +35 lessons · 35 tagged · 0 malformed · 0 untagged · ~23261 tokens total Columns: `id | type | concepts | title | promoted | ~tokens`. Every canon entry carries a tag line, so BOTH absence markers are unexpected: `-` = no tag line, i.e. an entry that reached canon without @@ -54,4 +54,5 @@ L31 | process | lesson-recurrence,dev-product-boundary,branch-coverage,floor-es L32 | process | verification-fidelity,doc-drift,command-prescription,false-green | A verification method that consults a mutable ALIAS proves reachability, not identity | 2026-08-23 | ~781 L33 | process | doc-drift,shipped-surface,false-green,enumeration | A "not yet built" claim expires the moment the work lands — nothing reads shipped prose, and the repair pass misses the variant spellings | 2026-08-23 | ~1083 L34 | floor | vacuous-truth,empty-set,fail-closed,eval-design | "For each X, assert P" says nothing when there are no X — a per-item assertion set certifies the suppressed emission | 2026-08-23 | ~1083 +L35 | process | redundant-identity,remedy-design,version-discipline,sync-cost | When one fact is stored twice, retire the second copy — a sync check is a third thing to keep in sync | 2026-08-23 | ~921 ``` From 00f7a7d2399ddda82eb0b42b7968920da507526e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Przemys=C5=82aw=20Galarowicz?= Date: Sun, 23 Aug 2026 19:03:37 +0200 Subject: [PATCH 3/3] feat(floor): register expired forward-looking claims on the existing marker checker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit L20's trigger has fired for the forward-looking-prose class: it recurred INSIDE #165, the increment whose entire purpose was fixing it — that PR corrected one "next increment" claim in pharn/pharn-contracts/eval-format.md and left a second in the same file. A discipline-only remedy ("remember to re-derive the tense") has had its second occurrence, so the class gets a check. NOT a new checker, and that inversion is the substance of the increment. The originating note specified "a manifest plus a checker ... a real increment". But .dev/floor/check-specified-markers.mjs already computes the exact truth table: probe live + marker present -> RED (direction 1: the doc UNDERSTATES reality) probe absent + marker absent -> RED (direction 2: silent return to overclaiming) An expired forward claim IS direction 1 — the named artifact shipped, the hedge remains. So the class is REGISTERED rather than reimplemented, reusing isLive(), readDoc(), the fail-closed manifest loader and both direction branches. Only the message wording and the summary count are new. This is L35 applied: a parallel manifest + checker + npm script + ci.yml step + wiring tests would be a second identity for logic that already exists. Adding no new wiring is the point — there is no new invoker to pin because there is no new invoker. Registered (probe = unambiguous path test): - pharn-eval probe .claude/commands/pharn-eval.md - product-capability-catalog probe pharn/floor/gen-capability-catalog.mjs Deliberately NOT registered, recorded in the manifest so the absence does not read as oversight: the ~20 P7 "future increment, when a real need surfaces" statements are conditional design postures, not pinned predictions — they would fire on correct sentences. The live-griller-runner class (13+ sites) and /pharn-verify's verifier runner are real expiring claims but neither subject has a NAMED path, so a probe would have to invent one; deferred with reopen triggers rather than guessed (P6). L34 applied to this increment's own code: every check is "for each registered claim, assert P", vacuously true over zero claims. A present-but-EMPTY forward_claims array is exit 2 (EMPTY_CLASS), never GREEN; OMITTING the key is the expressible "none registered" state. The two are distinguishable, which is what keeps the guard honest. Probe paths verified live, not just by fixture: landing either artifact in a scratch tree fires direction 1 with the correct message. npm run check green — 1634 tests (+14), 8 gates. Apparatus-only (.dev/**): no SKILLS_VERSION bump, no CHANGELOG entry. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_019LozSBkwtGWc1Rz5Ufv8yj --- .../forward-claim-marker-check/PLAN.md | 117 +++++++++++++ .dev/floor/check-specified-markers.mjs | 85 +++++++++- .dev/floor/check-specified-markers.test.mjs | 154 ++++++++++++++++++ .dev/floor/specified-primitives.json | 49 ++++++ 4 files changed, 401 insertions(+), 4 deletions(-) create mode 100644 .dev/features/forward-claim-marker-check/PLAN.md diff --git a/.dev/features/forward-claim-marker-check/PLAN.md b/.dev/features/forward-claim-marker-check/PLAN.md new file mode 100644 index 0000000..f1ebc76 --- /dev/null +++ b/.dev/features/forward-claim-marker-check/PLAN.md @@ -0,0 +1,117 @@ +# PLAN — register the forward-claim class on the EXISTING marker checker (no new checker) + +- spec_content_hash: 8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52 # fix #4 +- applied_lessons: [L6, L20, L29, L34, L35] +- increment: Give expired forward-looking claims a floor check by REGISTERING them as a new claim class on `.dev/floor/check-specified-markers.mjs` + `specified-primitives.json`, rather than building the parallel checker the originating note assumed. The existing checker's truth table already IS the forward-claim semantics; what was missing is a membership list. +- layer(s): none — build apparatus (`.dev/`). Nothing here ships. # pharn/ARCHITECTURE.md §4 +- constitution_refs: [P0, P5, P6, P7] + +## Applied lessons + +- L35 — The decisive one, and it inverted the increment. The originating note + (`.pharn/fixes/05-deferred-forward-claim-marker-check.md`) specified "a manifest plus a checker … + a real increment". Building it would have created a second manifest, a second checker, a second + `npm run check` entry, a second `ci.yml` step and a second pair of wiring tests — a parallel + identity for logic that already exists, which is exactly the shape L35 says to refuse. The first + question is whether the second copy must exist; here it must not. Registered as a claim class + instead. +- L20 — The trigger is genuinely fired, which is why this is built at all rather than deferred: the + forward-claim class recurred INSIDE `#165`, the increment whose entire purpose was fixing it (it + corrected one "next increment" claim in `pharn/pharn-contracts/eval-format.md` and left a second at + line 52). A discipline-only remedy — "remember to re-derive the tense" — has had its second + occurrence. +- L6 — Membership is read from the STRUCTURED manifest, never by scanning doc prose for what looks + like a forward-looking sentence. A prose scan cannot distinguish `check-spec.mjs`'s "a Draft is not + yet pinned" (a correct runtime-state sentence) from an expired claim, and it would count a CHANGELOG + sentence quoting a marker as a doc site — the identical defect the existing checker's header records. +- L29 — The remedy is quantified over "every registered forward claim", so the ENUMERATION is the + deliverable. `forward_claims[]` is that enumeration in one place, with the checker's loop iterating + it, so a claim added later is covered by both directions for free. +- L34 — Applied directly to this increment's own code, and it is the reason for the `EMPTY_CLASS` + guard below. Every check here is "for each registered claim, assert P", which is VACUOUSLY TRUE over + an empty `forward_claims[]`. Without a guard, deleting the array (or shipping the section with zero + entries) yields a confident GREEN over nothing. The legitimate "no claims registered yet" state must + stay expressible, and is — by OMITTING the key entirely, which is distinguishable from an empty array. + +## Files + +- `.dev/floor/specified-primitives.json` — add the `forward_claims[]` array + its `$comment` bound — layer apparatus +- `.dev/floor/check-specified-markers.mjs` — process `forward_claims[]` through the existing probe/direction logic; own wording, own count, `EMPTY_CLASS` fail-closed guard — layer apparatus +- `.dev/floor/check-specified-markers.test.mjs` — tests: both directions per class, the `EMPTY_CLASS` guard, manifest-validation fail-closed — layer apparatus +- `.dev/features/forward-claim-marker-check/PLAN.md` — this plan — layer apparatus + +## The design, and why it needed no new logic + +`check-specified-markers.mjs` already computes, per site: + +| probe | marker | verdict | +| ------ | ------- | -------------------------------------------------------- | +| live | present | RED — direction 1, the doc UNDERSTATES a live protection | +| absent | absent | RED — direction 2, silent return to overclaiming | +| live | absent | GREEN | +| absent | present | GREEN | + +An expired forward-looking claim is exactly row 1: the named artifact shipped, the "not yet built" +sentence remains. The inverse — the sentence deleted while the artifact is still absent — is exactly +row 2. So `forward_claims[]` reuses `isLive()`, `readDoc()`, the manifest loader's fail-closed exit 2, +and both direction branches. Only the MESSAGE WORDING and the summary COUNT are new, because +"specified primitive" and "forward claim" read differently to a human even though they compute +identically. + +## Scope — what is registered, and what is deliberately NOT + +**Registered.** Claims naming a CONCRETE artifact whose appearance falsifies the sentence, where the +probe is an unambiguous path test: + +- `pharn-eval` — the product `/pharn-eval` twin, probe `.claude/commands/pharn-eval.md` +- `product-capability-catalog` — probe `pharn/floor/gen-capability-catalog.mjs` + +**NOT registered, and the reason stated in the manifest.** The ~20 P7 "future increment, added when a +real need surfaces" statements across `pharn/pharn-review/*` and the `scan-code-*` scanners. These are +CONDITIONAL DESIGN POSTURES, not pinned predictions: they assert a policy ("we do not build +speculatively"), which stays true whether or not the thing is later built for a real reason. +Registering them would add ~20 entries that fire on a correct sentence — noise that trains the reader +to ignore the check. This is a scoping judgment, recorded rather than silently made. + +**Deferred with a named reopen trigger.** The `live griller runner` class — 13+ sites across every +griller, all citing one subject — is the highest-count class and is NOT registered, because the repo +has never named a path for that runner. A probe would have to invent one, and a probe pointing at a +path nobody has agreed on is a guess (P6). **Reopens when** the griller runner is named in a plan or +built. The same holds for `/pharn-verify`'s verifier runner, whose stated trigger is "the first +`role: verifier` capability authored outside PHARN's own surface" — a membership question over +frontmatter, not a path test, so it needs a probe type that does not exist yet. + +## Guarantee audit (P0) + +- "a REGISTERED forward claim cannot silently expire" → **FLOOR: enum-regex** (primitive #3 — path + existence + exact substring presence), both directions, inherited from the existing checker. +- "the docs carry no expired forward claims" → **NOT CLAIMED.** The manifest is a hand-maintained + address book. It cannot DISCOVER an unregistered claim; a doc that starts asserting some other + not-yet-built thing tomorrow is invisible until a human adds the entry. This is the same bound the + existing checker states about itself, inherited verbatim and re-stated for the new class. +- "an empty `forward_claims[]` is a real GREEN" → **FLOOR, and this is the L34 guard.** A present-but- + empty array is exit 2 (`EMPTY_CLASS`), never GREEN. Omitting the key entirely is the expressible + "none registered" state, and is GREEN by design — the two are distinguishable, which is what makes + the guard honest rather than an obstacle. +- "the probe proves the artifact WORKS" → **NOT CLAIMED.** It tests existence. A stub at the probe path + flips the claim to expired and REDs direction 1 — deliberately, as for the existing classes. +- "this checker RUNS" → **ADVISORY**, and unchanged: it inherits the EXISTING `check:markers` wiring in + `npm run check` and `ci.yml`. **Adding no new wiring is the point of the increment** (L35) — there is + no new invoker to pin, because there is no new invoker. + +## Trust audit (P2) + +Unchanged from the existing checker. The manifest is trusted, human-reviewed apparatus; the docs it +reads are trusted. Doc bytes are opaque DATA — compared with `String.prototype.includes`, never parsed, +interpreted or executed. No untrusted input is ingested and no free text steers a branch. + +## Open questions (HALT) + +None. The one genuine ambiguity — the probe path for the `live griller runner` class — is resolved by +NOT registering it and recording the reopen trigger, rather than inventing a path (P6: the terminal +fallback is to ask/defer, never to guess). + +## SKILLS_VERSION + +**No bump.** Everything here is `.dev/**` build apparatus; no product-surface byte changes. No +CHANGELOG entry is owed either, per the dev/product boundary in CLAUDE.md. diff --git a/.dev/floor/check-specified-markers.mjs b/.dev/floor/check-specified-markers.mjs index 93ec3d5..ce7e212 100644 --- a/.dev/floor/check-specified-markers.mjs +++ b/.dev/floor/check-specified-markers.mjs @@ -30,6 +30,25 @@ // 2. A MARKER IS DELETED, THE PRIMITIVE IS STILL ABSENT. The doc silently returns to overclaiming — // the original F7 defect. RED, naming the file and the missing marker. // +// THE SECOND CLASS: FORWARD-LOOKING CLAIMS (`forward_claims` in the manifest). A sentence like "the +// checker that runs these is the NEXT increment" is TRUE when written and FALSE the moment the named +// artifact lands, in a file nobody is editing. That is the SAME computation as above with a different +// marker vocabulary: direction 1 is the claim EXPIRING (the artifact shipped, the hedge remains), and +// direction 2 is the hedge being DELETED while the artifact is still absent (the doc now implies +// something exists that does not). Registered here rather than in a second checker with its own +// manifest, npm script, ci.yml step and wiring tests — lessons-learned L35: when the existing thing +// already computes the answer, a parallel identity is the wrong remedy. Adding NO new wiring is the +// point; there is no new invoker to pin because there is no new invoker. +// +// The trigger (P7 — a real failure, never a hypothetical): the class recurred INSIDE #165, the +// increment whose entire purpose was fixing it — it corrected one "next increment" claim in +// pharn/pharn-contracts/eval-format.md and left a second in the same file. Per L20 that second +// occurrence is when a discipline-only remedy has earned a check. +// +// AN EMPTY `forward_claims` ARRAY IS EXIT 2 (EMPTY_CLASS), NEVER GREEN — lessons-learned L34: every +// check here is "for each registered claim, assert P", which is vacuously true over zero claims, so an +// emptied array would certify nothing. OMITTING the key is the expressible "none registered" state. +// // WHY A STRUCTURED MANIFEST (lessons-learned L6). Membership is read from specified-primitives.json — // never by scanning doc prose for what LOOKS like a marker. L6's defect (a membership fact grepped out // of free text, so documentation ABOUT a declaration counts as a declaration) recurred inside F7's own @@ -94,6 +113,29 @@ if (!Array.isArray(primitives)) { process.exit(2); } +// FORWARD-LOOKING CLAIMS — the same two directions, a different marker vocabulary. See the manifest's +// $forward_claims_comment for what is deliberately unregistered. +// +// The key is OPTIONAL (omitted => none registered, a real GREEN), but a PRESENT-and-EMPTY array is +// exit 2. Every check below is "for each registered claim, assert P", which is VACUOUSLY TRUE over +// zero claims (lessons-learned L34), so an emptied array would otherwise certify nothing as GREEN. +// Omission and emptiness are distinguishable, which is what keeps the honest "none yet" state +// expressible — the same shape as check-structural.mjs's `finding_count == 0` escape. +const hasForwardKey = Object.hasOwn(manifest, "forward_claims"); +const forward = hasForwardKey ? manifest.forward_claims : []; +if (hasForwardKey && !Array.isArray(forward)) { + console.error(`RED (manifest unusable): ${MANIFEST} has a \`forward_claims\` key that is not an array`); + process.exit(2); +} +if (hasForwardKey && forward.length === 0) { + console.error( + `RED (manifest unusable): ${MANIFEST} declares an EMPTY \`forward_claims\` array (EMPTY_CLASS).\n` + + ` Per-claim assertions pass vacuously over zero claims, so this would certify nothing as GREEN.\n` + + ` To register no forward claims, OMIT the \`forward_claims\` key entirely.` + ); + process.exit(2); +} + /** * Is the primitive LIVE? A pure existence test (ARCHITECTURE §2 primitive #3). * Unknown probe type => throw, so a typo in the manifest fails CLOSED (exit 2) rather than @@ -175,6 +217,8 @@ function readDoc(rel) { let siteCount = 0; let liveCount = 0; +let claimCount = 0; +let landedCount = 0; try { for (const p of primitives) { @@ -207,6 +251,36 @@ try { } } + for (const c of forward) { + validatePrimitive(c); // identical record shape: id + probe + sites[{file, marker}] + const landed = isLive(c.probe, c.id); + if (landed) landedCount++; + for (const site of c.sites) { + claimCount++; + const src = readDoc(site.file); + if (src === null) { + red(`${site.file}: listed as a site for forward claim \`${c.id}\` but the file could not be read`); + continue; + } + const present = src.includes(site.marker); + if (landed && present) { + // DIRECTION 1 — the claim EXPIRED: the artifact landed, the sentence still says it has not. + red( + `${site.file}: forward claim \`${c.id}\` has EXPIRED — ${describe(c.probe)}, but the doc still ` + + `says it is not built. RE-DERIVE the sentence; the doc now describes the repo as weaker than it is.\n` + + ` marker: ${JSON.stringify(site.marker)}` + ); + } else if (!landed && !present) { + // DIRECTION 2 — the hedge was deleted while the artifact is still absent. + red( + `${site.file}: the forward-claim hedge for \`${c.id}\` is GONE, but the artifact is still absent ` + + `(${describe(c.probe)}). The doc now implies something exists that does not.\n` + + ` expected: ${JSON.stringify(site.marker)}` + ); + } + } + } + for (const a of named) { validateNamedArtifact(a); const src = readDoc(a.cited_in); @@ -241,14 +315,17 @@ if (reds.length) { console.error(`SPECIFIED-MARKERS: RED — ${reds.length} drifted annotation(s)\n`); for (const r of reds) console.error(` - ${r}`); console.error( - `\nManifest: ${MANIFEST}\n` + `This checks only the annotations that manifest LISTS. It cannot discover a new overclaim (P0).` + `\nManifest: ${MANIFEST}\n` + + `This checks only the annotations and forward claims that manifest LISTS. It cannot discover a new ` + + `overclaim, or a forward-looking claim nobody registered (P0).` ); process.exit(1); } console.log( `SPECIFIED-MARKERS: GREEN — ${siteCount} annotation(s) across ${primitives.length} specified primitive(s) ` + - `(${liveCount} now live), ${named.length} named artifact(s) checked in ${TARGET}.\n` + - `NOTE (P0): this proves the LISTED annotations still match reality. It never means the docs are accurate — ` + - `an overclaim not in the manifest is invisible here.` + `(${liveCount} now live), ${claimCount} forward-claim site(s) across ${forward.length} registered claim(s) ` + + `(${landedCount} now landed), ${named.length} named artifact(s) checked in ${TARGET}.\n` + + `NOTE (P0): this proves the LISTED annotations and claims still match reality. It never means the docs are ` + + `accurate — an overclaim or an expired claim not in the manifest is invisible here.` ); diff --git a/.dev/floor/check-specified-markers.test.mjs b/.dev/floor/check-specified-markers.test.mjs index 43a6feb..e9c891c 100644 --- a/.dev/floor/check-specified-markers.test.mjs +++ b/.dev/floor/check-specified-markers.test.mjs @@ -259,3 +259,157 @@ test("the GREEN line states its own P0 bound (the manifest is not a discovery me assert.equal(r.code, 0, r.out); assert.match(r.out, /never means the docs are accurate/); }); + +// ------------------------------------------------------ forward-looking claims (the second class) +// +// Same two directions, different vocabulary: direction 1 is the claim EXPIRING (artifact landed, hedge +// remains), direction 2 is the hedge being DELETED while the artifact is still absent. L4 again — the +// RED paths are driven with real fixture trees, and the ✧ cases are mutants. + +const HEDGE = "no `pharn-eval` command exists"; + +/** + * Fixture for the forward-claim class. `opts.landed` creates the probed artifact; + * `opts.hedge === false` omits the hedge sentence from the doc. + * `opts.forward` overrides the manifest's forward_claims wholesale (including omitting it). + */ +function fwFixture(opts = {}) { + const dir = mkdtempSync(join(tmpdir(), "pharn-forward-")); + mkdirSync(join(dir, ".claude", "commands"), { recursive: true }); + if (opts.landed) writeFileSync(join(dir, ".claude", "commands", "pharn-eval.md"), "# stub\n"); + writeFileSync(join(dir, "DOC.md"), opts.hedge === false ? "intro\noutro\n" : `intro\n${HEDGE}\noutro\n`); + + const manifest = { specified_primitives: [] }; + if (!opts.omitForward) { + manifest.forward_claims = opts.forward ?? [ + { + id: "pharn-eval", + probe: { type: "path", path: ".claude/commands/pharn-eval.md" }, + sites: [{ file: "DOC.md", marker: HEDGE }], + }, + ]; + } + const mPath = join(dir, "manifest.json"); + writeFileSync(mPath, JSON.stringify(manifest)); + return { dir, mPath, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +test("GREEN — artifact absent and the hedge is present (the steady state)", () => { + const f = fwFixture(); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /GREEN/); +}); + +test("GREEN — artifact landed and the hedge was re-derived away (the healthy transition)", () => { + const f = fwFixture({ landed: true, hedge: false }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); +}); + +test("✧ DIRECTION 1 — the artifact LANDED and the hedge remains: the claim EXPIRED, RED", () => { + const f = fwFixture({ landed: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /EXPIRED/); + assert.match(r.out, /pharn-eval/); + assert.match(r.out, /DOC\.md/); +}); + +test("✧ DIRECTION 1 names the marker to re-derive, not merely the file", () => { + const f = fwFixture({ landed: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.match(r.out, /marker: /); + assert.match(r.out, /no `pharn-eval` command exists/); +}); + +test("✧ DIRECTION 2 — the hedge was DELETED while the artifact is still absent, RED", () => { + const f = fwFixture({ hedge: false }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /is GONE/); + assert.match(r.out, /implies something exists that does not/); +}); + +test("✧ L34 — a PRESENT but EMPTY forward_claims array is exit 2 (EMPTY_CLASS), never GREEN", () => { + const f = fwFixture({ forward: [] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /EMPTY_CLASS/); + assert.match(r.out, /vacuously/); +}); + +test("✧ L34 — the EMPTY_CLASS refusal names the escape (omit the key), so the honest state stays expressible", () => { + const f = fwFixture({ forward: [] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.match(r.out, /OMIT the `forward_claims` key/); +}); + +test("GREEN — OMITTING forward_claims entirely is the expressible 'none registered' state", () => { + const f = fwFixture({ omitForward: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /0 registered claim/); +}); + +test("✧ a forward_claims key that is not an array fails CLOSED (exit 2)", () => { + const f = fwFixture({ forward: { id: "not-an-array" } }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /not an array/); +}); + +test("✧ a forward-claim site whose file cannot be read is RED, not a silent skip", () => { + const f = fwFixture({ + forward: [ + { + id: "pharn-eval", + probe: { type: "path", path: ".claude/commands/pharn-eval.md" }, + sites: [{ file: "NOPE.md", marker: HEDGE }], + }, + ], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /could not be read/); +}); + +test("✧ a malformed forward-claim record fails CLOSED (exit 2), same validator as primitives", () => { + const f = fwFixture({ forward: [{ id: "pharn-eval", probe: { type: "path", path: "x" } }] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /`sites` must be an array/); +}); + +test("✧ an unknown probe type on a forward claim fails CLOSED, never 'not landed'", () => { + const f = fwFixture({ + forward: [{ id: "pharn-eval", probe: { type: "vibes" }, sites: [{ file: "DOC.md", marker: HEDGE }] }], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /unknown probe type/); +}); + +test("integration — the REAL manifest registers at least one forward claim (the class stays live)", () => { + const r = run(REPO); + assert.equal(r.code, 0, r.out); + const m = r.out.match(/across (\d+) registered claim/); + assert.ok(m && Number(m[1]) >= 1, `expected >= 1 registered forward claim, got: ${r.out}`); +}); + +test("the GREEN line reports the forward-claim counts, so a silently emptied class is visible", () => { + const r = run(REPO); + assert.match(r.out, /forward-claim site\(s\) across \d+ registered claim\(s\)/); +}); diff --git a/.dev/floor/specified-primitives.json b/.dev/floor/specified-primitives.json index 39cdbe8..a326994 100644 --- a/.dev/floor/specified-primitives.json +++ b/.dev/floor/specified-primitives.json @@ -102,5 +102,54 @@ "citation": "(security\ngriller,", "must_exist": "pharn/pharn-pipeline/grillers/security" } + ], + + "$forward_claims_comment": [ + "FORWARD-LOOKING CLAIMS: sentences that are TRUE when written and become FALSE the moment the named", + "artifact lands. Computed by the SAME two directions as specified_primitives (a probe + an exact", + "substring), because the semantics are identical — an expired forward claim IS direction 1, the doc", + "understating reality. Registered here rather than in a second checker: a parallel manifest+checker", + "would be a second identity with its own wiring and invoker, which lessons-learned L35 says to", + "refuse when the existing one already computes the answer.", + "", + "WHAT IS DELIBERATELY NOT REGISTERED, so the absence does not read as an oversight:", + " * The ~20 P7 'future increment, added when a real need surfaces' statements across", + " pharn/pharn-review/* and the scan-code-* scanners. Those assert a POLICY (we do not build", + " speculatively), which stays true whether or not the thing is later built for a real reason.", + " They are conditional design postures, not pinned predictions; registering them would fire on", + " correct sentences and train the reader to ignore the check.", + " * The `live griller runner` class (13+ sites across every griller) and /pharn-verify's verifier", + " runner. Both are real expiring claims, but neither subject has a NAMED path in this repo, so a", + " probe would have to invent one. Deferred rather than guessed (P6). Reopens when the runner is", + " named in a plan or built, or when the first `role: verifier` capability is authored.", + "", + "An EMPTY forward_claims array is exit 2 (EMPTY_CLASS), never GREEN — per-claim assertions are", + "vacuously true over zero claims (lessons-learned L34). To express 'none registered', OMIT the key." + ], + + "forward_claims": [ + { + "id": "pharn-eval", + "what": "the product /pharn-eval twin of /pharn-dev-eval — DEFERRED 2026-08-23, reopening when the first role:-bearing capability is authored outside PHARN's own shipped surface", + "probe": { "type": "path", "path": ".claude/commands/pharn-eval.md" }, + "sites": [ + { + "file": "CLAUDE.md", + "marker": "no `pharn-eval` command exists" + } + ] + }, + + { + "id": "product-capability-catalog", + "what": "a generated capability catalog on the PRODUCT surface — DEFERRED 2026-08-07, reopening on the same trigger", + "probe": { "type": "path", "path": "pharn/floor/gen-capability-catalog.mjs" }, + "sites": [ + { + "file": "CLAUDE.md", + "marker": "nothing under `pharn/floor/` renders one" + } + ] + } ] }