diff --git a/.dev/features/forward-claim-marker-check/PLAN.md b/.dev/features/forward-claim-marker-check/PLAN.md new file mode 100644 index 0000000..f1ebc76 --- /dev/null +++ b/.dev/features/forward-claim-marker-check/PLAN.md @@ -0,0 +1,117 @@ +# PLAN — register the forward-claim class on the EXISTING marker checker (no new checker) + +- spec_content_hash: 8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52 # fix #4 +- applied_lessons: [L6, L20, L29, L34, L35] +- increment: Give expired forward-looking claims a floor check by REGISTERING them as a new claim class on `.dev/floor/check-specified-markers.mjs` + `specified-primitives.json`, rather than building the parallel checker the originating note assumed. The existing checker's truth table already IS the forward-claim semantics; what was missing is a membership list. +- layer(s): none — build apparatus (`.dev/`). Nothing here ships. # pharn/ARCHITECTURE.md §4 +- constitution_refs: [P0, P5, P6, P7] + +## Applied lessons + +- L35 — The decisive one, and it inverted the increment. The originating note + (`.pharn/fixes/05-deferred-forward-claim-marker-check.md`) specified "a manifest plus a checker … + a real increment". Building it would have created a second manifest, a second checker, a second + `npm run check` entry, a second `ci.yml` step and a second pair of wiring tests — a parallel + identity for logic that already exists, which is exactly the shape L35 says to refuse. The first + question is whether the second copy must exist; here it must not. Registered as a claim class + instead. +- L20 — The trigger is genuinely fired, which is why this is built at all rather than deferred: the + forward-claim class recurred INSIDE `#165`, the increment whose entire purpose was fixing it (it + corrected one "next increment" claim in `pharn/pharn-contracts/eval-format.md` and left a second at + line 52). A discipline-only remedy — "remember to re-derive the tense" — has had its second + occurrence. +- L6 — Membership is read from the STRUCTURED manifest, never by scanning doc prose for what looks + like a forward-looking sentence. A prose scan cannot distinguish `check-spec.mjs`'s "a Draft is not + yet pinned" (a correct runtime-state sentence) from an expired claim, and it would count a CHANGELOG + sentence quoting a marker as a doc site — the identical defect the existing checker's header records. +- L29 — The remedy is quantified over "every registered forward claim", so the ENUMERATION is the + deliverable. `forward_claims[]` is that enumeration in one place, with the checker's loop iterating + it, so a claim added later is covered by both directions for free. +- L34 — Applied directly to this increment's own code, and it is the reason for the `EMPTY_CLASS` + guard below. Every check here is "for each registered claim, assert P", which is VACUOUSLY TRUE over + an empty `forward_claims[]`. Without a guard, deleting the array (or shipping the section with zero + entries) yields a confident GREEN over nothing. The legitimate "no claims registered yet" state must + stay expressible, and is — by OMITTING the key entirely, which is distinguishable from an empty array. + +## Files + +- `.dev/floor/specified-primitives.json` — add the `forward_claims[]` array + its `$comment` bound — layer apparatus +- `.dev/floor/check-specified-markers.mjs` — process `forward_claims[]` through the existing probe/direction logic; own wording, own count, `EMPTY_CLASS` fail-closed guard — layer apparatus +- `.dev/floor/check-specified-markers.test.mjs` — tests: both directions per class, the `EMPTY_CLASS` guard, manifest-validation fail-closed — layer apparatus +- `.dev/features/forward-claim-marker-check/PLAN.md` — this plan — layer apparatus + +## The design, and why it needed no new logic + +`check-specified-markers.mjs` already computes, per site: + +| probe | marker | verdict | +| ------ | ------- | -------------------------------------------------------- | +| live | present | RED — direction 1, the doc UNDERSTATES a live protection | +| absent | absent | RED — direction 2, silent return to overclaiming | +| live | absent | GREEN | +| absent | present | GREEN | + +An expired forward-looking claim is exactly row 1: the named artifact shipped, the "not yet built" +sentence remains. The inverse — the sentence deleted while the artifact is still absent — is exactly +row 2. So `forward_claims[]` reuses `isLive()`, `readDoc()`, the manifest loader's fail-closed exit 2, +and both direction branches. Only the MESSAGE WORDING and the summary COUNT are new, because +"specified primitive" and "forward claim" read differently to a human even though they compute +identically. + +## Scope — what is registered, and what is deliberately NOT + +**Registered.** Claims naming a CONCRETE artifact whose appearance falsifies the sentence, where the +probe is an unambiguous path test: + +- `pharn-eval` — the product `/pharn-eval` twin, probe `.claude/commands/pharn-eval.md` +- `product-capability-catalog` — probe `pharn/floor/gen-capability-catalog.mjs` + +**NOT registered, and the reason stated in the manifest.** The ~20 P7 "future increment, added when a +real need surfaces" statements across `pharn/pharn-review/*` and the `scan-code-*` scanners. These are +CONDITIONAL DESIGN POSTURES, not pinned predictions: they assert a policy ("we do not build +speculatively"), which stays true whether or not the thing is later built for a real reason. +Registering them would add ~20 entries that fire on a correct sentence — noise that trains the reader +to ignore the check. This is a scoping judgment, recorded rather than silently made. + +**Deferred with a named reopen trigger.** The `live griller runner` class — 13+ sites across every +griller, all citing one subject — is the highest-count class and is NOT registered, because the repo +has never named a path for that runner. A probe would have to invent one, and a probe pointing at a +path nobody has agreed on is a guess (P6). **Reopens when** the griller runner is named in a plan or +built. The same holds for `/pharn-verify`'s verifier runner, whose stated trigger is "the first +`role: verifier` capability authored outside PHARN's own surface" — a membership question over +frontmatter, not a path test, so it needs a probe type that does not exist yet. + +## Guarantee audit (P0) + +- "a REGISTERED forward claim cannot silently expire" → **FLOOR: enum-regex** (primitive #3 — path + existence + exact substring presence), both directions, inherited from the existing checker. +- "the docs carry no expired forward claims" → **NOT CLAIMED.** The manifest is a hand-maintained + address book. It cannot DISCOVER an unregistered claim; a doc that starts asserting some other + not-yet-built thing tomorrow is invisible until a human adds the entry. This is the same bound the + existing checker states about itself, inherited verbatim and re-stated for the new class. +- "an empty `forward_claims[]` is a real GREEN" → **FLOOR, and this is the L34 guard.** A present-but- + empty array is exit 2 (`EMPTY_CLASS`), never GREEN. Omitting the key entirely is the expressible + "none registered" state, and is GREEN by design — the two are distinguishable, which is what makes + the guard honest rather than an obstacle. +- "the probe proves the artifact WORKS" → **NOT CLAIMED.** It tests existence. A stub at the probe path + flips the claim to expired and REDs direction 1 — deliberately, as for the existing classes. +- "this checker RUNS" → **ADVISORY**, and unchanged: it inherits the EXISTING `check:markers` wiring in + `npm run check` and `ci.yml`. **Adding no new wiring is the point of the increment** (L35) — there is + no new invoker to pin, because there is no new invoker. + +## Trust audit (P2) + +Unchanged from the existing checker. The manifest is trusted, human-reviewed apparatus; the docs it +reads are trusted. Doc bytes are opaque DATA — compared with `String.prototype.includes`, never parsed, +interpreted or executed. No untrusted input is ingested and no free text steers a branch. + +## Open questions (HALT) + +None. The one genuine ambiguity — the probe path for the `live griller runner` class — is resolved by +NOT registering it and recording the reopen trigger, rather than inventing a path (P6: the terminal +fallback is to ask/defer, never to guess). + +## SKILLS_VERSION + +**No bump.** Everything here is `.dev/**` build apparatus; no product-surface byte changes. No +CHANGELOG entry is owed either, per the dev/product boundary in CLAUDE.md. diff --git a/.dev/floor/check-specified-markers.mjs b/.dev/floor/check-specified-markers.mjs index 93ec3d5..ce7e212 100644 --- a/.dev/floor/check-specified-markers.mjs +++ b/.dev/floor/check-specified-markers.mjs @@ -30,6 +30,25 @@ // 2. A MARKER IS DELETED, THE PRIMITIVE IS STILL ABSENT. The doc silently returns to overclaiming — // the original F7 defect. RED, naming the file and the missing marker. // +// THE SECOND CLASS: FORWARD-LOOKING CLAIMS (`forward_claims` in the manifest). A sentence like "the +// checker that runs these is the NEXT increment" is TRUE when written and FALSE the moment the named +// artifact lands, in a file nobody is editing. That is the SAME computation as above with a different +// marker vocabulary: direction 1 is the claim EXPIRING (the artifact shipped, the hedge remains), and +// direction 2 is the hedge being DELETED while the artifact is still absent (the doc now implies +// something exists that does not). Registered here rather than in a second checker with its own +// manifest, npm script, ci.yml step and wiring tests — lessons-learned L35: when the existing thing +// already computes the answer, a parallel identity is the wrong remedy. Adding NO new wiring is the +// point; there is no new invoker to pin because there is no new invoker. +// +// The trigger (P7 — a real failure, never a hypothetical): the class recurred INSIDE #165, the +// increment whose entire purpose was fixing it — it corrected one "next increment" claim in +// pharn/pharn-contracts/eval-format.md and left a second in the same file. Per L20 that second +// occurrence is when a discipline-only remedy has earned a check. +// +// AN EMPTY `forward_claims` ARRAY IS EXIT 2 (EMPTY_CLASS), NEVER GREEN — lessons-learned L34: every +// check here is "for each registered claim, assert P", which is vacuously true over zero claims, so an +// emptied array would certify nothing. OMITTING the key is the expressible "none registered" state. +// // WHY A STRUCTURED MANIFEST (lessons-learned L6). Membership is read from specified-primitives.json — // never by scanning doc prose for what LOOKS like a marker. L6's defect (a membership fact grepped out // of free text, so documentation ABOUT a declaration counts as a declaration) recurred inside F7's own @@ -94,6 +113,29 @@ if (!Array.isArray(primitives)) { process.exit(2); } +// FORWARD-LOOKING CLAIMS — the same two directions, a different marker vocabulary. See the manifest's +// $forward_claims_comment for what is deliberately unregistered. +// +// The key is OPTIONAL (omitted => none registered, a real GREEN), but a PRESENT-and-EMPTY array is +// exit 2. Every check below is "for each registered claim, assert P", which is VACUOUSLY TRUE over +// zero claims (lessons-learned L34), so an emptied array would otherwise certify nothing as GREEN. +// Omission and emptiness are distinguishable, which is what keeps the honest "none yet" state +// expressible — the same shape as check-structural.mjs's `finding_count == 0` escape. +const hasForwardKey = Object.hasOwn(manifest, "forward_claims"); +const forward = hasForwardKey ? manifest.forward_claims : []; +if (hasForwardKey && !Array.isArray(forward)) { + console.error(`RED (manifest unusable): ${MANIFEST} has a \`forward_claims\` key that is not an array`); + process.exit(2); +} +if (hasForwardKey && forward.length === 0) { + console.error( + `RED (manifest unusable): ${MANIFEST} declares an EMPTY \`forward_claims\` array (EMPTY_CLASS).\n` + + ` Per-claim assertions pass vacuously over zero claims, so this would certify nothing as GREEN.\n` + + ` To register no forward claims, OMIT the \`forward_claims\` key entirely.` + ); + process.exit(2); +} + /** * Is the primitive LIVE? A pure existence test (ARCHITECTURE §2 primitive #3). * Unknown probe type => throw, so a typo in the manifest fails CLOSED (exit 2) rather than @@ -175,6 +217,8 @@ function readDoc(rel) { let siteCount = 0; let liveCount = 0; +let claimCount = 0; +let landedCount = 0; try { for (const p of primitives) { @@ -207,6 +251,36 @@ try { } } + for (const c of forward) { + validatePrimitive(c); // identical record shape: id + probe + sites[{file, marker}] + const landed = isLive(c.probe, c.id); + if (landed) landedCount++; + for (const site of c.sites) { + claimCount++; + const src = readDoc(site.file); + if (src === null) { + red(`${site.file}: listed as a site for forward claim \`${c.id}\` but the file could not be read`); + continue; + } + const present = src.includes(site.marker); + if (landed && present) { + // DIRECTION 1 — the claim EXPIRED: the artifact landed, the sentence still says it has not. + red( + `${site.file}: forward claim \`${c.id}\` has EXPIRED — ${describe(c.probe)}, but the doc still ` + + `says it is not built. RE-DERIVE the sentence; the doc now describes the repo as weaker than it is.\n` + + ` marker: ${JSON.stringify(site.marker)}` + ); + } else if (!landed && !present) { + // DIRECTION 2 — the hedge was deleted while the artifact is still absent. + red( + `${site.file}: the forward-claim hedge for \`${c.id}\` is GONE, but the artifact is still absent ` + + `(${describe(c.probe)}). The doc now implies something exists that does not.\n` + + ` expected: ${JSON.stringify(site.marker)}` + ); + } + } + } + for (const a of named) { validateNamedArtifact(a); const src = readDoc(a.cited_in); @@ -241,14 +315,17 @@ if (reds.length) { console.error(`SPECIFIED-MARKERS: RED — ${reds.length} drifted annotation(s)\n`); for (const r of reds) console.error(` - ${r}`); console.error( - `\nManifest: ${MANIFEST}\n` + `This checks only the annotations that manifest LISTS. It cannot discover a new overclaim (P0).` + `\nManifest: ${MANIFEST}\n` + + `This checks only the annotations and forward claims that manifest LISTS. It cannot discover a new ` + + `overclaim, or a forward-looking claim nobody registered (P0).` ); process.exit(1); } console.log( `SPECIFIED-MARKERS: GREEN — ${siteCount} annotation(s) across ${primitives.length} specified primitive(s) ` + - `(${liveCount} now live), ${named.length} named artifact(s) checked in ${TARGET}.\n` + - `NOTE (P0): this proves the LISTED annotations still match reality. It never means the docs are accurate — ` + - `an overclaim not in the manifest is invisible here.` + `(${liveCount} now live), ${claimCount} forward-claim site(s) across ${forward.length} registered claim(s) ` + + `(${landedCount} now landed), ${named.length} named artifact(s) checked in ${TARGET}.\n` + + `NOTE (P0): this proves the LISTED annotations and claims still match reality. It never means the docs are ` + + `accurate — an overclaim or an expired claim not in the manifest is invisible here.` ); diff --git a/.dev/floor/check-specified-markers.test.mjs b/.dev/floor/check-specified-markers.test.mjs index 43a6feb..e9c891c 100644 --- a/.dev/floor/check-specified-markers.test.mjs +++ b/.dev/floor/check-specified-markers.test.mjs @@ -259,3 +259,157 @@ test("the GREEN line states its own P0 bound (the manifest is not a discovery me assert.equal(r.code, 0, r.out); assert.match(r.out, /never means the docs are accurate/); }); + +// ------------------------------------------------------ forward-looking claims (the second class) +// +// Same two directions, different vocabulary: direction 1 is the claim EXPIRING (artifact landed, hedge +// remains), direction 2 is the hedge being DELETED while the artifact is still absent. L4 again — the +// RED paths are driven with real fixture trees, and the ✧ cases are mutants. + +const HEDGE = "no `pharn-eval` command exists"; + +/** + * Fixture for the forward-claim class. `opts.landed` creates the probed artifact; + * `opts.hedge === false` omits the hedge sentence from the doc. + * `opts.forward` overrides the manifest's forward_claims wholesale (including omitting it). + */ +function fwFixture(opts = {}) { + const dir = mkdtempSync(join(tmpdir(), "pharn-forward-")); + mkdirSync(join(dir, ".claude", "commands"), { recursive: true }); + if (opts.landed) writeFileSync(join(dir, ".claude", "commands", "pharn-eval.md"), "# stub\n"); + writeFileSync(join(dir, "DOC.md"), opts.hedge === false ? "intro\noutro\n" : `intro\n${HEDGE}\noutro\n`); + + const manifest = { specified_primitives: [] }; + if (!opts.omitForward) { + manifest.forward_claims = opts.forward ?? [ + { + id: "pharn-eval", + probe: { type: "path", path: ".claude/commands/pharn-eval.md" }, + sites: [{ file: "DOC.md", marker: HEDGE }], + }, + ]; + } + const mPath = join(dir, "manifest.json"); + writeFileSync(mPath, JSON.stringify(manifest)); + return { dir, mPath, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; +} + +test("GREEN — artifact absent and the hedge is present (the steady state)", () => { + const f = fwFixture(); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /GREEN/); +}); + +test("GREEN — artifact landed and the hedge was re-derived away (the healthy transition)", () => { + const f = fwFixture({ landed: true, hedge: false }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); +}); + +test("✧ DIRECTION 1 — the artifact LANDED and the hedge remains: the claim EXPIRED, RED", () => { + const f = fwFixture({ landed: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /EXPIRED/); + assert.match(r.out, /pharn-eval/); + assert.match(r.out, /DOC\.md/); +}); + +test("✧ DIRECTION 1 names the marker to re-derive, not merely the file", () => { + const f = fwFixture({ landed: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.match(r.out, /marker: /); + assert.match(r.out, /no `pharn-eval` command exists/); +}); + +test("✧ DIRECTION 2 — the hedge was DELETED while the artifact is still absent, RED", () => { + const f = fwFixture({ hedge: false }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /is GONE/); + assert.match(r.out, /implies something exists that does not/); +}); + +test("✧ L34 — a PRESENT but EMPTY forward_claims array is exit 2 (EMPTY_CLASS), never GREEN", () => { + const f = fwFixture({ forward: [] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /EMPTY_CLASS/); + assert.match(r.out, /vacuously/); +}); + +test("✧ L34 — the EMPTY_CLASS refusal names the escape (omit the key), so the honest state stays expressible", () => { + const f = fwFixture({ forward: [] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.match(r.out, /OMIT the `forward_claims` key/); +}); + +test("GREEN — OMITTING forward_claims entirely is the expressible 'none registered' state", () => { + const f = fwFixture({ omitForward: true }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 0, r.out); + assert.match(r.out, /0 registered claim/); +}); + +test("✧ a forward_claims key that is not an array fails CLOSED (exit 2)", () => { + const f = fwFixture({ forward: { id: "not-an-array" } }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /not an array/); +}); + +test("✧ a forward-claim site whose file cannot be read is RED, not a silent skip", () => { + const f = fwFixture({ + forward: [ + { + id: "pharn-eval", + probe: { type: "path", path: ".claude/commands/pharn-eval.md" }, + sites: [{ file: "NOPE.md", marker: HEDGE }], + }, + ], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 1, r.out); + assert.match(r.out, /could not be read/); +}); + +test("✧ a malformed forward-claim record fails CLOSED (exit 2), same validator as primitives", () => { + const f = fwFixture({ forward: [{ id: "pharn-eval", probe: { type: "path", path: "x" } }] }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /`sites` must be an array/); +}); + +test("✧ an unknown probe type on a forward claim fails CLOSED, never 'not landed'", () => { + const f = fwFixture({ + forward: [{ id: "pharn-eval", probe: { type: "vibes" }, sites: [{ file: "DOC.md", marker: HEDGE }] }], + }); + const r = run(f.dir, f.mPath); + f.cleanup(); + assert.equal(r.code, 2, r.out); + assert.match(r.out, /unknown probe type/); +}); + +test("integration — the REAL manifest registers at least one forward claim (the class stays live)", () => { + const r = run(REPO); + assert.equal(r.code, 0, r.out); + const m = r.out.match(/across (\d+) registered claim/); + assert.ok(m && Number(m[1]) >= 1, `expected >= 1 registered forward claim, got: ${r.out}`); +}); + +test("the GREEN line reports the forward-claim counts, so a silently emptied class is visible", () => { + const r = run(REPO); + assert.match(r.out, /forward-claim site\(s\) across \d+ registered claim\(s\)/); +}); diff --git a/.dev/floor/specified-primitives.json b/.dev/floor/specified-primitives.json index 39cdbe8..a326994 100644 --- a/.dev/floor/specified-primitives.json +++ b/.dev/floor/specified-primitives.json @@ -102,5 +102,54 @@ "citation": "(security\ngriller,", "must_exist": "pharn/pharn-pipeline/grillers/security" } + ], + + "$forward_claims_comment": [ + "FORWARD-LOOKING CLAIMS: sentences that are TRUE when written and become FALSE the moment the named", + "artifact lands. Computed by the SAME two directions as specified_primitives (a probe + an exact", + "substring), because the semantics are identical — an expired forward claim IS direction 1, the doc", + "understating reality. Registered here rather than in a second checker: a parallel manifest+checker", + "would be a second identity with its own wiring and invoker, which lessons-learned L35 says to", + "refuse when the existing one already computes the answer.", + "", + "WHAT IS DELIBERATELY NOT REGISTERED, so the absence does not read as an oversight:", + " * The ~20 P7 'future increment, added when a real need surfaces' statements across", + " pharn/pharn-review/* and the scan-code-* scanners. Those assert a POLICY (we do not build", + " speculatively), which stays true whether or not the thing is later built for a real reason.", + " They are conditional design postures, not pinned predictions; registering them would fire on", + " correct sentences and train the reader to ignore the check.", + " * The `live griller runner` class (13+ sites across every griller) and /pharn-verify's verifier", + " runner. Both are real expiring claims, but neither subject has a NAMED path in this repo, so a", + " probe would have to invent one. Deferred rather than guessed (P6). Reopens when the runner is", + " named in a plan or built, or when the first `role: verifier` capability is authored.", + "", + "An EMPTY forward_claims array is exit 2 (EMPTY_CLASS), never GREEN — per-claim assertions are", + "vacuously true over zero claims (lessons-learned L34). To express 'none registered', OMIT the key." + ], + + "forward_claims": [ + { + "id": "pharn-eval", + "what": "the product /pharn-eval twin of /pharn-dev-eval — DEFERRED 2026-08-23, reopening when the first role:-bearing capability is authored outside PHARN's own shipped surface", + "probe": { "type": "path", "path": ".claude/commands/pharn-eval.md" }, + "sites": [ + { + "file": "CLAUDE.md", + "marker": "no `pharn-eval` command exists" + } + ] + }, + + { + "id": "product-capability-catalog", + "what": "a generated capability catalog on the PRODUCT surface — DEFERRED 2026-08-07, reopening on the same trigger", + "probe": { "type": "path", "path": "pharn/floor/gen-capability-catalog.mjs" }, + "sites": [ + { + "file": "CLAUDE.md", + "marker": "nothing under `pharn/floor/` renders one" + } + ] + } ] }