From 30fc8c1b85c827cfa292bc3f4d631c19160603b2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Przemys=C5=82aw=20Galarowicz?= Date: Sun, 23 Aug 2026 02:54:43 +0200 Subject: [PATCH] docs: re-derive stale claims against the live tree (2.7.14) Shipped prose still called landed floor runners "not yet built"; patch bump restores each confirmed-stale sentence from the live tree (L25) and keeps the honest write-time bounds. Co-authored-by: Cursor --- .dev/features/README.md | 4 +- .dev/features/docs-drift-resync/PLAN.md | 58 +++++++++++++++++++ .github/ISSUE_TEMPLATE/bug_report.md | 4 +- .github/PULL_REQUEST_TEMPLATE.md | 6 +- CHANGELOG.md | 1 + CLAUDE.md | 16 +++-- CONTRIBUTING.md | 2 +- README.md | 4 +- SECURITY.md | 2 +- SKILLS_VERSION | 2 +- pharn/floor/README.md | 14 +++-- pharn/pharn-contracts/eval-format.md | 19 +++--- pharn/pharn-contracts/finding-shape.md | 34 ++++++----- .../hallucinated-api/hallucinated-api.md | 9 +-- .../input-validation/input-validation.md | 9 +-- 15 files changed, 128 insertions(+), 56 deletions(-) create mode 100644 .dev/features/docs-drift-resync/PLAN.md diff --git a/.dev/features/README.md b/.dev/features/README.md index a5ae1b9..5c31e0b 100644 --- a/.dev/features/README.md +++ b/.dev/features/README.md @@ -9,8 +9,8 @@ holding its **process and audit artifacts** from the build loop (the `pharn-dev- These record _how_ an increment of PHARN was specified, planned, grilled, built, and reviewed. This is the **developer / contributor** side of the dev/product boundary — the apparatus a PHARN contributor -uses, **not** what a PHARN user receives. The built capabilities themselves live in their modules at the -repo root (`pharn-contracts/`, `pharn-review/`, …); the PRODUCT pipeline's artifacts live in the +uses, **not** what a PHARN user receives. The built capabilities themselves live in their modules under +`pharn/` (`pharn/pharn-contracts/`, `pharn/pharn-review/`, …); the PRODUCT pipeline's artifacts live in the **root-level `features/`** (see `../../features/README.md`). Artifacts are written only when they genuinely exist. An increment that had no SPEC or PLAN — e.g. a diff --git a/.dev/features/docs-drift-resync/PLAN.md b/.dev/features/docs-drift-resync/PLAN.md new file mode 100644 index 0000000..b4cad51 --- /dev/null +++ b/.dev/features/docs-drift-resync/PLAN.md @@ -0,0 +1,58 @@ +# PLAN — docs-drift-resync: re-derive stale doc claims across the shipped and meta surfaces + +- spec_content_hash: 8f5ec002e3b18cbfd2f094b08a3671f7ed42a05a3fbaf01a11bbbd28da30fb52 # fix #4 +- applied_lessons: [L25] +- increment: Correct every doc sentence a full-tree audit CONFIRMED stale against the live repo — no reframing, no new claims; each edit restores a sentence to what the tree already proves. +- layer(s): none — doc corrections across contracts, floor README, two lenses, and repo-meta # pharn/ARCHITECTURE.md §4 +- constitution_refs: [P0, P6] + +## Applied lessons + +- **L25** — every target sentence here is exactly L25's defect at doc scale: prose written when a + state was true (checker unbuilt, path pre-relocation, gate list shorter) and trusted after the + state moved. Each fix RE-DERIVES the sentence from the live tree rather than patching the visible + symptom: the "3c runner not yet built" sites are rewritten to name the runners that now invoke + `check-structural.mjs` (not merely to delete "not yet"), and the honest bounds they carried + (nothing fires at write time) are preserved rather than dropped. + +## What this is + +An audited resync, run at the maintainer's direct request ("analyze, update README/CLAUDE.md, check +if docs need updating"). A six-agent audit verified every falsifiable doc claim against the live +tree; this increment applies only the CONFIRMED-stale corrections that are framing-independent. +It deliberately does NOT touch the four trusted docs (hook-denied, human-only — their issues are +reported to the human instead) and does NOT change the repo's adoptability framing. + +## Scope of correction (why each file) + +- Shipped surface (bumps SKILLS_VERSION, patch): `eval-format.md` still calls `check-structural.mjs` + "the next increment"; `finding-shape.md` still calls the 3c runner "not yet built" and cites + pre-relocation `floor/…` paths plus a pre-move `features/trust-fence` example; + `pharn/floor/README.md` says "nothing in the build loop invokes it automatically yet" (stale — the + verify/eval stages do) and understates the hook matcher; the two lenses repeat the "3c not yet + wired" bound. +- Repo-meta (no bump): CLAUDE.md (7-gate list vs the live 8; "21 tagged" vs 32; the four-constants + enumeration omits `GEN`; a present-tense `pharn` CLI that does not exist); README.md line 60's + "authorization checked" inside a "can guarantee" sentence (the repo's own security griller records + that floor candidate as REJECTED); SECURITY.md pointing at the deliberately inert `package.json` + version; CONTRIBUTING.md's `pharn/floor/` bump sentence missing the `*.test.mjs` carve-out; both + GitHub templates citing pre-relocation `floor/validate.mjs` and pre-rename `/plan`-style commands; + `.dev/features/README.md` placing modules at the repo root. + +## Files + +- `.dev/features/docs-drift-resync/PLAN.md` — this record +- `README.md` — line-60 guarantee triple; version badge (bump) +- `CLAUDE.md` — gate list, tagged count, four-constants enumeration, planned-CLI tense +- `SECURITY.md` — version-of-record sentence +- `CONTRIBUTING.md` — `pharn/floor/` bump sentence test-file carve-out +- `.github/PULL_REQUEST_TEMPLATE.md` — gate list, floor path, trusted-doc paths +- `.github/ISSUE_TEMPLATE/bug_report.md` — command names, floor path, module path +- `.dev/features/README.md` — module locations under `pharn/` +- `pharn/pharn-contracts/eval-format.md` — checker landed; `pharn/floor/` spellings +- `pharn/pharn-contracts/finding-shape.md` — 3c landed; path spellings; trust-fence example path +- `pharn/floor/README.md` — invokers named; hook matcher +- `pharn/pharn-review/input-validation/input-validation.md` — 3c bound re-derived +- `pharn/pharn-review/hallucinated-api/hallucinated-api.md` — 3c bound re-derived +- `SKILLS_VERSION` — 2.7.13 → 2.7.14 (patch: corrections to already-shipped bytes) +- `CHANGELOG.md` — the matching entry diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 67095f6..291dfa0 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -12,7 +12,7 @@ A clear description of the bug. ## Which part of PHARN -- **Artifact**: +- **Artifact**: - **PHARN version** (`SKILLS_VERSION`): ## Steps to reproduce @@ -37,4 +37,4 @@ What actually happened. Paste the relevant Claude Code output (redact any secret ## Additional context -Anything else that helps — the relevant files, and the floor output (`node floor/validate.mjs .`). +Anything else that helps — the relevant files, and the floor output (`node pharn/floor/validate.mjs .`). diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 3df3a9e..0fdff86 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -18,7 +18,7 @@ Closes # ## Checklist -- [ ] I read the four trusted docs (`CONSTITUTION.md` → `ARCHITECTURE.md` → `THREAT-MODEL.md` → `LIMITS.md`) and did **not** edit them — they are human-only and hook-protected. +- [ ] I read the four trusted docs (`pharn/CONSTITUTION.md` → `pharn/ARCHITECTURE.md` → `THREAT-MODEL.md` → `LIMITS.md`) and did **not** edit them — they are human-only and hook-protected. - [ ] No constitutional principle (P0–P7) is violated. - [ ] If this adds a Capability: it has a `role:` and ships evals (`evals/cases/` + `evals/expected/`), and every `enforces` rule_id is produced by ≥1 eval case (P1). - [ ] Findings cite a file-qualified `rule_id`; no guarantee is claimed without a floor reduction, otherwise it is labeled `advisory` (P0). @@ -26,8 +26,8 @@ Closes # ## Quality gates -- [ ] `npm run check` passes locally (`format:check` + `lint` + `lint:md` + `test`). -- [ ] `node floor/validate.mjs .` is GREEN. +- [ ] `npm run check` passes locally (every gate `package.json`'s `scripts.check` chains — the authoritative list). +- [ ] `node pharn/floor/validate.mjs .` is GREEN. ## Notes for the reviewer diff --git a/CHANGELOG.md b/CHANGELOG.md index f05b550..427dad7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,6 +46,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Fixed +- **Shipped prose that still described landed floor machinery as unbuilt, re-derived against the live tree** (`SKILLS_VERSION` 2.7.13 → **2.7.14**, patch). A full-tree claim audit (every falsifiable doc sentence checked against the live repo) found the shipped surface carrying "not yet built" claims that had been true once and false since: [`pharn/pharn-contracts/eval-format.md`](./pharn/pharn-contracts/eval-format.md) still called the `structural[]` checker "the **next increment**" (it landed as `pharn/floor/check-structural.mjs`); [`pharn/pharn-contracts/finding-shape.md`](./pharn/pharn-contracts/finding-shape.md) still said "no runner yet invokes it … increment **3c, not yet built**" (3c landed as `/pharn-dev-eval` via `.dev/floor/check-variance.mjs`, and `/pharn-verify` / `/pharn-dev-verify` invoke the checker per committed eval pair); [`pharn/floor/README.md`](./pharn/floor/README.md) said "nothing in the build loop invokes it automatically yet" and understated the hook matcher (the live wiring is `Write|Edit|MultiEdit|NotebookEdit`); the `input-validation` and `hallucinated-api` lenses repeated the "3c not yet wired" bound. Each site was **re-derived, not deleted** (L25): the bound that survives — nothing fires at write time; the enforcement moment is the verify/eval stage — is kept, and the invokers that now exist are named. The same pass corrected pre-relocation `floor/…` path spellings and a pre-move `features/trust-fence` example inside the shipped contracts. Repo-meta corrections rode along without bumping: `CLAUDE.md`'s seven-gate aggregate list vs the live eight (`check:contributing` was missing — the exact drift class `check-contributing-gates` guards, one file over), its stale "21 tagged" count (now count-free per P6), its four-constants enumeration (the pinned set is `{CANON_PATH, OUT_PATH, GEN, REGEN}`; the absent-canon semantics are pinned separately as divergent functions), and its present-tense `pharn` CLI (specified, not built); `README.md`'s "authorization checked" inside a "can guarantee" sentence (the security griller records that floor candidate as **REJECTED**, so the sentence now names the writes-scope guarantee instead); `SECURITY.md` pointing at the deliberately inert `package.json` version; `CONTRIBUTING.md`'s missing test-file carve-out on the `pharn/floor/` bump rule; both GitHub templates' pre-relocation `floor/validate.mjs` paths, the PR template's four-of-eight gate list, and the bug template's pre-rename `/plan`-style command names; `.dev/features/README.md`'s modules-at-repo-root claim. Audit + scope record: `.dev/features/docs-drift-resync/PLAN.md`. - **Six confirmed product-floor defects, each reproduced live before it was scoped and each now pinned by a test that fails without its fix** (`SKILLS_VERSION` 2.7.12 → 2.7.13). None was hypothetical; all six came from an external review and were re-derived against the live tree, which mattered — one of the six reports carried a reproduce command that did not reproduce the defect it described. - **A prototype-walking gate comparison let a real gate-set mismatch pass silently** ([`pharn/floor/check-regress.mjs`](./pharn/floor/check-regress.mjs)). The mismatch check used `k in obj`, which walks the prototype chain, so a gate id colliding with an `Object.prototype` member (`toString`, `valueOf`) read as PRESENT in a map that did not have it. The extra failing gate was treated as shared and the verdict came back `no-regressions` at exit 0, where a gate-set mismatch is contractually `inconclusive`/exit 2 — a silent pass inside the checker whose entire promise is that there is never one. Now `Object.hasOwn`, which is the own-property test `lessons-learned` **L15** already prescribes; this is that lesson recurring. - **A UTF-8 BOM defeated every frontmatter anchor in the floor** — and the fix is a new shared core, [`pharn/floor/frontmatter-core.mjs`](./pharn/floor/frontmatter-core.mjs), because the defect was a **set** problem. `FM_RE` had been copy-pasted byte-identically into **six** checkers (`check-spec`, `check-loop-record`, `check-plan-lessons`, `check-plan-spec-agree`, `check-ship-briefing`, `render-ship-briefing`) with nothing ranging over them, so a byte-valid file written by a BOM-emitting editor RED'd with "no YAML frontmatter block" in all six, and fixing whichever one surfaced would have left five broken with no test able to tell. That is **L31** exactly. The anchor now has one definition, the BOM strip happens at **read** (beside the existing CRLF fold, so the two input-normalisation defences live together — **L25**), and a consumer-set pin asserts no checker re-declares its own anchor, imports the core, and actually calls `stripBom`. **Narrowed:** exactly one leading `U+FEFF` is stripped, only at offset 0 — a doubled BOM stays malformed, and a genuinely frontmatter-less file still REDs. The fix removes a FALSE red; it creates no path to a false GREEN. diff --git a/CLAUDE.md b/CLAUDE.md index a3d808b..0b382c3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -44,8 +44,9 @@ Packaging later = "ship root minus `.dev/`". `.dev/` (committed apparatus) is un ## SKILLS_VERSION discipline (versioning the shipped surface) `SKILLS_VERSION` (repo root) versions the **product surface** — the bytes a PHARN user receives (the -`pharn` CLI installs them; `pharn status` / `pharn update` compare a user's install against this -file). It does **not** version the build apparatus. +planned `pharn` CLI — specified, **not built**; no installer exists yet — would install them, and its +`pharn status` / `pharn update` would compare a user's install against this file). It does **not** +version the build apparatus. - **Any change that alters product-surface bytes MUST bump `SKILLS_VERSION` and add a `CHANGELOG.md` entry — prose-only edits included.** A clarified `/pharn-*` command step, a reworded contract, or a @@ -278,7 +279,8 @@ echo '{"tool_name":"Write","tool_input":{"file_path":"pharn/pharn-core/rules/x.m - **Dev tooling is real; the methodology stays stdlib-only.** The floor, the hook, and the commands have **zero runtime dependencies** (Node stdlib; Node 24). The repo carries **dev-only** devDependencies (ESLint, Prettier, markdownlint) wired as npm scripts: `npm run check` - (`format:check` + `lint` + `lint:md` + `docs:check` + `check:markers` + `check:badge` + `test`) is the + (`format:check` + `lint` + `lint:md` + `docs:check` + `check:markers` + `check:badge` + + `check:contributing` + `test`) is the aggregate gate, and `npm test` runs `node --test` over the hook, product-floor, and dev-floor suites (`.claude/hooks/*.test.cjs` + `pharn/floor/*.test.mjs` + `.dev/floor/*.test.mjs`) — **green** at this writing; read the count live @@ -501,7 +503,8 @@ framework-specific`), via the first-match-wins procedure in `pharn/ARCHITECTURE. never against the index. The index's `type` / `concepts` columns are model-drafted values a human ratified at the promote gate, so **"typed `floor`" never means "about the floor"** — selecting on them is advisory context selection. **Every dev canon entry is now tagged** — the legacy L1–L17 were - retro-tagged, so the index renders `21 tagged · 0 malformed · 0 untagged` and **both** absence markers + retro-tagged, so the index renders every entry tagged (`0 malformed · 0 untagged`; read the live + counts from `docs/lessons-index.md`, never from this doc — P6) and **both** absence markers are now unexpected: a `-` means no tag line, i.e. an entry that reached canon without the promote gate's `type`/`concepts`; a `?` means a tag line is present but **failed its gate**. Read that entry in canon and flag it either way. (The PRODUCT twin keeps the benign reading of `-` on purpose — a user's @@ -523,8 +526,9 @@ framework-specific`), via the first-match-wins procedure in `pharn/ARCHITECTURE. advisory: skipping it just yields a `STALE` the next plan degrades on. - **The two cores are deliberate SEPARATE COPIES** (`pharn/floor/lessons-index-core.mjs` vs `.dev/floor/lessons-index-core.mjs`), the `check-provenance.mjs` precedent. Four constants diverge on - purpose — `CANON_PATH`, `OUT_PATH`, `REGEN`, and the **absent/empty-canon semantics** (the product - copy treats no-canon as a benign no-op where the dev copy throws). ✧ tests in + purpose — `CANON_PATH`, `OUT_PATH`, `GEN`, `REGEN` — as does the **absent/empty-canon semantics** + (a divergent function, pinned separately: the product copy treats no-canon as a benign no-op where + the dev copy throws). ✧ tests in `.dev/floor/lessons-index-core.test.mjs` pin **both** halves: every shared constant must AGREE and those four must DIFFER. The pin lives on the dev side because a user's install ships `pharn/floor/` **without** `.dev/`, so the dependency may only point `.dev/` → `pharn/`; the honest consequence is diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 276e55f..0c13fad 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -63,7 +63,7 @@ When you add a PHARN capability, follow the conventions in [`CLAUDE.md`](./CLAUD The repo separates the **product** (what a user receives) from the **build apparatus** (what a contributor uses), in the filesystem and in command names: - **`.dev/`** holds the apparatus — `.dev/floor/` (dev-only checkers + tests), `.dev/features/` (build-loop audit trails), `.dev/memory-bank/`. It is committed but is **not** what a user receives. -- **The product lives under `pharn/`** — `pharn/pharn-contracts/`, `pharn/pharn-core/`, `pharn/pharn-pipeline/`, `pharn/pharn-review/`, and the **product floor** `pharn/floor/` (the checkers the `/pharn-*` commands run on a user's code) — plus a root `features/` for product-pipeline artifacts. `pharn/floor/validate.mjs` **is** that product floor, and it excludes `.dev/**` wholesale: it scans the product surface only. So a change under `pharn/floor/` **ships** and is not apparatus — which means it must bump [`SKILLS_VERSION`](./SKILLS_VERSION) and add a `CHANGELOG.md` entry; see [`CLAUDE.md`](./CLAUDE.md) ("SKILLS_VERSION discipline") for what counts as product surface and how big the bump is. +- **The product lives under `pharn/`** — `pharn/pharn-contracts/`, `pharn/pharn-core/`, `pharn/pharn-pipeline/`, `pharn/pharn-review/`, and the **product floor** `pharn/floor/` (the checkers the `/pharn-*` commands run on a user's code) — plus a root `features/` for product-pipeline artifacts. `pharn/floor/validate.mjs` **is** that product floor, and it excludes `.dev/**` wholesale: it scans the product surface only. So a change under `pharn/floor/` **ships** and is not apparatus — which means it must bump [`SKILLS_VERSION`](./SKILLS_VERSION) and add a `CHANGELOG.md` entry (the one exception: the checkers' `*.test.mjs` files never ship, so a test-only change does not bump); see [`CLAUDE.md`](./CLAUDE.md) ("SKILLS_VERSION discipline") for what counts as product surface and how big the bump is. - **Two floors exist on purpose:** `.dev/floor/` is contributor tooling that never ships; `pharn/floor/` ships. [`CLAUDE.md`](./CLAUDE.md) ("Repo layout — the dev/product boundary") is the authoritative split — defer to it rather than to this summary. - **Commands split by name prefix** (they cannot move out of `.claude/`): build-apparatus commands are **`pharn-dev-*`** (`pharn-dev-plan`, `-build`, …); product commands are **`pharn-*`** without `-dev-`. The prefix is naming/UX only — **not** an access gate. diff --git a/README.md b/README.md index 9f7f804..f21fcaa 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ runs on Claude Code today, and the discipline itself ships as readable markdown lenses, rules — that you read, diff, and version yourself. PHARN does not make anyone understand the code; it keeps a deterministic floor under it and the record available the moment someone needs it. -[![pharn](https://img.shields.io/badge/pharn-2.7.13-blue)](./CHANGELOG.md) +[![pharn](https://img.shields.io/badge/pharn-2.7.14-blue)](./CHANGELOG.md) [![License: Apache 2.0](https://img.shields.io/badge/license-Apache%202.0-green)](./LICENSE) [![CI](https://github.com/pharn-dev/pharn-oss/actions/workflows/ci.yml/badge.svg)](https://github.com/pharn-dev/pharn-oss/actions/workflows/ci.yml) [![CodeQL](https://github.com/pharn-dev/pharn-oss/actions/workflows/codeql.yml/badge.svg)](https://github.com/pharn-dev/pharn-oss/actions/workflows/codeql.yml) @@ -57,7 +57,7 @@ This isn't hypothetical. A [2026 Anthropic RCT](https://www.anthropic.com/resear measured developers scoring ~17% lower on comprehension of code they shipped with AI assistance, even as the volume of AI-generated code keeps climbing — more code, understood less. -PHARN doesn't force you to understand the code — nothing can, and anything that tries just gets bypassed. Instead it does two things it can guarantee: a deterministic floor that holds without you (secrets blocked, authorization checked, the plan actually built), and a markdown-canonical record — spec, constitution, diff, audit trail — kept in your repo, readable and diffable, available the moment you need it. The agent does the typing. PHARN keeps the result legible for whoever reads it next — if anyone does. +PHARN doesn't force you to understand the code — nothing can, and anything that tries just gets bypassed. Instead it does two things it can guarantee: a deterministic floor that holds without you (secrets screened at the plan gate, every write confined to its declared scope, the plan actually built), and a markdown-canonical record — spec, constitution, diff, audit trail — kept in your repo, readable and diffable, available the moment you need it. The agent does the typing. PHARN keeps the result legible for whoever reads it next — if anyone does. > **Your chat history is gone. Your spec isn't.** diff --git a/SECURITY.md b/SECURITY.md index e6bca6b..60f7bec 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -10,7 +10,7 @@ PHARN's security model (`THREAT-MODEL.md`, threat model B) starts from one axiom ## Supported versions -We patch security issues against the **latest** released version only. The current version is recorded in [`SKILLS_VERSION`](./SKILLS_VERSION) and `package.json`. +We patch security issues against the **latest** released version only. The current version is recorded in [`SKILLS_VERSION`](./SKILLS_VERSION) — the one version of record (`package.json`'s `version` is deliberately inert; see the [`CHANGELOG.md`](./CHANGELOG.md) preamble). | Version | Supported | | -------- | ------------------ | diff --git a/SKILLS_VERSION b/SKILLS_VERSION index ecc17b8..9bbf492 100644 --- a/SKILLS_VERSION +++ b/SKILLS_VERSION @@ -1 +1 @@ -2.7.13 +2.7.14 diff --git a/pharn/floor/README.md b/pharn/floor/README.md index 7d1b010..98f4304 100644 --- a/pharn/floor/README.md +++ b/pharn/floor/README.md @@ -67,11 +67,13 @@ run the skill; it checks an output the skill already produced. **What this changes (P0).** Before, `eval-format.md` labeled `structural[]` **floor-reducible-but-not-yet-enforced** and named this checker as the backstop. With it landed, -`structural[]` is **floor-executable and CI-tested** (but not yet auto-enforced — `/pharn-dev-build` does not -invoke `check-structural.mjs`, and its live-eval wiring is still deferred): when the checker is run, -if a model laundered an untrusted needle (e.g. `skip authz`) into an enum-gated field, or routed a -`deterministic` skill's judgment through `semantic[]`, that is a deterministic **RED**, not a hope — -but you must run it; nothing in the build loop invokes it automatically yet. +`structural[]` is **floor-executable and CI-tested**, and the invokers now exist at the verify/eval +stages — `/pharn-verify` and `/pharn-dev-verify` run it per committed eval pair, and `/pharn-dev-eval` +runs it over each live-emitted run via `.dev/floor/check-variance.mjs` (`/pharn-dev-build` still does +**not** invoke it): when the checker is run, if a model laundered an untrusted needle (e.g. `skip +authz`) into an enum-gated field, or routed a `deterministic` skill's judgment through `semantic[]`, +that is a deterministic **RED**, not a hope — the enforcement moment is the verify/eval stage, never +the write. **Honest scope (P0) — the boundary that keeps this from overselling.** The checker enforces `structural[]` **over a provided finding output**. It does **not** run the skill and does **not** @@ -83,7 +85,7 @@ never evaluates a `judge` string (no LLM). ## Wire the write-guard hooks Two `PreToolUse` hooks are wired in `.claude/settings.json` (committed), both on -`Write|Edit|MultiEdit`; a deny from **either** blocks. **`protect-trusted-paths.cjs` (fix #2)** blocks +`Write|Edit|MultiEdit|NotebookEdit`; a deny from **either** blocks. **`protect-trusted-paths.cjs` (fix #2)** blocks any write to a protected path. Paths are matched **repo-relative and exact**, case-folded, against the guard's own location — never by bare basename, so a user's own `docs/ARCHITECTURE.md` stays writable. The default set is the four trusted spec docs (`pharn/CONSTITUTION.md`, `pharn/ARCHITECTURE.md`, diff --git a/pharn/pharn-contracts/eval-format.md b/pharn/pharn-contracts/eval-format.md index 9e49a3b..1b53ecc 100644 --- a/pharn/pharn-contracts/eval-format.md +++ b/pharn/pharn-contracts/eval-format.md @@ -136,17 +136,20 @@ v0.80 routed all eight through one LLM judge. Honest scope, because the disease this repo exists to prevent is "written in the contract" mistaken for "therefore guaranteed": -- **`structural[]` is floor-reducible, not yet floor-enforced.** Each kind maps to an enum / regex / - path-resolution primitive (table above). But the deterministic checker that _executes_ the reduction - is the **next increment**. Until it lands, `structural[]` assertions are - **floor-reducible-but-not-yet-enforced** — this contract must **not** call them "guaranteed" today. - The named floor backstop is that next-increment checker. +- **`structural[]` is floor-executable — the checker has landed.** Each kind maps to an enum / regex / + path-resolution primitive (table above), and the deterministic checker that _executes_ the reduction + is **`pharn/floor/check-structural.mjs`** (see `pharn/floor/README.md`). Run against a finding + output, a violated assertion is a deterministic **RED**. **Still bounded (P0):** it fires only when + invoked — `/pharn-verify` / `/pharn-dev-verify` run it per committed eval pair, `/pharn-dev-eval` + per live run — and **nothing invokes it at write time**, so "floor-executable" never means + "always-on guarantee". - **`semantic[]` is advisory** — an LLM judge, no floor reduction. It never alone gates a guaranteed decision (`pharn/ARCHITECTURE.md §8`). - **`skill_kind` partitions evaluation by enum membership** (`{deterministic, llm, llm-judge}`) — - floor-reducible (enum check), enforced when the checker lands. -- **The only floor guarantee about this contract _today_** is that the file passes - `floor/validate.mjs` — including CHECK 5, by documenting the enum-gated vs free-text split here. + floor-reducible (enum check), executed by `check-structural.mjs`: a `deterministic` skill with a + non-empty `semantic[]` is a RED. +- **The floor guarantee about this contract file _itself_** is that it passes + `pharn/floor/validate.mjs` — including CHECK 5, by documenting the enum-gated vs free-text split here. ## Trust class (P2) diff --git a/pharn/pharn-contracts/finding-shape.md b/pharn/pharn-contracts/finding-shape.md index d8fec74..ec0b504 100644 --- a/pharn/pharn-contracts/finding-shape.md +++ b/pharn/pharn-contracts/finding-shape.md @@ -56,9 +56,9 @@ the finding object defined above (zero or more) — **in addition to** any human split is a real JSON field boundary at the capability's output, it is **structural**, not something a downstream model re-interprets from prose. - **Naming/location:** the file is named `findings.json` and is colocated with the capability's - human-facing output (the same directory it `writes:`). Example: `trust-fence` writes - `features/trust-fence/REVIEW.md`, so its findings array is `features/trust-fence/findings.json`. -- **Consumer (cited):** this is exactly the `actual.json` that `floor/check-structural.mjs` reads; the + human-facing output (the same directory it `writes:`). Example: `trust-fence`'s dogfood run wrote + `.dev/features/trust-fence/REVIEW.md`, so its findings array is `.dev/features/trust-fence/findings.json`. +- **Consumer (cited):** this is exactly the `actual.json` that `pharn/floor/check-structural.mjs` reads; the emission contract is what gives that checker a real capability output to range over. ### Emission enforcement audit (P0) — what is and is NOT floor-backed @@ -71,21 +71,23 @@ for "therefore guaranteed." The `MUST` above is a **three-way split**, not one b fix #7) pins the **path** — a hook reduction (`pharn/ARCHITECTURE.md §2`): the Capability may write its findings array only where it declared, nowhere else. - **Emitting it at all → advisory.** Nothing on the floor forces a Capability to declare or write - `findings.json` — `floor/validate.mjs` does not check for it, and no hook fabricates an emission. So + `findings.json` — `pharn/floor/validate.mjs` does not check for it, and no hook fabricates an emission. So `MUST emit` is a **conformance requirement** on a conforming Capability, **not** a floor-guaranteed production. Labeled plainly per P0, lest the `MUST` read as a guarantee that the artifact exists. -- **Shape + the no-laundering trip-wire → floor-checkable at eval time, once the 3c runner lands (not - at write time).** The array's structure, and the rule that no `trust: untrusted` needle reaches an - enum-gated field, reduce to `floor/check-structural.mjs` (enum / regex-substring / path-resolution — - `pharn/ARCHITECTURE.md §2`). That **primitive exists and is tested today**, but **no runner yet invokes it - over a capability's emitted `findings.json`** — that wiring is increment **3c, not yet built**. So - this facet is **floor-reducible-but-not-yet-enforced**: honest today, it must **not** be read as an - always-on guarantee until 3c lands. - -So: **declaring** the path is guaranteed _today_ (fix #7, live); **shape/laundering** is -floor-reducible but **not enforced until the 3c runner lands** (`check-structural.mjs` exists and is -tested; the wiring over emitted output is unbuilt); **emitting at all** is an advisory conformance -requirement. No reading of the `MUST` above is a blanket floor guarantee that the artifact exists. +- **Shape + the no-laundering trip-wire → floor-checked at eval time (never at write time).** The + array's structure, and the rule that no `trust: untrusted` needle reaches an enum-gated field, + reduce to `pharn/floor/check-structural.mjs` (enum / regex-substring / path-resolution — + `pharn/ARCHITECTURE.md §2`). That primitive exists and is tested, and the runners that invoke it + over emitted output have landed: `/pharn-verify` / `/pharn-dev-verify` run it per committed eval + pair, and the dev-side `/pharn-dev-eval` (increment 3c) runs it over each live-emitted + `runs//findings.json` via `.dev/floor/check-variance.mjs`. **Still bounded (P0):** it fires when + those stages run — nothing checks at write time, so this facet is eval-time enforcement, never an + always-on guarantee. + +So: **declaring** the path is guaranteed _today_ (fix #7, live); **shape/laundering** is floor-checked +**at eval time, when a verify/eval stage runs** (`check-structural.mjs` and its invokers exist; +nothing checks at write time); **emitting at all** is an advisory conformance requirement. No reading +of the `MUST` above is a blanket floor guarantee that the artifact exists. ## The rule of the contract (P0, P2) diff --git a/pharn/pharn-review/hallucinated-api/hallucinated-api.md b/pharn/pharn-review/hallucinated-api/hallucinated-api.md index 73c7945..a7a61de 100644 --- a/pharn/pharn-review/hallucinated-api/hallucinated-api.md +++ b/pharn/pharn-review/hallucinated-api/hallucinated-api.md @@ -164,10 +164,11 @@ input reaches an enum-gated field). That the lens **emits** it at all, and emits - `needle_absent_from_enum_gated` + `file_resolves`) is floor-CHECKED at **eval time** by `pharn/floor/check-structural.mjs` (primitive #3, exit 1 on RED / 0 on GREEN). It pins behavior on known inputs and proves the needle cannot be laundered into an enum-gated field. **Honestly bounded (P0):** the _automated_ - runner over a live-emitted `findings.json` is increment **3c** (not yet wired — `finding-shape.md` §Emission); - today the trip-wire is realized when `check-structural.mjs` is run against the committed expected + actual - (e.g. at `/pharn-dev-verify`). It is **NOT** a runtime guarantee that "the API exists" is deterministic - (mirrors `trust-fence` / `input-validation` exactly). + runner over a live-emitted `findings.json` is the dev-side `/pharn-dev-eval` (increment **3c** — landed; no + product twin ships, a recorded deferral — `finding-shape.md` §Emission); otherwise the trip-wire is realized + when `check-structural.mjs` is run against the committed expected + actual + (e.g. at `/pharn-verify` / `/pharn-dev-verify`). It is **NOT** a runtime guarantee that "the API exists" is + deterministic (mirrors `trust-fence` / `input-validation` exactly). - **Fixture behavior** → the finding OUTPUT on the committed fixtures is floor-CHECKED at eval time by `check-structural.mjs` (primitive #3). On the advisory `llm` lens, `finding_count` pins the **expected output of a model judgment**, not a deterministic API-existence computation — the model's conformance is advisory (no diff --git a/pharn/pharn-review/input-validation/input-validation.md b/pharn/pharn-review/input-validation/input-validation.md index 562a7d0..3334ee2 100644 --- a/pharn/pharn-review/input-validation/input-validation.md +++ b/pharn/pharn-review/input-validation/input-validation.md @@ -148,10 +148,11 @@ input reaches an enum-gated field). That the lens **emits** it at all, and emits - `needle_absent_from_enum_gated` + `file_resolves`) is floor-CHECKED at **eval time** by `pharn/floor/check-structural.mjs` (primitive #3, exit 1 on RED / 0 on GREEN). It pins behavior on known inputs and proves the needle cannot be laundered into an enum-gated field. **Honestly bounded (P0):** the _automated_ - runner over a live-emitted `findings.json` is increment **3c** (not yet wired — `finding-shape.md` §Emission); - today the trip-wire is realized when `check-structural.mjs` is run against the committed expected + actual - (e.g. at `/pharn-dev-verify`). It is **NOT** a runtime guarantee that "input is validated" is deterministic - (mirrors `trust-fence` / `injection` exactly). + runner over a live-emitted `findings.json` is the dev-side `/pharn-dev-eval` (increment **3c** — landed; no + product twin ships, a recorded deferral — `finding-shape.md` §Emission); otherwise the trip-wire is realized + when `check-structural.mjs` is run against the committed expected + actual + (e.g. at `/pharn-verify` / `/pharn-dev-verify`). It is **NOT** a runtime guarantee that "input is validated" is + deterministic (mirrors `trust-fence` / `injection` exactly). - **Fixture behavior** → the finding OUTPUT on the committed fixtures is floor-CHECKED at eval time by `check-structural.mjs` (primitive #3). It pins behavior on known inputs and proves the trust-fence holds — it is **NOT** a runtime guarantee that a clean read means "validated". The **clean** fixture's `finding_count ==