Skip to content

The registry generator: a tested mechanism for minting instances - #49

Merged
Hafeok merged 9 commits into
mainfrom
claude/registry-generator-mechanism-wf1dsr
Aug 18, 2026
Merged

The registry generator: a tested mechanism for minting instances#49
Hafeok merged 9 commits into
mainfrom
claude/registry-generator-mechanism-wf1dsr

Conversation

@Hafeok

@Hafeok Hafeok commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Status: the generator is the mechanism; G0 regenerates through it.

Four gates, all ratified: design (cdb571c), implementation (c30a885), template changes (23f8e19), close (a7936d6). Full report: docs/g-track/registry-generator-session-2026-08-18.md; design: docs/g-track/registry-generator-design-2026-08-18.md.

Why

The G0 session generated a registry instance with a shell one-liner and hit a silent corruption of the base IRI — a sigil in a parameter value re-read as code, the wrong value propagated into every file including the birth provenance, caught only by a human reading the output. The base IRI is the one registry parameter with no supersession path: a changed IRI orphans identifiers rather than superseding them (g-dec-03). Two faults, not one — string substitution re-reads values as code in every engine that does it, and a scratchpad script cannot be tested.

The finding: the generator paid for itself before it minted anything

graphs/canonical/_exemplar.ttl attributed its provenance to <https://REGISTRY-HOST.example/agent/g1-session> — a second occurrence of the placeholder host, outside the /ns# base every other file used. Substituting the base IRI alone would have left that IRI pointing at REGISTRY-HOST.example in every generated instance's ratified data file: the G0 class of bug, invisible to anyone reading the diff for the parameter they supplied. The gate's Check B found it before any tree was written. Correction: the attribution now sits in the instance's own namespace, reg:agent-g1-session, leaving one host token in the template.

The subcommand

product registry generate --owner <org> --repo <name> --ratifier <person> \
    --display-name <name> --base-iri <iri> --date <YYYY-MM-DD> \
    --generated-by <who> --out <dir>

product registry check <dir>

Slice + adapter: pure product-core/src/registry/ (params · template · substitute · verify · plan · apply · check/), thin product-cli/src/commands/registry.rs. No MCP mirror — it mutates no .product graph, and repository minting does not belong inside a phase-gated authoring session. The template stays at docs/g-track/registry-template/, embedded by include_str! with a manifest-drift test.

Generate / publish split. Generation is local and offline: render in memory, gate, refuse a non-empty target, write only beneath --out, then git init and one birth commit with inline identity, no system or global git config, dates pinned to the mint date — the same parameters produce the same tree bytes and the same commit id. Publication is a separate act and is not a subcommand: no remote, no network call, no repository created; the two publish commands are printed for a human to run. Only the local half can be a fixture, and a generator doing both would put the untested half on the tested half's back.

Why the renderer cannot re-read a value

Typed arguments, no substitution table. The renderer walks the template once: match the longest token from a closed table, copy the parameter's bytes, record the span written, advance past the token. The cursor never moves backwards and the output is never re-scanned. The invariant: the output is the concatenation of literal spans of the template with byte-for-byte copies of parameter values — so &, \1, $0, backslash, backtick, $(…) are data, and substitution order is irrelevant because there is no order.

Sequential str::replace is the mechanism that nearly works and does not: a value inserted by an earlier replacement sits in the buffer later replacements scan. sequential_replace_would_have_re_read_it runs both on the same input and shows the naive one producing SECOND where the renderer produces what was supplied — the bug this session would have shipped, kept as a test rather than a comment. No templating dependency: every engine brings an expression language and an escaping mode, surface whose purpose is to interpret its inputs.

The gate

Runs on the in-memory tree before the first byte reaches disk. A failure is a refusal: no target directory is created at all.

  • Check A — round-trip. Recorded spans read back out of the rendered bytes and compared to the parameters; the per-parameter tally cross-checked against a naive occurrence count over the template; every parameter must reach the output at least once. This is what would have caught the G0 bug, loudly, at generation time.
  • Check B — no survivors. No {{IDENT}}, no base-IRI sentinel. A placeholder without a typed parameter fails generation rather than shipping. TEMPLATE.md is asserted byte-identical to the template's copy rather than exempted.

The value-span masking rule. Check B reads the template text that survived rendering — value spans masked — not the raw output. The first run of the hostile-value fixture refused a display name containing {{OWNER_ORG}}; refusing it would have been fail-closed but wrong, because "values are data" that stops holding once a value looks like a template is not a property at all. Masking keeps both guarantees whole: an unwired placeholder outside a value still aborts, and a value is data end to end. Same for a supplied IRI that happens to contain REGISTRY-HOST.example.

Fixtures — 70 new tests, all passing

52 unit (product-core/src/registry/), 17 generation fixtures, 1 divergence measure.

Fixture Asserts Result
parameters_round_trip_byte_identically every parameter byte-identical at every site pass
g0_regression_a_sigil_in_the_ratifiers_address_survives strike out every whole value; no fragment of the address or base IRI remains — the shape the G0 corruption took pass
the_g0_handoff_parameters_mint_the_handed_over_instance the handoff's five parameters + mint date; tag:emil@okkels-klein.dk,2026-08-17:ground/ byte for byte in prefix, reg:baseIri, shapes, exemplars pass
hostile_parameter_values_are_data &, \1, $0, backtick, $(…), |, /, a literal {{OWNER_ORG}} pass
sequential_replace_would_have_re_read_it the bug that would have shipped, shown failing pass
no_placeholder_survives_the_generated_tree · template_md_travels_verbatim zero survivors; TEMPLATE.md verbatim with placeholders intact pass
the_birth_provenance_is_complete_as_rdf queried as RDF, not string-matched: both versions, every parameter, the agent pass
a_fresh_instance_passes_its_own_rules file rule + shapes clean on a fresh instance pass
Negative shapes institutional Reading without a trust decision · two assertions in one file · decision missing its falsifier · decision dated by a bare string · provenance outside the vocabulary · an empty file all fail as required
unsupported_shacl_construct_fails_closed · an_unreadable_shape_fails_closed_with_its_own_heading unreadable shape fails, own section, no violations section pass
a_refused_parameter_writes_nothing · generation_refuses_a_non_empty_target no directory left; prior content untouched pass
generation_is_deterministic identical bytes and identical commit id pass
two_generations_in_one_run_do_not_interfere · generation_leaves_no_residue_outside_its_target teardown provable pass
generation_configures_no_remote no remote; publish commands printed, never run pass
embedded_template_matches_on_disk · version-constant drift manifest and header cannot drift pass

Gates green at every hold: cargo t (1352 tests, 0 failed), cargo clippy --workspace -- -D warnings -D clippy::unwrap_used, cargo xtask check.

Template 0.1.0 → 0.2.0

One bump covering every change this session made — the template is not released between gates, so two bumps would record a state that never existed.

  • shapes/decision.ttl — title, resolution, region, falsifier, ratifier, status, date required; basis / acceptedCost / revisitIf deliberately not, because a decision may honestly have none and demanding them produces filler. reg:made constrained to xsd:date; reg:ratifiedBy left unconstrained as to node kind, since whether a ratifier is named by literal or IRI is an instance's ruling.
  • graphs/canonical/_exemplar-decision.ttl — a conforming example, so a ratifier learns the form from a file rather than from SHACL; carries {{RATIFIER}} with a status stating plainly that it is not ground.
  • The CI file rule generalised from a filename exemption to the class rule it stood in for: exactly one assertion or exactly one decision per graphs/**/*.ttl. The zero-triple tolerance went with the empty founding-decision file, which no longer ships — the slot is a path the ratifier creates.
  • TEMPLATE.md step 1 no longer reads both ways: the file travels verbatim, unsubstituted.
  • Two base-IRI routes documented with trade-offs — durable host, or location-independent minting per g-dec-03 with the HTTP form published later as a projection. Neither is the default. The README records that which route an instance took is a decision filed in that registry, not a fact recoverable from the IRI.
  • GENERATION.ttl parameterised, filled by the same renderer under the same gate, recording the generator's version alongside the template's, and correcting the PROV (prov:wasGeneratedBy the act, prov:wasAttributedTo the agent).

The pySHACL divergence measure

The instance's CI runs pyshacl; the fixtures run a fail-closed native reader (a defined SHACL subset compiled to SPARQL over oxigraph) because a gate depending on a Python toolchain becomes a habit the moment it is skipped when absent. Anything outside the subset is reported as unevaluable, fails the check, and is reported distinctly from data violating a shape.

Two readers of one rule set is a standing finding, so it is measured rather than assumed small. Run in this session — pySHACL 0.40.1, nine cases, no disagreement; non-vacuity confirmed by pySHACL independently reporting Conforms: False on the negatives. A fresh 0.2.0 instance was additionally validated by the instance workflow's own two steps run verbatim (pyshacl -a over shapes/, and the workflow's rdflib file-rule script) — both clean.

What G0 must do differently when it resumes

  1. Regenerate through the subcommand — no shell substitution, no hand-editing of the tree.
  2. The same five parameters and the same mint date, 2026-08-17: it records when the authority was demonstrably controlled, not when the tree was written.
  3. File the two ratified texts verbatim — the founding decision and g-dec-03. They were ratified as text, not as the tree they were written into.
  4. The founding decision creates its own file; there is no empty slot to fill.
  5. Do not re-apply shapes/decision.ttl or the file rule as instance-local — they are in the template at 0.2.0, and re-adding them forks it.
  6. The instance records template 0.2.0 and the generator's version; re-pinning is possible against either.
  7. Publishing stays a separate, explicit act.

Out of scope, untouched

G0 itself, the extractor, the PRD, canon. No instance was published to any remote; Hafeok/ground-registry-g0 remains empty. No repository was created under any organisation.


Generated by Claude Code

claude added 9 commits August 18, 2026 07:58
The G0 instance was generated by a shell one-liner; a sigil in a parameter
value was re-read as code and the corrupted base IRI propagated into every
file, birth provenance included. The base IRI has no supersession path.

Proposes `product registry generate` / `product registry check`: typed
arguments, a single-pass renderer that copies values without ever re-scanning
its own output, an abort-before-writing verification gate (parameter
round-trip byte-identical, zero surviving placeholders), a parameterised
GENERATION.ttl recording generator version alongside template version, and
the generate/publish boundary that keeps generation offline and testable.

Design only — no implementation. Gates green: cargo t, clippy, xtask check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
…s point

`product registry generate` mints a ground-registry instance from the
versioned template; `product registry check` reads a generated instance
against its own rules.

The mechanism, per the Gate 1 design:

- Typed arguments, no substitution table. A single-pass renderer walks the
  template once, copies each value's bytes, records the span it wrote, then
  advances past the token. The cursor never moves backwards and the output is
  never re-scanned, so a value carrying `&`, `\1`, `$0` or a literal
  `{{OWNER_ORG}}` is data. A unit test shows sequential `str::replace`
  failing on the same input.
- The gate runs on the in-memory tree before the first byte reaches disk.
  Check A reads the recorded spans back out and cross-checks the tally
  against a naive occurrence count over the template. Check B scans the
  template text that survived rendering — values masked out — for surviving
  placeholders and for the base-IRI sentinel. A failure is a refusal: no
  directory is created at all.
- Generation is local. `git init` plus one birth commit with inline identity,
  no system or global config, dates pinned to the mint date, so the same
  parameters produce the same tree and the same commit id. No remote is
  configured; the publish commands are printed for a human to run.
- `check` evaluates a defined SHACL subset compiled to SPARQL over oxigraph,
  reading the instance's own shapes files. Any construct outside the subset
  is reported as unevaluable and fails the check, in its own section, never
  silently skipped.

Template changes the mechanism forced (the rest are held for Gate 3):

- GENERATION.ttl is now a parameterised Turtle file, so the birth provenance
  is produced by the mechanism rather than typed beside it. It records the
  generator's version alongside the template's, both parameters and agent.
- The exemplar's attribution IRI moves into the instance namespace: Check B
  caught a second host occurrence that base-IRI substitution alone would have
  left corrupt.
- The empty founding-decision slot is dropped, per the Q1 ruling — the slot
  is a path the ratifier creates, not a file needing a CI carve-out.

69 new tests: 51 unit, 17 generation fixtures, 1 pySHACL divergence measure
(both readers agree across six cases, including the sh:sparql constraint).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
… → 0.2.0

One bump covering every template change this session made — the template is
not released between gates, so two bumps would record a state that never
existed.

- shapes/decision.ttl: title, resolution, region, falsifier, ratifier,
  status, date required; basis / acceptedCost / revisitIf deliberately not,
  because a decision may honestly have none and demanding them produces
  filler. `reg:ratifiedBy` is left unconstrained as to node kind — whether a
  ratifier is named by literal or IRI is an instance's ruling.
- graphs/canonical/_exemplar-decision.ttl: a conforming example, so a
  ratifier learns the form from a file rather than from SHACL. It shows the
  optional fields precisely because they are optional.
- The CI file rule is generalised from a filename exemption to the class rule
  it stood in for: exactly one assertion OR exactly one decision, per
  graphs/**/*.ttl, no exemptions. The zero-triple tolerance went with the
  empty slot.
- TEMPLATE.md step 1 no longer reads both ways: this file travels verbatim,
  unsubstituted, and that is what lets an instance re-pin. The generation
  steps now describe the generator, the gate, the local/publish split, and
  the founding-decision slot as a path rather than a shipped file.
- The base-IRI row documents both routes with their trade-offs: a durable
  host, or location-independent minting per g-dec-03 with the HTTP form
  published later as a projection. Neither is the default; the choice is
  made deliberately or deferred deliberately.
- README follows: the slot as a path, the decision shape, the file rule, and
  why the base IRI's route is a filed decision rather than a fact recoverable
  from the IRI.

Verified on a fresh instance by the instance's own CI steps run verbatim —
pyshacl -a over shapes/, and the workflow's rdflib file-rule script — as well
as by `product registry check`. The pySHACL divergence measure now covers the
decision shape, including sh:datatype: nine cases, no disagreement.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
Closes the four-gate session: the finding that paid for the mechanism, the
subcommand surface and the generate/publish split, the single-pass renderer,
the gate's two checks with the value-span masking rule, the fixture results,
the template at 0.2.0, the pySHACL divergence measurement, and what G0 must
do differently when it resumes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The contract-surface gate counted 151 governed changes, which is a fair
reading of what was published: every internal type of the renderer, the
gate, the shapes reader stood as public API, plus a test-support module
that is scaffolding rather than surface.

The slice's public surface is now its re-export list — the acts (plan,
apply, check, evaluate) with the types they carry. Submodules are
crate-internal, so a caller depends on the acts and never on how the
template is rendered or how the gate is spelled. Test support is
pub(crate) within its own test binary.

No behaviour change; all registry tests pass unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
Ten seams over the registry generator's surface, each carrying what a
caller actually learns at that boundary and what the boundary cannot do —
the apply seam states that publishing is not among its powers and that
COMMIT_EMAIL is a reserved .invalid address rather than an identity; the
check seam states its own limit, that it is a second reader of rules the
instance's CI runs under pySHACL and the divergence is measured rather
than assumed; the params seam states that validation judges meaning and
never character safety, because a character rule is an escaping rule and
an escaping rule is what corrupted the first generated instance.

59 bindings filed by `ddd bind` over 61222d2..HEAD.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The enum's variants mirror SHACL's own constraint-component names, which
is deliberate — the reader's vocabulary should be the shapes' vocabulary.
Naming the enum Kind put NodeKind in it, which clippy reads as a variant
repeating its enum's name; the lint was masked while the type was public
API and surfaced when the surface narrowed. Renaming the enum keeps the
SHACL names intact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The binding chain composes over file hashes, so marking TEMPLATE_DIR
test-only invalidated the chain for the symbol beside it. Re-filed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The gate failed the first CI run at 151 undischarged changes, which was a
fair reading of an over-wide public surface. Narrowing the slice to its
acts took it to 59 — the surface actually intended — discharged by ten
seam declarations.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
@Hafeok
Hafeok merged commit 6fc95de into main Aug 18, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants