The registry generator: a tested mechanism for minting instances - #49
Merged
Conversation
The G0 instance was generated by a shell one-liner; a sigil in a parameter value was re-read as code and the corrupted base IRI propagated into every file, birth provenance included. The base IRI has no supersession path. Proposes `product registry generate` / `product registry check`: typed arguments, a single-pass renderer that copies values without ever re-scanning its own output, an abort-before-writing verification gate (parameter round-trip byte-identical, zero surviving placeholders), a parameterised GENERATION.ttl recording generator version alongside template version, and the generate/publish boundary that keeps generation offline and testable. Design only — no implementation. Gates green: cargo t, clippy, xtask check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
…s point
`product registry generate` mints a ground-registry instance from the
versioned template; `product registry check` reads a generated instance
against its own rules.
The mechanism, per the Gate 1 design:
- Typed arguments, no substitution table. A single-pass renderer walks the
template once, copies each value's bytes, records the span it wrote, then
advances past the token. The cursor never moves backwards and the output is
never re-scanned, so a value carrying `&`, `\1`, `$0` or a literal
`{{OWNER_ORG}}` is data. A unit test shows sequential `str::replace`
failing on the same input.
- The gate runs on the in-memory tree before the first byte reaches disk.
Check A reads the recorded spans back out and cross-checks the tally
against a naive occurrence count over the template. Check B scans the
template text that survived rendering — values masked out — for surviving
placeholders and for the base-IRI sentinel. A failure is a refusal: no
directory is created at all.
- Generation is local. `git init` plus one birth commit with inline identity,
no system or global config, dates pinned to the mint date, so the same
parameters produce the same tree and the same commit id. No remote is
configured; the publish commands are printed for a human to run.
- `check` evaluates a defined SHACL subset compiled to SPARQL over oxigraph,
reading the instance's own shapes files. Any construct outside the subset
is reported as unevaluable and fails the check, in its own section, never
silently skipped.
Template changes the mechanism forced (the rest are held for Gate 3):
- GENERATION.ttl is now a parameterised Turtle file, so the birth provenance
is produced by the mechanism rather than typed beside it. It records the
generator's version alongside the template's, both parameters and agent.
- The exemplar's attribution IRI moves into the instance namespace: Check B
caught a second host occurrence that base-IRI substitution alone would have
left corrupt.
- The empty founding-decision slot is dropped, per the Q1 ruling — the slot
is a path the ratifier creates, not a file needing a CI carve-out.
69 new tests: 51 unit, 17 generation fixtures, 1 pySHACL divergence measure
(both readers agree across six cases, including the sh:sparql constraint).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
… → 0.2.0 One bump covering every template change this session made — the template is not released between gates, so two bumps would record a state that never existed. - shapes/decision.ttl: title, resolution, region, falsifier, ratifier, status, date required; basis / acceptedCost / revisitIf deliberately not, because a decision may honestly have none and demanding them produces filler. `reg:ratifiedBy` is left unconstrained as to node kind — whether a ratifier is named by literal or IRI is an instance's ruling. - graphs/canonical/_exemplar-decision.ttl: a conforming example, so a ratifier learns the form from a file rather than from SHACL. It shows the optional fields precisely because they are optional. - The CI file rule is generalised from a filename exemption to the class rule it stood in for: exactly one assertion OR exactly one decision, per graphs/**/*.ttl, no exemptions. The zero-triple tolerance went with the empty slot. - TEMPLATE.md step 1 no longer reads both ways: this file travels verbatim, unsubstituted, and that is what lets an instance re-pin. The generation steps now describe the generator, the gate, the local/publish split, and the founding-decision slot as a path rather than a shipped file. - The base-IRI row documents both routes with their trade-offs: a durable host, or location-independent minting per g-dec-03 with the HTTP form published later as a projection. Neither is the default; the choice is made deliberately or deferred deliberately. - README follows: the slot as a path, the decision shape, the file rule, and why the base IRI's route is a filed decision rather than a fact recoverable from the IRI. Verified on a fresh instance by the instance's own CI steps run verbatim — pyshacl -a over shapes/, and the workflow's rdflib file-rule script — as well as by `product registry check`. The pySHACL divergence measure now covers the decision shape, including sh:datatype: nine cases, no disagreement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
Closes the four-gate session: the finding that paid for the mechanism, the subcommand surface and the generate/publish split, the single-pass renderer, the gate's two checks with the value-span masking rule, the fixture results, the template at 0.2.0, the pySHACL divergence measurement, and what G0 must do differently when it resumes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The contract-surface gate counted 151 governed changes, which is a fair reading of what was published: every internal type of the renderer, the gate, the shapes reader stood as public API, plus a test-support module that is scaffolding rather than surface. The slice's public surface is now its re-export list — the acts (plan, apply, check, evaluate) with the types they carry. Submodules are crate-internal, so a caller depends on the acts and never on how the template is rendered or how the gate is spelled. Test support is pub(crate) within its own test binary. No behaviour change; all registry tests pass unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
Ten seams over the registry generator's surface, each carrying what a caller actually learns at that boundary and what the boundary cannot do — the apply seam states that publishing is not among its powers and that COMMIT_EMAIL is a reserved .invalid address rather than an identity; the check seam states its own limit, that it is a second reader of rules the instance's CI runs under pySHACL and the divergence is measured rather than assumed; the params seam states that validation judges meaning and never character safety, because a character rule is an escaping rule and an escaping rule is what corrupted the first generated instance. 59 bindings filed by `ddd bind` over 61222d2..HEAD. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The enum's variants mirror SHACL's own constraint-component names, which is deliberate — the reader's vocabulary should be the shapes' vocabulary. Naming the enum Kind put NodeKind in it, which clippy reads as a variant repeating its enum's name; the lint was masked while the type was public API and surfaced when the surface narrowed. Renaming the enum keeps the SHACL names intact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The binding chain composes over file hashes, so marking TEMPLATE_DIR test-only invalidated the chain for the symbol beside it. Re-filed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
The gate failed the first CI run at 151 undischarged changes, which was a fair reading of an over-wide public surface. Narrowing the slice to its acts took it to 59 — the surface actually intended — discharged by ten seam declarations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HeV8CH1FZLGoSES6gFMGHw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Status: the generator is the mechanism; G0 regenerates through it.
Four gates, all ratified: design (
cdb571c), implementation (c30a885), template changes (23f8e19), close (a7936d6). Full report:docs/g-track/registry-generator-session-2026-08-18.md; design:docs/g-track/registry-generator-design-2026-08-18.md.Why
The G0 session generated a registry instance with a shell one-liner and hit a silent corruption of the base IRI — a sigil in a parameter value re-read as code, the wrong value propagated into every file including the birth provenance, caught only by a human reading the output. The base IRI is the one registry parameter with no supersession path: a changed IRI orphans identifiers rather than superseding them (
g-dec-03). Two faults, not one — string substitution re-reads values as code in every engine that does it, and a scratchpad script cannot be tested.The finding: the generator paid for itself before it minted anything
graphs/canonical/_exemplar.ttlattributed its provenance to<https://REGISTRY-HOST.example/agent/g1-session>— a second occurrence of the placeholder host, outside the/ns#base every other file used. Substituting the base IRI alone would have left that IRI pointing atREGISTRY-HOST.examplein every generated instance's ratified data file: the G0 class of bug, invisible to anyone reading the diff for the parameter they supplied. The gate's Check B found it before any tree was written. Correction: the attribution now sits in the instance's own namespace,reg:agent-g1-session, leaving one host token in the template.The subcommand
Slice + adapter: pure
product-core/src/registry/(params·template·substitute·verify·plan·apply·check/), thinproduct-cli/src/commands/registry.rs. No MCP mirror — it mutates no.productgraph, and repository minting does not belong inside a phase-gated authoring session. The template stays atdocs/g-track/registry-template/, embedded byinclude_str!with a manifest-drift test.Generate / publish split. Generation is local and offline: render in memory, gate, refuse a non-empty target, write only beneath
--out, thengit initand one birth commit with inline identity, no system or global git config, dates pinned to the mint date — the same parameters produce the same tree bytes and the same commit id. Publication is a separate act and is not a subcommand: no remote, no network call, no repository created; the two publish commands are printed for a human to run. Only the local half can be a fixture, and a generator doing both would put the untested half on the tested half's back.Why the renderer cannot re-read a value
Typed arguments, no substitution table. The renderer walks the template once: match the longest token from a closed table, copy the parameter's bytes, record the span written, advance past the token. The cursor never moves backwards and the output is never re-scanned. The invariant: the output is the concatenation of literal spans of the template with byte-for-byte copies of parameter values — so
&,\1,$0, backslash, backtick,$(…)are data, and substitution order is irrelevant because there is no order.Sequential
str::replaceis the mechanism that nearly works and does not: a value inserted by an earlier replacement sits in the buffer later replacements scan.sequential_replace_would_have_re_read_itruns both on the same input and shows the naive one producingSECONDwhere the renderer produces what was supplied — the bug this session would have shipped, kept as a test rather than a comment. No templating dependency: every engine brings an expression language and an escaping mode, surface whose purpose is to interpret its inputs.The gate
Runs on the in-memory tree before the first byte reaches disk. A failure is a refusal: no target directory is created at all.
{{IDENT}}, no base-IRI sentinel. A placeholder without a typed parameter fails generation rather than shipping.TEMPLATE.mdis asserted byte-identical to the template's copy rather than exempted.The value-span masking rule. Check B reads the template text that survived rendering — value spans masked — not the raw output. The first run of the hostile-value fixture refused a display name containing
{{OWNER_ORG}}; refusing it would have been fail-closed but wrong, because "values are data" that stops holding once a value looks like a template is not a property at all. Masking keeps both guarantees whole: an unwired placeholder outside a value still aborts, and a value is data end to end. Same for a supplied IRI that happens to containREGISTRY-HOST.example.Fixtures — 70 new tests, all passing
52 unit (
product-core/src/registry/), 17 generation fixtures, 1 divergence measure.parameters_round_trip_byte_identicallyg0_regression_a_sigil_in_the_ratifiers_address_survivesthe_g0_handoff_parameters_mint_the_handed_over_instancetag:emil@okkels-klein.dk,2026-08-17:ground/byte for byte in prefix,reg:baseIri, shapes, exemplarshostile_parameter_values_are_data&,\1,$0, backtick,$(…),|,/, a literal{{OWNER_ORG}}sequential_replace_would_have_re_read_itno_placeholder_survives_the_generated_tree·template_md_travels_verbatimthe_birth_provenance_is_complete_as_rdfa_fresh_instance_passes_its_own_rulesunsupported_shacl_construct_fails_closed·an_unreadable_shape_fails_closed_with_its_own_headinga_refused_parameter_writes_nothing·generation_refuses_a_non_empty_targetgeneration_is_deterministictwo_generations_in_one_run_do_not_interfere·generation_leaves_no_residue_outside_its_targetgeneration_configures_no_remoteembedded_template_matches_on_disk· version-constant driftGates green at every hold:
cargo t(1352 tests, 0 failed),cargo clippy --workspace -- -D warnings -D clippy::unwrap_used,cargo xtask check.Template 0.1.0 → 0.2.0
One bump covering every change this session made — the template is not released between gates, so two bumps would record a state that never existed.
shapes/decision.ttl— title, resolution, region, falsifier, ratifier, status, date required;basis/acceptedCost/revisitIfdeliberately not, because a decision may honestly have none and demanding them produces filler.reg:madeconstrained toxsd:date;reg:ratifiedByleft unconstrained as to node kind, since whether a ratifier is named by literal or IRI is an instance's ruling.graphs/canonical/_exemplar-decision.ttl— a conforming example, so a ratifier learns the form from a file rather than from SHACL; carries{{RATIFIER}}with a status stating plainly that it is not ground.graphs/**/*.ttl. The zero-triple tolerance went with the empty founding-decision file, which no longer ships — the slot is a path the ratifier creates.g-dec-03with the HTTP form published later as a projection. Neither is the default. The README records that which route an instance took is a decision filed in that registry, not a fact recoverable from the IRI.GENERATION.ttlparameterised, filled by the same renderer under the same gate, recording the generator's version alongside the template's, and correcting the PROV (prov:wasGeneratedBythe act,prov:wasAttributedTothe agent).The pySHACL divergence measure
The instance's CI runs
pyshacl; the fixtures run a fail-closed native reader (a defined SHACL subset compiled to SPARQL over oxigraph) because a gate depending on a Python toolchain becomes a habit the moment it is skipped when absent. Anything outside the subset is reported as unevaluable, fails the check, and is reported distinctly from data violating a shape.Two readers of one rule set is a standing finding, so it is measured rather than assumed small. Run in this session — pySHACL 0.40.1, nine cases, no disagreement; non-vacuity confirmed by pySHACL independently reporting
Conforms: Falseon the negatives. A fresh 0.2.0 instance was additionally validated by the instance workflow's own two steps run verbatim (pyshacl -aovershapes/, and the workflow's rdflib file-rule script) — both clean.What G0 must do differently when it resumes
g-dec-03. They were ratified as text, not as the tree they were written into.shapes/decision.ttlor the file rule as instance-local — they are in the template at 0.2.0, and re-adding them forks it.Out of scope, untouched
G0 itself, the extractor, the PRD, canon. No instance was published to any remote;
Hafeok/ground-registry-g0remains empty. No repository was created under any organisation.Generated by Claude Code