v1.9.0 — Resolve by catalogue - #10
Merged
Merged
Conversation
…tion) Implements the v1.9.0 milestone from ROADMAP.md in full (README item 5) and nothing from v1.10.0 onwards. - src/catalogue.ts: catalogue import (JSON + line format) for table/view/column metadata; normalized exact/synonym lookup; conflict detection. - Verified resolution replaces the v1.5.0 external label where catalogue evidence proves the match, in both the estate dependency graph (ir.ts) and query sources (lineage.ts); the catalogue-absent path stays byte-identical. - E diagnostics: catalogue_parse_error, catalogue_empty, catalogue_conflict (document), and region-scoped catalogue_conflict / catalogue_partial. Resolution metadata survives draw.io round-trip. - UI: Catalogue menu (paste/import/apply/clear + status). - Workspace schema -> v2 carrying the catalogue text; v1 snapshots migrate. - tests/catalogue.ts (13 fixtures, gated), 2 UI tests, cataloguePassRate added to docs/metrics-v1.9.0.json; package bumped to 1.9.0. - Docs: RELEASE_NOTE_v1.9.0.md, PR_NOTE_1.9.0.md, README, examples/dbo.v190_demo.sql.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR: v1.9.0 — Resolve by catalogue
Summary
Implements the v1.9.0 — Resolve by catalogue milestone from
ROADMAP.mdin full and nothing from v1.10.0 onwards: README post-v1.0.0 item 5 —
catalogue metadata import (paste/import table/view/column catalogues) with
exact synonym, linked-server, and cross-database resolution replacing
v1.5.0's conservative external label where catalogue evidence exists, plus
E diagnostics for partial or conflicting catalogue data. The golden suite
grows from 207 to 208; fuzz (400) stays green; the UI suite grows from 20 to
22; catalogue fixtures (13) cover import parsing (JSON + line format), exact
and synonym verification in the estate and query graphs, conservative conflict
and cross-database partial handling, draw.io metadata, and the workspace
round-trip. Column lineage (v1.10.0+) and RDL (v1.12.0) remain deferred.
What's included
Catalogue model and import (
src/catalogue.ts)A new runtime module (loaded after
tokenizer.js, before the analysis stages):parseCatalogue(text, format?)auto-detects JSON ({objects:[{name, kind, synonyms[]}], columns:[{table, name}]}or a bare array) and aforgiving one-object-per-line format (
# comment,name KIND syn1, syn2,COL table.column) and returns{catalogue, diagnostics, format, objectCount, columnCount}.buildCatalogueIndexnormalizes names (case-insensitive, bracket/quotestripping) into
byName/bySynonymlookup maps and records conflicts(duplicate object names, or a synonym colliding with an object name) so
ambiguous evidence is never invented into a verification.
resolveCatalogue(cat, name)returnsverified(exact full-name orexplicit-synonym match with the resolved canonical name),
conflict(conservative), or
external(unproven).suffixCatalogueMatches(cat, name)lists near-miss candidates (the referencecarries extra leading server/database parts over a catalogued object) used
for conservative reporting.
catalogue_parse_error,catalogue_empty,catalogue_conflict(document-scoped), plus the region-scopedcatalogue_partial/catalogue_conflictraised against a reference.Resolution wired into both views
src/ir.tsdependencyGraph(objects, opts)now accepts the catalogue:unmatched leaves that the catalogue proves render as their canonical name
with
provenance: external,resolution: verified, and the originalobjectIdpreserved; conflicts stay conservative with a marker and reason;unmatched names keep the exact v1.5.0 behaviour (byte-identical without a
catalogue).
src/lineage.tsbuildQueryGraph/buildObjectQueryGraphresolve querysource nodes the same way whenever
opts.catalogueis present; thecatalogue-absent path is unchanged (regression-guarded by a fixture).
src/ir.tsanalyse()merges document-scoped catalogue parse diagnosticsand attaches region-scoped
catalogue_conflict/catalogue_partialdiagnostics to each reference (span-attached, conservative).
src/exporters.tscarriesresolution=andresolved=provenance metadataon draw.io vertices so verification survives the round-trip.
UI (
index.html,src/app.ts)A Catalogue command menu with a paste textarea, Import file,
Apply catalogue, and Clear, plus a status line that reports the parsed
object/column/conflict counts. Applying re-runs the current analysis so
verified resolution appears immediately; the catalogue is an analysis input and
clearing it re-runs without it.
Workspace persistence (schema v2)
src/workspace.tsbumpsWORKSPACE_SCHEMA_VERSIONto 2 and carries the rawcatalogue text as an optional top-level snapshot field; v1 snapshots migrate
forward with
catalogue: null, so save → reload reproduces an identicalanalysis even when a catalogue was applied.
app.tssaves/restores thecatalogue text and re-parses it on restore.
Fixtures and tests
tests/catalogue.ts(+tests/index.html,tests/metrics.html, andcatalogue.jswiring) — 13 fixture-corpus records covering JSON and lineparsing, malformed/empty input diagnostics, duplicate-name and synonym
collision conflicts, estate verification (exact + synonym + linked server),
query-source verification, conservative cross-database partials with
region-scoped diagnostics, the catalogue-absent regression guard, draw.io
metadata survival, and the workspace catalogue round-trip + v1 migration.
Gates the golden page via
PROCFLOW_CATALOGUE_PASS.tests/ui-tests.ts— 2 new browser tests: catalogue verification in thedependency view and clearing the catalogue resetting its status.
tests/metrics.ts→docs/metrics-v1.9.0.json— addscataloguePassRateand the
cataloguecorpus count (13/13); the v1.8.0 snapshot is renamed tov1.9.0.
examples/dbo.v190_demo.sql— a multi-object demo with a matching catalogueyou can paste to see verification, a catalogued synonym, and an unmatched
external name.
Verification
npm run typecheck— passesnpm run build— passesnpm run test:file— passes (withCHROME_PATHset)npm run metrics— snapshot is current;docs/metrics-v1.9.0.jsoncommitted(v1.8.0 snapshot renamed)
dist/committed in syncNot changed / deferred (per roadmap)
projections) and RDL/report import — is out of scope for this release.
green (verified by the unchanged catalogue-absent byte-identical paths and
the full golden suite).