fix(supersimple): two-sided join keys, table prefix, and case control - #21
Open
andrebaaij wants to merge 3 commits into
Open
fix(supersimple): two-sided join keys, table prefix, and case control#21andrebaaij wants to merge 3 commits into
andrebaaij wants to merge 3 commits into
Conversation
Hex's Semantic Model Sync parses dbt MetricFlow YAML straight from a git repo
(not via an API), and binds each semantic model to a physical table through
`config.meta.hex.table`. Without that key the sync imports models that resolve to
nothing: the agent then answers about no data, which a benchmark grades as a
wrong answer rather than the setup failure it is. That exact failure mode cost a
full Lightdash run earlier.
Adds two options, both off by default:
table-prefix logical -> physical name (ClickHouse materialises fct_orders
as marts__fct_orders)
dbt-hex-meta emit the config.meta.hex.table binding
`dbt` becomes Configurable to receive them. A plain dbt build is unchanged and
carries no Hex key — asserted by a test, because the dbt target also writes the
shared ground-truth reference every arm reads, and one vendor's meta must not
leak into it.
Also fixes a defaulting bug this surfaced. `schema` defaulted to MAIN
unconditionally, so the binding came out as `ecomm.MAIN.marts__fct_orders`, which
does not resolve on ClickHouse's two-part namespace. `schema` is now a pointer so
an EXPLICIT empty value is distinguishable from an omitted one; omitted still
defaults to MAIN. Changing the default globally was the wrong fix: ossie,
nao-yaml, supersimple and the warehouse dialects all read Schema, and Snowflake
targets genuinely need the qualified form.
Verified against the pinned eval reference: 38 semantic models, 22 measures, 14
metrics, all 38 bound to ecomm.marts__<name>.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Schema *string change altered the struct's field-alignment column, which gofmt reflows. CI checks gofmt -l and I did not run it before pushing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three changes needed to target a case-sensitive warehouse with prefixed tables.
The first is a correctness bug that would have produced wrong numbers silently.
TWO-SIDED JOIN KEYS. The emitter wrote `join_key: <parent column>` for every
relation. That is Supersimple's shorthand for "both sides share this column
name", which held for 20 of 23 relations and was WRONG for the rest:
fct_orders.order_date -> dim_date.date_day was emitted as `join_key: date_day`,
addressing a column the child does not have. `supersimple validate` caught three
of these ("references unknown property 'date_day' on related model"), but the
dangerous case is the one it could not catch — the golden encoded a role-playing
relation joining BILLING_CUSTOMER_SK as `join_key: CUSTOMER_SK`, a column that
DOES exist on both sides, so it would have joined on the wrong one and returned
plausible wrong answers rather than erroring. The vendor's own discovery snapshot
uses join_key_on_base/join_key_on_related; we now emit the shorthand only when
both sides genuinely agree. Golden updated: it captured the bug.
TablePrefix maps the IR's logical name to the physical one (ClickHouse
materialises fct_orders as marts__fct_orders). LowerCaseIdentifiers keeps
identifiers as spelled instead of upper-casing them: the upper-case default suits
Snowflake, where unquoted identifiers fold up, but ClickHouse is case-SENSITIVE
and MARTS__FCT_ORDERS does not exist. Both default off, so Snowflake output is
byte-identical.
Schema now honours an EXPLICIT empty value rather than always defaulting to
MAIN — ClickHouse has a two-part namespace, so MAIN.marts__fct_orders resolves
to nothing.
Verified end to end: 38 models, 338 properties, 23 relations, 14 metrics
(on_time_delivery_rate intact as a real operations expression, not degraded to
prose), no NOTES sidecar, `supersimple validate` reports Configuration is valid.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three changes needed to target a case-sensitive warehouse with prefixed tables. The first is a correctness bug that would have produced wrong numbers silently.
Two-sided join keys (the bug)
The emitter wrote
join_key: <parent column>for every relation. That is Supersimple's shorthand for both sides share this column name — true for 20 of 23 relations here, wrong for the rest:supersimple validatecaught three of these (references unknown property 'date_day' on related model). The dangerous case is the one it could not catch: the golden encoded a role-playing relation joiningBILLING_CUSTOMER_SKasjoin_key: CUSTOMER_SK, a column that exists on both models. That validates cleanly and joins on the wrong column, returning plausible wrong answers.The vendor's own discovery snapshot uses the two-sided form:
We now emit the shorthand only when both sides genuinely agree. The golden is updated because it captured the bug.
Two new options, both default-off
table-prefixfct_ordersasmarts__fct_orderslowercase-identifiersMARTS__FCT_ORDERSdoes not existschemanow honours an explicit empty value rather than always defaulting toMAIN— ClickHouse's two-part namespace makesMAIN.marts__fct_ordersunresolvable.Snowflake output is byte-identical with the options off.
Verification
38 models, 338 properties, 23 relations, 14 metrics.
on_time_delivery_ratesurvives as a realoperationsexpression rather than degrading to prose. NoNOTES.mdsidecar, so nothing was dropped.supersimple validate→Configuration is valid!🤖 Generated with Claude Code
https://claude.ai/code/session_01P1Ae794MJcezTixWn2wZ4J