feat(test): spareval differential oracle — orchestration fully in Rust (#17)#59
Merged
Conversation
#17) Lands the W3C SPARQL differential oracle as a standalone crate (tests/oracle, pgrdf-oracle: spareval + oxrdf/oxttl/spargebra, deliberately independent of the pgrx build) with ALL harness orchestration in the binary — the bash per-fixture wiring from the prototype was a spike and is fully eliminated per the issue's course-correction 2. tests/w3c-sparql/run.sh is now env-resolution + cargo build + ONE invocation of 'pgrdf-oracle run'. The comparator/eval core (canonical JSONL row shapes, numeric-lexical canonicalization, bag equivalence, blank-node isomorphism via backtracking bijection) and the 53 per-fixture eligibility markers carry over from the prototype handoff unchanged. New in this cut: - 'pgrdf-oracle run' subcommand: fixture walk (data.ttl|setup.sql discovery, fixtures/ reserved for the official W3C rdf-tests suite, course-correction 1), byte-for-byte bash-parity SQL assembly (verified: identical graph ids for all 53 fixtures), engine execution via --engine-cmd (SQL on stdin -> psql rows on stdout, so the crate needs no Postgres linkage), golden gate (bag-equivalent: BOTH sides sorted — goldens on disk predate the C-locale ACCEPT convention; fixture 35 caught this), marker-driven differential pass, verdict reporting, exit-code gating. - The differential is now ALWAYS ON (no ORACLE=1 opt-in): an eligible divergence fails the run; the two known divergences the prototype found on day one are marker-tracked (32 -> #54 CONSTRUCT set semantics, 08 -> #55 HAVING alias) and non-fatal until adjudicated, with ORACLE-RESOLVED? staleness reporting if they start matching. The oracle also second-opinions every ACCEPT-regenerated golden — closing the exact blind spot this issue was opened for. - New fixtures 52/53 (expression surface #51, aggregates-over- expressions #50) marked eligible and oracle-verified. - ci.yml regression job: rust toolchain + cargo cache for the oracle build; the w3c step is now golden + differential in one gate. Verified against an isolated container (main-built .so): 53/53 golden, oracle 37 match / 2 known-divergence / 0 diverge / 14 ineligible-skip, exit codes correct both ways; regression 100/100 untouched; crate: 30 unit + 5 CLI tests, fmt + clippy -D warnings clean. Closes #17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes #17. Lands the differential oracle as the standalone
pgrdf-oraclecrate (tests/oracle/— spareval + oxrdf, no pgrx linkage) with all harness orchestration in the binary: the prototype's bash per-fixture wiring was a spike and is fully eliminated (course-correction 2).tests/w3c-sparql/run.shis now env-resolution +cargo build+ onepgrdf-oracle runinvocation.The comparator/eval core (canonicalization, bag equivalence, blank-node isomorphism) and the 53 per-fixture markers carry over from the prototype handoff unchanged — credit to the prototype session; findings #54/#55 came from its first run.
Key properties
ORACLE=1opt-in): aneligibledivergence fails the run. Known divergences are marker-tracked (32 → #54,08 → #55) and non-fatal until adjudicated, withORACLE-RESOLVED?staleness reporting if they start matching. The oracle also second-opinions everyACCEPT-regenerated golden — the exact blind spot spareval differential test harness (W3C oracle) #17 was opened to close.--engine-cmd: SQL on stdin → psql rows on stdout), so the crate builds anywhere;fixtures/remains reserved for the official W3C rdf-tests suite (course-correction 1, follow-up).Verification
-D warnings(crate)New fixtures 52 (#51) and 53 (#50) are
eligibleand oracle-verified — the scoring-loop expression surface now sits under a standing second judge.