Skip to content

bundle full schema and constraint catalogs in partition snapshots#61

Merged
fabracht merged 1 commit into
mainfrom
bundle-broadcast-catalogs
May 19, 2026
Merged

bundle full schema and constraint catalogs in partition snapshots#61
fabracht merged 1 commit into
mainfrom
bundle-broadcast-catalogs

Conversation

@fabracht
Copy link
Copy Markdown
Contributor

Summary

Closes #57 and #58. After PR #56 surfaced "constraints don't reach all nodes uniformly" as a follow-up, this PR fixes the late-join half of that gap.

Schemas and constraints already broadcast to every alive node on write (replication_ops.rs:81-85 and :345-369 — likely added in #54 after the issues were filed). The remaining hole is the snapshot path: SchemaStore::export_for_partition and ConstraintStore::export_for_partition filter entry.partition() == partition, so a joining node only ever received entries whose schema_partition(entity) happened to hash into a partition it owned. Catalog entries under any other partition were unreachable through snapshots — a node could rebalance into the cluster missing most of the catalog, and a freshly-joined 4th node empirically held 0 constraints in the #57 trace.

Both stores now expose export_all(), which serializes every entry regardless of partition using the same wire format as export_for_partition (so import_schemas / import_constraints are unchanged). StoreManager::export_partition uses these for DB_SCHEMA and DB_CONSTRAINT. Catalogs are small in-memory state and import is idempotent overwrite, so re-applying on every partition snapshot a joining node fetches is safe.

A shared private serialize_entries helper in each store removes the duplicated count-prefixed loop between the per-partition and full-catalog exports.

Test plan

  • cargo test -p mqdb-cluster --lib — 3 new tests pass:
    • schema_store::export_all_carries_every_schema_regardless_of_partition
    • constraint_store::export_all_carries_every_constraint_regardless_of_partition
    • partition_io::partition_snapshot_carries_full_schema_and_constraint_catalog — multi-entity setup spanning many partitions; asserts dst has the full catalog after a single-partition snapshot, plus full counts match source.
  • partition_io::export_import_roundtrip_covers_all_db_stores updated — previously asserted per-partition isolation for schemas/constraints, now asserts they arrive regardless of partition.
  • cargo make clippy — clean (pedantic, all targets + wasm).
  • cargo make format-check — clean.
  • cargo make test — full workspace clean.
  • Pre-commit hook (format-check + clippy) ran on the commit and passed.
  • examples/cluster-rebalance-stores/run.sh actually run twice with an enterprise license generated locally:
    • Pre-fix (per constraints not replicated uniformly across cluster nodes #57 trace): schemas/constraints visible only on a subset of nodes; FK cascade left up to 12 of 12 eligible children alive.
    • Post-fix run 1: 10/11 hard assertions pass; all 5 OBSERVATION lines OK (schema get posts/comments via node 4, all duplicates rejected, FK orphan rejected); FK cascade removed 17 of 19 eligible children. The single hard-assertion failure ("18 of 20 extra child comments") is a confirmed-flaky cluster-warmup transient unrelated to schema/constraint replication.
    • Post-fix run 2: 11/11 hard assertions pass; all 5 OBSERVATIONs OK; FK cascade removes all 21 of 21 eligible children via node 4.

Out of scope

Remaining FK-cascade survivors that still show up under hash-unlucky distributions are #22 (cluster cascade does not filter by ownership on remote nodes) — separate follow-up.

@fabracht fabracht merged commit 27d36a3 into main May 19, 2026
5 checks passed
@fabracht fabracht deleted the bundle-broadcast-catalogs branch May 19, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

constraints not replicated uniformly across cluster nodes

1 participant