feat(db-sync): support configurable layouts and schema policies

[Scitz0]

Overview

This PR lets Midnight use supported Cardano db-sync layouts without requiring Midnight to own or modify the database schema.

It adds consistent support for:

• Inputs stored in tx_in or represented by tx_out.consumed_by_tx_id.
• Addresses stored inline in tx_out.address or normalized through the address table.
• Midnight-managed indexes, read-only verification, or operator-managed schema setup.

This allows operators to reuse an existing db-sync deployment instead of maintaining another full copy or granting the node DDL privileges.

A read-only consumed + address_table deployment can use:

db_sync_tx_input_mode = "consumed"
db_sync_address_mode = "address_table"
db_sync_schema_mode = "verify"

What changed

Configuration

Setting	Values	Default
db_sync_tx_input_mode	auto, tx_in, consumed	auto
db_sync_address_mode	inline, address_table	inline
db_sync_schema_mode	apply, verify, skip	apply

Equivalent DB_SYNC_* environment variables are supported. The selected layout is validated once and shared by all data sources. Explicit modes fail early when required structures are missing. auto selects a populated input representation and rejects an ambiguous empty dual-layout database rather than guessing.

Schema policy

Mode	Behavior
apply	Accept compatible indexes, create missing indexes concurrently, and apply cNight autovacuum tuning.
verify	Read-only catalog validation with one error listing missing or unusable indexes; autovacuum differences only warn.
skip	Skip index and autovacuum management while still validating the query layout.

Indexes are matched structurally rather than by name. Compatible operator-owned indexes are accepted when their access method and leading keys satisfy the query, including wider composite and descending B-tree indexes. This avoids unnecessary duplicate indexes.

Query and node integration

The resolved layout is now used consistently by Partner Chains candidate and bridge queries, Midnight candidate queries, cNight registration/deregistration/create/spend/bound queries, federated-authority observation, and ICS/reserve genesis lookups.

Only fixed internal SQL fragments are selected dynamically; values remain bound parameters. Datum and metadata queries also support retained text-backed JSON through jsonb casts, and multi-asset policy/name matching now uses raw bytes and ordinary B-tree indexes.

Normal startup resolves the layout once, applies the selected schema policy, and passes the result to all data sources. The broader cNight genesis path manages its additional indexes and autovacuum settings. ICS, reserve, and federated-authority paths validate the layout without installing their own manifest.

Changed areas

Area	Main files
Shared configuration and index verification	partner-chains/toolkit/utils/db-sync-sqlx/*
Partner Chains candidates and bridge	partner-chains/toolkit/data-sources/db-sync/src/*
Midnight candidate, cNight, and federated queries	primitives/mainchain-follower/src/*
Node startup and genesis wiring	node/src/*, res/cfg/default.toml
Documentation, tests, and release metadata	docs/*, scripts/genesis/README.md, changes/*, Cargo.lock

db-sync-sqlx adds serde and log; the node directly consumes its shared types. testcontainers-modules is a dev-only dependency for PostgreSQL layout tests and was already present in the workspace lock graph.

Compatibility and limitations

• Defaults remain auto / inline / apply, preserving behavior for initialized standard db-sync deployments.
• Existing high-level constructors and compatibility helpers remain available.
• Layout validation confirms schema shape, not historical completeness. Required ledger, multi-asset, datum/Plutus, and bridge metadata history must still be retained; disabled or pruned required data remains unsupported.
• This is a node-client/database-query change with no runtime, consensus, or on-chain storage changes.
• This covers the configurable-layout and read-only schema-management portions of Test various configurations to increase dbsync performance #1160, not its broader pruning, filtering, benchmarking, or performance work.

SQLx cache note

The pre-existing .sqlx/query-*.json files are intentionally unchanged. The affected layout-dependent queries now use runtime-selected SQL and no longer consume those entries. Keeping them avoids unrelated generated-cache churn; they can be reconciled later through the normal SQLx cache workflow.

☑ TODO before merging

• Add the final PR URL to both change files in a follow-up commit.
• Add the DCO remediation commit for the existing unsigned commit.
• Confirm and apply the required node minor-version bump.
• Confirm PostgreSQL container tests pass in CI.
• Add the bot:ai-assisted label.
• Ready

📌 Submission Checklist

• All commits are signed off (git commit -s) for the DCO
• Changes are backward-compatible (or deliberate changes are identified above)
• Pull request description explains why the change is needed
• Self-reviewed the diff
• I have included node and toolkit change files
• If the changes introduce a new feature, I have bumped the node minor version
• Update documentation (if relevant)
• Updated AGENTS.md if build commands, architecture, or workflows changed — not required; existing guidance remains accurate
• No new todos introduced

🧪 Testing Evidence

Passed locally:

• Formatting, patch-integrity, workspace compilation, and strict Clippy checks.
• Node default-configuration, configuration parsing, and layout-aware manifest tests.
• Compilation and linking of the affected Partner Chains and primitive PostgreSQL test targets.

PostgreSQL tests cover structural index verification and all four input/address combinations, including consumed + address_table after physically removing tx_in and tx_out.address. They compiled locally but were not executed because Docker was unavailable; CI must confirm them.

• Additional tests are provided (if possible)

🔱 Fork Strategy

• Node Runtime Update
• Node Client Update
• Other
• N/A

This can be deployed as a node-client update without a runtime fork or network reset. Existing deployments may retain the defaults; read-only deployments install the documented indexes before selecting verify.

Links

• Related to #1160.
• Motivating deployment discussion.
• cardano-db-sync configuration reference.

This intentionally does not use Closes #1160 because that issue has a broader scope.

[CLAassistant]

All committers have signed the CLA.