ledger-v9: port to the Midnight 2.x line (compactc 0.34.0, ledger 9, midnight-js 5)

[acedward]

DO NOT MERGE UNTIL LEDGER v9

This PR is meant to stay OPEN

ledger-v9 is a long-lived branch, not a change queued for merge. main remains the Midnight 1.x / preview line — it is what https://shielded-night.pages.dev runs against the locked contract 80b89b9a…bee1396. This branch is the 2.x port, and it is merged into main only when the Midnight network itself moves to 2.x. The PR exists so the branch is reviewable and its head is a citable, fetchable SHA.

Not a breaking change to main: nothing on main is touched, and no published deployment changes behaviour.

The two lines are kept apart deliberately: one dependency tree cannot hold both ledger wasm modules — two copies give two class identities and every cross-copy instanceof fails. That is the same hazard both package.json overrides blocks already exist to prevent.

	main	ledger-v9 (this branch)
Toolchain	compactc 0.31.1, language 0.23	compactc 0.34.0, language 0.26.0
Runtime	compact-runtime 0.16.0	0.19.0
Ledger	@midnight-ntwrk/ledger-v8 8.1.0	@midnightntwrk/ledger-v9 1.0.0-rc.3 (new scope, no hyphen)
midnight-js	4.1.1	5.0.0-beta.7
wallet-sdk	@midnightntwrk/wallet-sdk ^1.2.0	2.0.0-beta.2
compact-js	^2.5.1	2.5.5-rc.8
address-format	@midnight-ntwrk/… 3.1.0	@midnightntwrk/… 4.0.0-beta.2
dapp-connector-api	4.0.1	4.0.1 (unchanged)
Devnet	node 1.0.0 / indexer 4.3.3 / proof-server 8.1.0	2.0.0-rc.4 / 4.4.0-rc.1 / 9.0.0-rc.5

The contract needed one line changed

pragma language_version 0.23; → 0.26; and nothing else. None of the language 0.25/0.26 tripwires bit: no as Field casts (Field is no longer a supertype of Uint, but this contract's arithmetic was already annotated), no point/scalar-type import moves (JubjubPoint/Secp256k1* are unused), no stdlib renames — receiveUnshielded, sendUnshielded, receiveShielded, sendImmediateShielded, shieldedBurnAddress, mintShieldedToken, tokenType, kernel.self() and nativeToken() all kept their signatures.

That includes the part most at risk: the two ATOMIC circuits (convertToShielded, convertToUnshielded), which net a shielded and an unshielded move inside a single segment.

All 44 files under src/managed/keys/ and src/managed/zkir/ are byte-identical to the 0.31.1 build. The verifier keys of the deployed preview contract stay valid across the toolchain bump; only the generated TypeScript moved. Two independent full compiles of this source reproduce src/managed/ byte-for-byte.

No --feature-zkir-v3: the emitted keys are tagged midnight:verifier-key[v6] and the zkir is version 2, so the plain proof-server:9.0.0-rc.5 proves them — the _experimental build is not needed.

New in compactc 0.33+: compiler/contract-manifest.json. It is now load-bearing rather than decorative — midnight-js 5's FetchZkConfigProvider verifies every artifact against it with verify defaulting to 'require' (fail-closed), so any deployment must serve the whole src/managed/ tree, manifest included.

Why midnight-js 5.0.0-beta.7 and not beta.6

The generated contract opens with __compactRuntime.checkRuntimeVersion('0.19.0'), and that guard rejects a minor mismatch outright while the major is 0. @midnight-ntwrk/midnight-js-protocol@5.0.0-beta.6 pins compact-runtime 0.18.0-rc.1; beta.7 pins 0.19.0-rc.0. So beta.6 — the version the sibling ledger-v9 ports in this org use — cannot run compactc 0.34.0 output.

compact-runtime is pinned to the released 0.19.0 rather than beta.7's 0.19.0-rc.0: the two published tarballs differ in three lines (the version string, its source map, and package.json) and are otherwise identical, and the runtime guard strips the prerelease suffix. The overrides block collapses them to one copy.

@midnightntwrk/ledger-v9 is pinned at 1.0.0-rc.3 because midnight-js-protocol beta.7 and every wallet-sdk@2.0.0-beta.2 subpackage pin exactly that. 1.0.0-rc.4 exists but nothing in the stack references it, so adopting it would fork the ledger into two wasm copies.

What the port actually cost

@midnight-ntwrk/ledger-v8 → @midnightntwrk/ledger-v9 is a pure specifier rename at all 11 import sites; no ledger call had to change. The real work was elsewhere:

• compact-runtime 0.19 made every circuit and Contract.initialState async, and collapsed currentQueryContext / currentPrivateState / currentZswapLocalState / gasCost under a single callContext; createCircuitContext also gained the executing circuit's id as its first argument. The simulator now has a static create() factory and async accessors, the unit tests await, and negative cases became await expect(...).rejects.toThrow(...). 78/78 unit tests still pass — the same count as main, so nothing was dropped in the conversion.
• wallet-sdk 2: createKeystore takes { kind, secret } instead of raw bytes (ledger-v9 turned SigningKey/SignatureVerifyingKey/Signature into { tag, value } records). kind: 'schnorr' is what reproduces the same addresses the v8 line derived — anything else compiles fine and silently derives a different, unfunded wallet. TransactionHistoryCommonSchema was renamed to TransactionHistoryEntryCommonSchema, and SignSegment is now async, so the keystore's own signDataAsync replaces the hand-rolled wrappers.
• Frontend: FetchZkConfigProvider's second argument is an options bag; ZswapChainState.postBlockUpdate gained a required retentionDuration (3600n, the value midnight-js 5 uses itself — note LedgerState.postBlockUpdate is a different method that did not change this way). resolve.dedupe follows the new package names, and the build still emits exactly one ledger wasm and one onchain-runtime wasm.

One thing worth a reviewer's eye: ProvingProvider.lookupKey

ledger-v9 widened ProvingProvider from { check, prove } to { check, prove, lookupKey }, and midnight-js 5's createProofProvider requires the wider shape. @midnight-ntwrk/dapp-connector-api has not followed — neither 4.0.1 nor 4.1.0-beta.1 declares lookupKey (both tarballs were read to confirm). So a v9 wallet that does implement it still returns a value the connector types describe without it.

frontend/src/lib/providers.ts handles this in withLookupKey(): the wallet's lookupKey is used when present, and otherwise the dApp serves it from its own zkConfigProvider — the identical public artifacts it already serves at /contract/compiled/shielded-night/ and already handed the wallet through asKeyMaterialProvider() one line earlier. check and prove are untouched, so proving stays wallet-owned and the page still never names a proof server. Which lane was taken is logged rather than silent.

Devnet stack

envs/docker-compose-dynamic.yml moves to the 2.x triple. Three things there are easy to get wrong and are commented in place:

• the node volume must be at /node, not /data — the 2.x image sets BASE_PATH=/node/chain, so a /data volume loses the chain db without an error;
• indexer-standalone:4.4.0-rc.1 is published for linux/amd64 only, so it runs emulated on Apple Silicon (hence the longer healthcheck budget), and its 4.3.x APP__INFRA__{STORAGE,PUB_SUB,LEDGER_STATE_STORAGE}__PASSWORD keys are gone;
• node-data and indexer-data are wiped together: a v8→v9 chain cannot be upgraded in place, so a fresh genesis against a stale indexer db is worse than no db.

CI

COMPACT_VERSION: '0.34.0' for all four jobs, and on.push.branches now includes ledger-v9 so the branch gets full CI (unit, frontend, byte-exact rebuild, integration on the 2.x triple) on every push, not only inside this PR.

Measured: the plain proof-server 9.0.0-rc.5 proves this, and the atomic circuits settle on ledger 9

The first CI run on this branch was killed by timeout-minutes: 60 partway through transient.test.ts. It was not stuck — it had passed 15 of the 17 integration tests against node 2.0.0-rc.4 / indexer 4.4.0-rc.1 / proof-server 9.0.0-rc.5 and was still going:

✓ shielded-night.security.test.ts       (5 tests)  1373448ms
✓ shielded-night.test.ts                (8 tests)  1101204ms
✓ shielded-night.multiparty.test.ts     (1 test)    753889ms
✓ maintenance-authority.test.ts         (1 test)    153846ms

Every one of those proves and submits a real transaction, so compactc 0.34.0's zkir-v2 artifacts are proven by the plain proof-server image — the _experimental build is not needed.

It also settles the risk this port was expected to hit. The passing security test "combined circuits: forged or double-spent coins cannot drain the reserve; nonce reuse cannot re-mint" is an on-chain test that calls convertToShielded and then convertToUnshielded three times and asserts the caller's NIGHT comes back — i.e. a shielded and an unshielded move netted inside one segment proves, applies and balances on ledger 9. No two-step-only fallback is needed.

Two other things it proves that typechecking could not: kind: 'schnorr' really is the keystore envelope that reproduces the funded v8 addresses (every test runs on a genesis-funded wallet derived through it), and the SigningKey → { tag, value } change is threaded correctly — maintenance-authority.test.ts signed a MaintenanceUpdate the chain accepted.

Why the timeout moved to 150

The suite is 1.2–1.6× slower per file on the 2.x triple, and 60 was already thin on main (its last green run spent 50.3 of its 60 minutes inside vitest):

File	main (1.x)	ledger-v9	ratio
security.test.ts (5)	1078.7 s	1373.4 s	1.27×
shielded-night.test.ts (8)	885.7 s	1101.2 s	1.24×
multiparty.test.ts (1)	471.8 s	753.9 s	1.60×
maintenance-authority.test.ts (1)	153.8 s	153.8 s	1.00×

The 1.00× row is the diagnostic one: that file is dominated by fixed waits rather than proving and did not move at all. Only the proving-bound files stretched, which is what says this is per-transaction cost on the 2.x stack rather than a hang. No test, assertion or gate was weakened, skipped or removed to fit the budget.

---

Update 2026-09-03: merged main @ f7fcefa

Merged main @ f7fcefa7921bf2c3f634871f9ad3aa3a32251af0 into this branch (merge commit 36caf59), bringing in PR #11 (frontend/.env PREPROD_ADDRESS=e354e672…) and PR #12 (scripts/verify-deployment.ts --allow-unlocked + scripts/verify-args.ts + tests + docs, plus main's own bun-1.4 lockfile regeneration).

Conflicts were in package.json / frontend/package.json (overrides) and both bun.lock files. This branch's own v9 pins win (@midnightntwrk/ledger-v9 1.0.0-rc.3, @midnight-ntwrk/compact-runtime 0.19.0, @midnightntwrk/onchain-runtime-v4 4.0.0-rc.3) — main's v8-line overrides (compact-js 2.5.1, ledger-v8 8.1.0, onchain-runtime-v3 3.0.0) do not apply to this dependency graph and were not copied over; the //overrides comment now says so explicitly. Both lockfiles were not hand-merged: deleted and regenerated with bun 1.4.0, then verified to hold exactly one copy each of the v9 packages and zero v8/onchain-runtime-v3 copies in both trees.

Locally verified: byte-exact bun run compact rebuild, bun install --frozen-lockfile clean on both trees, 95/95 unit tests (incl. the 17 new verify-args tests), root + frontend typecheck clean, frontend build clean (1469 modules, one ledger wasm, one onchain-runtime wasm, PreProd + Local(undeployed) networks configured in the built bundle). CI dispatched on the push to 36caf59.