feat(nds): update registry arity to 2

[emturner]

• Closes TZX-211

What

Ensure registry is included in database proof size limit, and update arity of vector to 2.

Why

Bundling these three changes together keeps the net effect on the number of keys/values we support in proofs (as reported by proof_size.rs) visible in one place. As the bounds are now tighter and more accurate, the overall effect is to raise confidence in the figures.

Impact on database sizes

A database proof never ships on its own, so it is now measured against DATABASE_MAX_PROOF_SIZE (15288 B) rather than the full 16 KiB MAXIMUM_PROOF_SIZE: the difference is the registry envelope that wraps it (1096 B for the deepest registry registry_resize admits).

Against that reservation, the binding operation gets 677 B cheaper to model — 644 B from capping value_open at MAX_PROOF_LENGTH, and 33 B from no longer charging the terminal search-path node's blinded data child when the value is opened. The arity change takes a further 546 B off the envelope itself. That leaves 419 B unpaid, so the binding operation loses one level of AVL depth; because minimal AVL node counts are Fibonacci, a level is roughly 1.6x the number of keys:

┌─────────────────────────────────────────┬────────────────────────┬───────────────────────┐
│                operation                │         before         │         after         │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ Delete (existing)                       │ d=53 → 225,851,433,715 │ d=50 → 53,316,291,171 │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ Set / Write (fresh insert)              │ d=48 → 20,365,011,072  │ d=45 → 4,807,526,974  │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ Read / Write (max chunk, 64 MiB value)  │   d=35 → 39,088,167    │   d=34 → 24,157,815   │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ Set (replacing a 64 MiB value)          │   d=40 → 433,494,435   │   d=36 → 63,245,984   │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ ValueLength / Exists / Delete (missing) │ d=49 → 32,951,280,097  │ d=46 → 7,778,742,047  │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ Hash                                    │       unbounded        │       unbounded       │
├─────────────────────────────────────────┼────────────────────────┼───────────────────────┤
│ all operations                          │   d=35 → 39,088,167    │   d=34 → 24,157,815   │
└─────────────────────────────────────────┴────────────────────────┴───────────────────────┘

The reduction is a correction rather than a loss of capability. The before column was measured two ways that both overstated it: against a ceiling that ignored the registry wrapper the proof has to fit inside, and with a value_open that charged every touched page a full root-to-leaf branch of blinded siblings. MAX_PROOF_LENGTH (4097 B) is the worst case actually derivable for a three-page read or write over a 64 MiB leaf, sharing included.

For the record, d=34 now clears its ceiling by 5 bytes, and reaching d=35 would need another 324 B — out of the 1096 B envelope, or out of the 329 B per search-path level, which is dominated by the 255-byte key leaf.

How

Tighten the bounds, by taking into account the true maximum size of Byte proofs, include the registry proof in the database proof-size limit, and reduce the arity of the registry.

Manually Testing

make all

Regressions

Updated due to arity change of Vector.

Tasks for the Author

• Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
• Eliminate dead code and other spurious artefacts introduced in your changes.
• Document new public functions, methods and types.
• Make sure the documentation for updated functions, methods, and types is correct.
• Add tests for bugs that have been fixed.
• Explain changes to regression test captures when applicable.
• Write commit messages in agreement with our guidelines.
• Self-review your changes to ensure they are high-quality.
• Complete all of the above before assigning this MR to reviewers.

[github-actions]

Benchmark results for revision dad9179:

Metric	Duration	TPS
Mean	1.507085078s	26.541
Worst	1.51659872s	26.375
Best	1.501803521s	26.635
Standard Deviation	±3.648743ms	±0.064

Full results

Run	Transfers	Duration	TPS
1	40	1.51659872s	26.375
2	40	1.505971958s	26.561
3	40	1.510174274s	26.487
4	40	1.504842323s	26.581
5	40	1.510859988s	26.475
6	40	1.505244907s	26.574
7	40	1.508706691s	26.513
8	40	1.507833989s	26.528
9	40	1.508353256s	26.519
10	40	1.503953089s	26.597
11	40	1.505441497s	26.570
12	40	1.509854179s	26.493
13	40	1.511960171s	26.456
14	40	1.509942985s	26.491
15	40	1.505928408s	26.562
16	40	1.501803521s	26.635
17	40	1.502973178s	26.614
18	40	1.503603009s	26.603
19	40	1.505546877s	26.568
20	40	1.502108537s	26.629

Compare the results above with those for the default branch.

[github-actions]

Durable storage space metrics for revision dad9179, at pull request
scale. The scale itself lives in durable-storage/Makefile.

Metric	Value	Unit
dead_node_bytes_per_commit	1.48	MiB
dead_node_fraction	37.4	%
repo_bytes_per_commit	15.25	MiB
checkpoint_shared_fraction	76.8	%
checkpoint_shared_fraction_steady	91.6	%

Full report

repository: /tmp/nix-shell.EEiL0i/.tmpZPYU9V
shape: 1 database(s) x 100000 keys x 32 B values, 10 commits x 1000 modified keys
prepopulated 100000 keys in 0.6s
committing the base state...
base state committed as 74a7f22c056f09f3c367b72a7eaec754cf238036fd6333a8583fcfb42241ea7d in 1.0s

 commit    blob MiB    live MiB    dead MiB   dead %   value MiB    disk MiB  commit ms  commits MiB     new MiB  shared %  rewrote MiB
      0        24.8        24.8         0.0     0.0%        15.4        42.6          0         42.5           -         -            -
      1        26.3        24.8         1.5     5.6%        15.4        44.3        118         44.2         1.7     96.1%          0.0
      2        27.7        24.8         2.9    10.6%        15.4        47.1        138         47.0         2.7     94.2%          0.0
      3        29.2        24.8         4.4    15.2%        15.4        97.8        135         50.6         3.6     92.9%          0.0
      4        30.7        24.8         5.9    19.2%        15.4       102.2        143        102.1        51.5      0.0%         50.5
      5        32.2        24.8         7.4    22.9%        15.4       107.3        163        107.2         5.1     91.0%          0.0
      6        33.7        24.8         8.9    26.3%        15.4       113.0        182        112.9         5.7     90.8%          0.0
      7        35.1        24.8        10.3    29.4%        15.4       156.6        195        119.7         6.7     90.1%          0.5
      8        36.6        24.8        11.8    32.3%        15.4       163.4        186        163.2        43.5     28.4%         51.2
      9        38.1        24.8        13.3    34.9%        15.4       170.7        207        170.5         7.3     89.3%          0.0
     10        39.6        24.8        14.8    37.4%        15.4       195.1        230        178.3         7.8     89.8%          0.0

over 10 commit(s):
  dead node data grew by 14.8 MiB, 1.48 MiB per commit
  blob column family grew by 14.8 MiB, 1.48 MiB per commit
  59689 node(s) written, 5968 per commit, averaging 260 B stored per node
  history pins 178.3 MiB (142.8 MiB Merkle, 35.3 MiB values, 0.1 MiB other), growing 13.6 MiB per commit
  80.1% of what the history pins is Merkle node data, which a shared store would stop duplicating per commit
  sharing: 76.8% of the bytes checkpoints pinned were already on disk (449.4 MiB of 585.0 MiB), adding 13.6 MiB in 2 new file(s) per commit
  3 of 10 measured commit(s) followed a compaction; without them sharing is 91.6%, so compaction costs 14.8 points of it, rewriting 102.3 MiB the earlier checkpoints still pin
  levels at the last commit:
    L0: 7 file(s), 22.5 MiB
    L6: 2 file(s), 53.3 MiB
  37.4% of the blob column family is now dead (14.8 MiB of 39.6 MiB)
  repository occupies 195.1 MiB across 11 commit directories (702.5 MiB apparent, and hard links save 507.7 MiB)

simulated directory-level collection: removed 9 database commit(s) and 9 manifest(s)
  repository went from 195.1 MiB to 135.2 MiB, freeing 59.9 MiB
  retained history now occupies 118.4 MiB
  still dead and now unreachable by any directory deletion: 14.8 MiB of node data (37.4% of the surviving blob column family)

These figures are for catching a regression before it merges. Compare them with
the longer run on the default branch, which measures more commits.

[codecov]

Codecov Report

❌ Patch coverage is 91.17647% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.88%. Comparing base (9591718) to head (a0fddd8).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
durable-storage/src/test_helpers/proof_size.rs	90.62%	5 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1190      +/-   ##
==========================================
+ Coverage   89.85%   89.88%   +0.03%     
==========================================
  Files         152      153       +1     
  Lines       32554    32731     +177     
  Branches    32554    32731     +177     
==========================================
+ Hits        29250    29420     +170     
- Misses       2290     2296       +6     
- Partials     1014     1015       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.