Skip to content

Switch to Rust core#70

Merged
valentinfernandez1 merged 45 commits into
mainfrom
codex/bulletin-preimage-in-core-dotli
Jul 24, 2026
Merged

Switch to Rust core#70
valentinfernandez1 merged 45 commits into
mainfrom
codex/bulletin-preimage-in-core-dotli

Conversation

@pgherveou

@pgherveou pgherveou commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Port the dot.li web host to the shared TrUAPI Rust core

This supersedes #54 and keeps its Rust-runtime port while adding the Bulletin preimage path now owned by the core. dot.li moves off the @novasamatech/host-api stack and onto the shared TrUAPI Rust core. Protocol logic previously maintained in TypeScript now lives once in truapi-server (Rust -> WASM); the web host becomes a byte transport plus typed platform callbacks.

Architecture

A product iframe speaks TrUAPI over a MessageChannel. The host forwards SCALE frames to a Web Worker running truapi-server WASM without decoding the protocol. The core executes protocol logic and calls typed truapi-platform callbacks for browser capabilities.

Product iframe --MessageChannel--> host bridge (byte pipe) --> Web Worker (truapi-server WASM)
                                                                          | typed callbacks
                                                                          v
                                                            dot.li platform capabilities
  • Core owns: wire framing, dispatch, subscriptions, permission gating, auth/SSO state, session interpretation, signing orchestration, Bulletin preimage submission, and the chainHead runtime.
  • Host owns: storage, navigation, confirmation UI, physical chain providers and RPC transport, theme, preimage content lookup, and product identity/pairing configuration.

The same core is embedded by native hosts over UniFFI, keeping protocol behavior and wire formats aligned across platforms.

Changed surface

  • Removes packages/auth, the old Nova container/statement-store adapters, allowance-signer bridge, wallet queue, and related @novasamatech/* dependencies.
  • Adds generated typed host callbacks backed by @parity/truapi-host and @parity/truapi.
  • Rewrites the bridge around the worker-backed Rust core.
  • Keeps chain ownership in dot.li while moving JSON-RPC shaping, chainHead state, transaction construction, signing, dry-run, and Bulletin submission into Rust.
  • Keeps Bulletin available to the core's internal gateway provider without advertising it through the sandboxed dApp chain gate.
  • Keeps preimage content lookup in the host; returned bytes are verified by the core.
  • Preserves in-flight Rust callback requests when granting non-device permissions; browser-policy device grants still reload the product iframe.

Permission grants are core-owned state. The dot.li topbar admin screen projects them into browser-specific iframe permissions and reload behavior rather than maintaining a second protocol policy store.

Legacy product compatibility

Existing applications such as host-playground still use the Nova host wrapper. Nova exposes synthetic PAPI chain-head follow IDs such as follow_0, while the Rust core intentionally requires the exact wire request ID from the follow subscription start frame. Passing the synthetic ID through makes otherwise valid legacy header, body, storage, call, unpin, continue, and stop-operation requests fail core validation.

The legacy window bridge now tracks active follow wire IDs per genesis hash and rewrites only those follow-bound legacy frames. Modern MessagePort products are unchanged, and strict subscription-ID validation remains in the shared Rust core. Keeping this compatibility at the deprecated boundary also avoids a new TrUAPI package release solely for a host-specific migration shim.

Packages

This branch depends on the published @parity/truapi@0.4.0 and @parity/truapi-host@0.1.0. bun run link:truapi swaps them for symlinks to a sibling TrUAPI checkout (and unlink:truapi restores the published packages), so the parent truapi repo's make e2e-dotli runs against this branch directly while a direct dotli checkout installs and tests the published packages.

Scope

The primary PR contains only the Nova-stack removal, the typed Rust-runtime host architecture, the Bulletin preimage integration, and the permission-lifecycle behavior required to keep Rust callback requests alive. Everything else is consolidated in the stacked follow-up #77: preimage lookup retries, protocol-broker hardening, login-failure UX copy, permission-prompt rate limiting, allowance-key at-rest encryption, the per-frame wire debug tap, debug-panel fixes, resolver dependency hygiene, and chainSend flush error surfacing.

Deferred architecture work: SharedWorker topology, eliminating the per-CID *.app.dot.li sandbox iframe, and moving content fetching into Rust behind the PreimageHost boundary.

Left before merge

Related

Verification

Primary branch:

  • bun install --frozen-lockfile
  • bun run format:check
  • bunx --bun turbo run lint --force
  • bunx --bun turbo run typecheck --force
  • bunx --bun turbo run test --force
  • bunx --bun turbo run build:prod --force
  • bun audit

Compatibility patch:

  • Dotli format, lint, typecheck, unit tests, and production build pass locally.
  • Parent TrUAPI make test passes.
  • Two signer-backed make e2e-dotli runs each completed 43 methods successfully, including every Chain/* legacy compatibility method. The sole failure in both runs was Preimage/lookup_subscribe: the external signer-bot authority request timed out at its 45-second boundary. A subsequent authority request in each run succeeded, isolating the failure from the chain bridge patch.

@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Bundle Size Report

Chunks over 500 KB:

File Raw Brotli Gzip
host/assets/paseo.smol-DboPaEh1.json 1.84 MB 941.7 KB 1019.4 KB
host/assets/paseo-people-next.smol.json 3.36 MB 1.68 MB 1.82 MB
host/assets/previewnet.smol.json 1.88 MB 181.4 KB 353.0 KB
host/assets/smoldot.js 2.98 MB 2.21 MB 2.22 MB
host/assets/smoldot_worker.js 2.95 MB 2.21 MB 2.21 MB
host/assets/wasm/web/truapi_server_bg.wasm 1.96 MB 617.3 KB 812.7 KB
Total 15.90 MB (+5.11 MB) 8.11 MB (+2.88 MB) (-49%) 8.74 MB (+3.08 MB)
All files
File Raw Brotli Gzip
host/.well-known/apple-app-site-association 738 B 738 B 738 B
host/.well-known/assetlinks.json 1.3 KB 317 B 391 B
host/assets/bridge.js 71.8 KB (+66.4 KB) 19.6 KB (+17.8 KB) 22.5 KB (+20.4 KB)
host/assets/browser.js 22.9 KB (-10 B) 7.6 KB (-6 B) 8.6 KB (-9 B)
host/assets/client.js 100.1 KB (+6.9 KB) 29.4 KB (+2.0 KB) 32.4 KB (+2.2 KB)
host/assets/dist.js 39.0 KB (+14.2 KB) 12.9 KB (+5.2 KB) 14.6 KB (+6.0 KB)
host/assets/dotli-debug-bus.js 646 B (+151 B) 646 B (+151 B) 646 B (+151 B)
host/assets/get-sync-provider.js 2.8 KB 1.1 KB 1.2 KB
host/assets/hex.js 153 B 153 B 153 B
host/assets/index.js 137.0 KB (+30.4 KB) 37.5 KB (+8.4 KB) 43.8 KB (+9.9 KB)
host/assets/index.css 44.8 KB (-3.1 KB) 7.1 KB (-382 B) 7.9 KB (-423 B)
host/assets/manifest.js 22.5 KB (-132 B) 7.2 KB (-29 B) 7.9 KB (-60 B)
host/assets/panel.js 72.5 KB (-12.3 KB) 19.7 KB (-3.3 KB) 22.3 KB (-4.0 KB)
host/assets/paseo.smol-DboPaEh1.json 1.84 MB 941.7 KB 1019.4 KB
host/assets/paseo-people-next.smol.json 3.36 MB 1.68 MB 1.82 MB
host/assets/paseo.smol.json 25.4 KB 4.9 KB 5.6 KB
host/assets/previewnet.smol.json 1.88 MB 181.4 KB 353.0 KB
host/assets/resolve.js 128 B (-24 B) 128 B (-24 B) 128 B (-24 B)
host/assets/rpc-resolve.js 2.4 KB (-71 B) 1.0 KB (-38 B) 1.1 KB (-33 B)
host/assets/shared-mode.js 1.8 KB (-82 B) 752 B (-41 B) 852 B (-43 B)
host/assets/smoldot.js 2.98 MB 2.21 MB 2.22 MB
host/assets/smoldot_worker.js 2.95 MB 2.21 MB 2.21 MB
host/assets/src.js 1.8 KB (-119 B) 846 B (-44 B) 946 B (-55 B)
host/assets/styles.css 15.1 KB 3.2 KB 3.8 KB
host/assets/wasm/web/README.md 10.9 KB 10.9 KB 10.9 KB
host/assets/wasm/web/package.json 371 B 371 B 371 B
host/assets/wasm/web/truapi_server.d.ts 6.9 KB 6.9 KB 6.9 KB
host/assets/wasm/web/truapi_server.js 35.6 KB 6.2 KB 7.2 KB
host/assets/wasm/web/truapi_server_bg.wasm 1.96 MB 617.3 KB 812.7 KB
host/assets/wasm/web/truapi_server_bg.wasm.d.ts 2.5 KB 2.5 KB 2.5 KB
host/assets/web.js 13.2 KB 3.6 KB 4.0 KB
host/assets/worker-runtime.js 6.3 KB 1.6 KB 1.8 KB
host/assets/worker-runtime.js 106 B 106 B 106 B
host/assets/ws.js 23.1 KB (-2.8 KB) 7.5 KB (-873 B) 8.2 KB (-943 B)
host/dotli.png 11.5 KB 11.5 KB 11.5 KB
host/favicon.svg 1.8 KB 1.8 KB 1.8 KB
host/host-sw.js 2.7 KB (-110 B) 1.1 KB (+10 B) 1.2 KB (-9 B)
host/icon-192.png 12.5 KB 12.5 KB 12.5 KB
host/icon-512.png 42.8 KB 42.8 KB 42.8 KB
host/index.html 19.9 KB (-320 B) 4.4 KB (-65 B) 5.4 KB (-70 B)
host/manifest.webmanifest 441 B 441 B 441 B
host/workbox.js 14.8 KB 4.6 KB 5.1 KB
sandbox/app-sw.js 9.6 KB 3.1 KB (+9 B) 3.5 KB (-1 B)
sandbox/assets/bitswap-bridge.js 840 B 840 B 840 B
sandbox/assets/fetch.js 3.4 KB 1.2 KB (-1 B) 1.4 KB (-1 B)
sandbox/assets/index.css 44.8 KB (-3.1 KB) 7.1 KB (-382 B) 7.9 KB (-423 B)
sandbox/assets/index.js 118.0 KB 33.7 KB (-5 B) 39.6 KB (+1 B)
sandbox/favicon.svg 1.8 KB 1.8 KB 1.8 KB
sandbox/index.html 1.7 KB 581 B (-3 B) 785 B (-2 B)
Total 15.90 MB (+5.11 MB) 8.11 MB (+2.88 MB) (-49%) 8.74 MB (+3.08 MB)

Commit: 2a478cf

@socket-security

socket-security Bot commented Jul 13, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​parity/​truapi-host@​0.2.0781009593100
Added@​parity/​truapi@​0.5.18110010097100

View full report

@github-actions

Copy link
Copy Markdown
Contributor

⚡ Performance Report

⚠️ No baseline found on main. This PR's results are recorded but cannot be compared.
Merge to main to establish a baseline.

Replace the Nova host-container, auth, and signing path with the worker-backed Rust core. Keep dotli responsible only for browser policy, persistence, UI, and physical chain transports.
Notification grants do not alter iframe Permissions Policy. Reloading disposed the in-flight Rust request before its response reached the product.
@pgherveou
pgherveou force-pushed the codex/bulletin-preimage-in-core-dotli branch from ff8115e to 95b810c Compare July 15, 2026 09:03
The pending-message flush drops send failures silently, matching the
pre-port behavior. Surfacing them as JSON-RPC errors moves to a
follow-up PR with test coverage.
Independent providers drained the same smoldot response queue, allowing
concurrent Statement Store calls to steal replies.
Override AJV's transitive fast-uri to 3.1.4 to address
GHSA-v2hh-gcrm-f6hx.
@TorstenStueber TorstenStueber changed the title Swith to Rust core Switch to Rust core Jul 22, 2026
@valentinfernandez1

Copy link
Copy Markdown
Collaborator

Hi, I have created this tracking issue. If you happen to find any bugs while testing this branch please add it on the issue so we can investigate.

Please add details, steps to reproduce, etc.

@leonardocustodio leonardocustodio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. I still have some issues when using smoldot as described here, but I think we should merge it. It seems the main branch is no longer compatible with a few methods, as their enum values have changed.

@pgherveou

Copy link
Copy Markdown
Collaborator Author

Approved. I still have some issues when using smoldot as described here, but I think we should merge it. It seems the main branch is no longer compatible with a few methods, as their enum values have changed.

Nice thanks we will look into these extra smoldot issues.
I have a wip branch where I refactored a few things that might address this already #97

…ge-in-core-dotli

# Conflicts:
#	packages/metrics/src/sentry.ts
TarikGul
TarikGul previously approved these changes Jul 22, 2026

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did one more code review - looks solid from that perspective.

@valentinfernandez1

Copy link
Copy Markdown
Collaborator

I'm fine if you want to merge I can keep investigating the smoldot issues on the side

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay so worked back and forth with claude for a while investigating 2 bugs, and instead of just writing per line I had the reproducible and fixes documented, they are shared here with some other nits: https://gist.github.com/TarikGul/df0ec350fc436449846bf779ec9ee7d3

@TarikGul
TarikGul self-requested a review July 23, 2026 16:24

@TarikGul TarikGul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Chatted with @valentinfernandez1 about the gist above. BR-1 good for a follow up - LGTM lets get it merged :)

@valentinfernandez1
valentinfernandez1 merged commit 59ccf02 into main Jul 24, 2026
14 of 15 checks passed
@valentinfernandez1
valentinfernandez1 deleted the codex/bulletin-preimage-in-core-dotli branch July 24, 2026 01:56
valentinfernandez1 added a commit that referenced this pull request Jul 24, 2026
origin/main gained the squash-merge of #70, whose tree is identical to
commit 1edd74e already in this branch's history; -s ours records the
merge without changing the tree.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants