cardano-rpc: Implement FollowTip SyncService method - #1268
Conversation
7e9bf4c to
0e5a9c1
Compare
There was a problem hiding this comment.
Pull request overview
This PR implements the SyncService.FollowTip server-streaming RPC in cardano-rpc, providing a ChainSync-like block stream backed by in-process ChainDB followers (ADR-019), and factors shared block/point conversion logic so FollowTip and FetchBlock reuse the same proto assembly.
Changes:
- Implement
followTipMethod/followTipStreamwith ChainDB follower intersection, streaming apply/undo/reset actions, and per-message tip reporting. - Add
NodeKernelAccess.withFollowerplusChainFollower/ChainChangewrappers; plumb consensussecurityParamthroughNodeKernelAccessto size the undo window. - Add conversion helpers (
mkAnyChainBlock,mkTipBlockRef,chainPointToBlockRef) and unit tests for the streaming loop; wire tracing + README coverage update.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FollowTipStream.hs | New unit tests covering followTipStream sequencing, intersection failure, and rollback/undo vs reset fallback. |
| cardano-rpc/src/Cardano/Rpc/Server/NodeKernelAccess/Type.hs | Add securityParam to NodeKernelAccess to size rollback tracking. |
| cardano-rpc/src/Cardano/Rpc/Server/NodeKernelAccess.hs | Add follower lifecycle bracket (withFollower) and follower wrappers (ChainFollower, ChainChange). |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/ChainPoint.hs | Add shared helpers for tip/blockref projection and timestamp encoding. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Block.hs | New shared block-to-proto conversion (mkAnyChainBlock) used by both fetch and follow. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs | Implement FollowTip; refactor FetchBlock/ReadTip to reuse shared conversion helpers; add followTipStream. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/Tracing.hs | Add TraceRpcFollowTipSpan; remove dead TraceRpcForkerError. |
| cardano-rpc/src/Cardano/Rpc/Server.hs | Wire FollowTip into the gRPC method table as a server-streaming handler. |
| cardano-rpc/README.md | Mark FollowTip as supported in the SyncService coverage table. |
| cardano-rpc/cardano-rpc.cabal | Register new internal module and test; add test dependency needed for fixture decoding. |
| cardano-api/src/Cardano/Api/Consensus/Internal/Reexport.hs | Re-export follower/security-param/registry types and functions used by cardano-rpc. |
| cardano-api/src/Cardano/Api/Consensus.hs | Expose follower API and security-param helpers via the public consensus facade. |
| cardano-api/cardano-api.cabal | Add resource-registry dependency needed for withRegistry / ResourceRegistry re-exports. |
| .changes/20260722_cardano_rpc_follow_tip.yml | Changelog fragment for the FollowTip feature (needs kind update per review comment). |
| .changes/20260722_cardano_api_chaindb_follower_reexports.yml | Changelog fragment for new cardano-api re-exports. |
| (Consensus.CardanoBlock Consensus.StandardCrypto) | ||
| (ByteString, BlockInMode) | ||
| component = | ||
| (,) <$> fmap BSL.toStrict Consensus.GetRawBlock <*> fmap fromConsensusBlock Consensus.GetBlock |
There was a problem hiding this comment.
What about clients who just want the raw bytes and don't want to pay the decode cost at this point?
There was a problem hiding this comment.
Great question - and I'm glad you asked, because this is exactly where I want to take this next!
Right now they can't avoid it: the AnyChainBlock response message carries both native_bytes and the parsed cardano block, so the handler has to decode every block just to populate the response, regardless of what the client wants.
The exciting part: the spec already has the answer built in - the field_mask in FollowTipRequest/FetchBlockRequest, which cardano-rpc currently ignores (like every other UTxO RPC method here).
I've just opened #1273 to fix that properly: the mask will drive response construction itself, so a mask excluding the parsed cardano.* fields selects a raw-bytes-only BlockComponent and the decode never happens at all.
Raw-bytes-only followers get dramatically cheaper streaming, practically for free.
Out of scope for this PR, but very much on the roadmap!
0e5a9c1 to
35f8d03
Compare
Context
Implements the
SyncService.FollowTipmethod (#1219), the primary blocker of the Kupo/Ogmios replacement effort (#1229). It is the gRPC analogue of the Ouroboros ChainSync mini-protocol, built on in-process ChainDB followers (ADR-019).Stacked on #1259; review the commits after #1259's.
FollowTipstreams fully parsed blocks as the chain advances, reusing the same block-to-proto conversionFetchBlockuses. The client's intersect list is honoured in preference order, and the first streamed message is always a reset announcing the resolved start point. An empty-hash block ref denotes origin, handled as an infallible catch-all rather than a special case. An empty intersect list follows from the current tip, matching Dolos's behaviour; an intersect list that matches nothing on the chain fails withNOT_FOUNDbefore any message is sent. Every response carries the current tip alongside the action, for lag measurement.Rollbacks are delivered as
undomessages carrying the rolled-back blocks, newest first. Consensus'sRollBackis point-only, it never carries block data, so each stream tracks the points of its last k applied blocks (the security parameter, about 86 KB per stream on mainnet) and re-fetches the undone blocks from the ChainDB on rollback. The stream falls back to aresetat the rollback point in the two cases whereundois impossible: the rollback target is outside the tracked window, or a re-fetch misses because the block was garbage-collected first.Implementation notes
NodeKernelAccessgainswithFollower, a bracket overChainDB.newFollowerthat closes the follower and its resource registry on every exit path, including client disconnect, plus theChainFollower/ChainChangetypes wrapping the raw follower API.Cardano.Api.Consensusgains the follower re-exports (newFollower,Follower,ChainType,ChainUpdate,withRegistry,ResourceRegistry).FollowTipis the first server-streaming handler in cardano-rpc, wired via grapesy'smkServerStreaming.TraceRpcFollowTipSpantrace wraps the handler; the node-side rendering and therpc.request.SyncService.FollowTipprometheus counter land in gRPC: fetchBlock method cardano-node#6579, the fetchblock-series node PR that already carries the earlier tracing and E2E work (there is no dedicated FollowTip node PR).TraceRpcForkerErrorconstructor is removed.How to trust this PR
The unit suite (83 tests) covers the conversion layer.
The E2E test in IntersectMBO/cardano-node#6579 exercises the streaming semantics live: full sync from origin, resuming from a point, an empty intersect list resolving from the current tip, and the
NOT_FOUNDpath for an unmatched intersect list. It also adds a live-tail scenario, being added right now in the node working tree, that submits a transaction over gRPC and asserts it arrives viaFollowTipas a parsed apply action.Run it with:
Checklist
.changes/