Skip to content

cardano-rpc: Implement FollowTip SyncService method - #1268

Merged
carbolymer merged 6 commits into
masterfrom
mgalazyn/feature/grpc-follow-tip
Jul 30, 2026
Merged

cardano-rpc: Implement FollowTip SyncService method#1268
carbolymer merged 6 commits into
masterfrom
mgalazyn/feature/grpc-follow-tip

Conversation

@carbolymer

@carbolymer carbolymer commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Context

Implements the SyncService.FollowTip method (#1219), the primary blocker of the Kupo/Ogmios replacement effort (#1229). It is the gRPC analogue of the Ouroboros ChainSync mini-protocol, built on in-process ChainDB followers (ADR-019).

Stacked on #1259; review the commits after #1259's.

FollowTip streams fully parsed blocks as the chain advances, reusing the same block-to-proto conversion FetchBlock uses. The client's intersect list is honoured in preference order, and the first streamed message is always a reset announcing the resolved start point. An empty-hash block ref denotes origin, handled as an infallible catch-all rather than a special case. An empty intersect list follows from the current tip, matching Dolos's behaviour; an intersect list that matches nothing on the chain fails with NOT_FOUND before any message is sent. Every response carries the current tip alongside the action, for lag measurement.

Rollbacks are delivered as undo messages carrying the rolled-back blocks, newest first. Consensus's RollBack is point-only, it never carries block data, so each stream tracks the points of its last k applied blocks (the security parameter, about 86 KB per stream on mainnet) and re-fetches the undone blocks from the ChainDB on rollback. The stream falls back to a reset at the rollback point in the two cases where undo is impossible: the rollback target is outside the tracked window, or a re-fetch misses because the block was garbage-collected first.

Implementation notes

  • NodeKernelAccess gains withFollower, a bracket over ChainDB.newFollower that closes the follower and its resource registry on every exit path, including client disconnect, plus the ChainFollower/ChainChange types wrapping the raw follower API.
  • Cardano.Api.Consensus gains the follower re-exports (newFollower, Follower, ChainType, ChainUpdate, withRegistry, ResourceRegistry).
  • FollowTip is the first server-streaming handler in cardano-rpc, wired via grapesy's mkServerStreaming.
  • A TraceRpcFollowTipSpan trace wraps the handler; the node-side rendering and the rpc.request.SyncService.FollowTip prometheus counter land in gRPC: fetchBlock method cardano-node#6579, the fetchblock-series node PR that already carries the earlier tracing and E2E work (there is no dedicated FollowTip node PR).
  • The dead TraceRpcForkerError constructor is removed.

How to trust this PR

The unit suite (83 tests) covers the conversion layer.

The E2E test in IntersectMBO/cardano-node#6579 exercises the streaming semantics live: full sync from origin, resuming from a point, an empty intersect list resolving from the current tip, and the NOT_FOUND path for an unmatched intersect list. It also adds a live-tail scenario, being added right now in the node working tree, that submits a transaction over gRPC and asserts it arrives via FollowTip as a parsed apply action.

Run it with:

cabal test cardano-rpc-test
TASTY_PATTERN='/RPC FollowTip/' cabal test cardano-testnet-test

Checklist

  • Commit sequence broadly makes sense and commits have useful messages
  • New tests are added if needed and existing tests are updated. See Running tests for more details
  • Self-reviewed the diff
  • Changelog fragment added in .changes/

@carbolymer carbolymer linked an issue Jul 27, 2026 that may be closed by this pull request
8 tasks
@carbolymer carbolymer self-assigned this Jul 27, 2026
@carbolymer carbolymer moved this to In Progress in DevTools roadmap Jul 27, 2026
@carbolymer carbolymer changed the title cardano-rpc: Add FollowTip method cardano-rpc: Implement FollowTip SyncService method Jul 27, 2026
@carbolymer
carbolymer force-pushed the mgalazyn/feature/grpc-follow-tip branch 6 times, most recently from 7e9bf4c to 0e5a9c1 Compare July 28, 2026 14:48
@carbolymer
carbolymer marked this pull request as ready for review July 28, 2026 14:54
Copilot AI review requested due to automatic review settings July 28, 2026 14:54
@carbolymer
carbolymer requested a review from palas as a code owner July 28, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements the SyncService.FollowTip server-streaming RPC in cardano-rpc, providing a ChainSync-like block stream backed by in-process ChainDB followers (ADR-019), and factors shared block/point conversion logic so FollowTip and FetchBlock reuse the same proto assembly.

Changes:

  • Implement followTipMethod / followTipStream with ChainDB follower intersection, streaming apply/undo/reset actions, and per-message tip reporting.
  • Add NodeKernelAccess.withFollower plus ChainFollower/ChainChange wrappers; plumb consensus securityParam through NodeKernelAccess to size the undo window.
  • Add conversion helpers (mkAnyChainBlock, mkTipBlockRef, chainPointToBlockRef) and unit tests for the streaming loop; wire tracing + README coverage update.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FollowTipStream.hs New unit tests covering followTipStream sequencing, intersection failure, and rollback/undo vs reset fallback.
cardano-rpc/src/Cardano/Rpc/Server/NodeKernelAccess/Type.hs Add securityParam to NodeKernelAccess to size rollback tracking.
cardano-rpc/src/Cardano/Rpc/Server/NodeKernelAccess.hs Add follower lifecycle bracket (withFollower) and follower wrappers (ChainFollower, ChainChange).
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/ChainPoint.hs Add shared helpers for tip/blockref projection and timestamp encoding.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Block.hs New shared block-to-proto conversion (mkAnyChainBlock) used by both fetch and follow.
cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs Implement FollowTip; refactor FetchBlock/ReadTip to reuse shared conversion helpers; add followTipStream.
cardano-rpc/src/Cardano/Rpc/Server/Internal/Tracing.hs Add TraceRpcFollowTipSpan; remove dead TraceRpcForkerError.
cardano-rpc/src/Cardano/Rpc/Server.hs Wire FollowTip into the gRPC method table as a server-streaming handler.
cardano-rpc/README.md Mark FollowTip as supported in the SyncService coverage table.
cardano-rpc/cardano-rpc.cabal Register new internal module and test; add test dependency needed for fixture decoding.
cardano-api/src/Cardano/Api/Consensus/Internal/Reexport.hs Re-export follower/security-param/registry types and functions used by cardano-rpc.
cardano-api/src/Cardano/Api/Consensus.hs Expose follower API and security-param helpers via the public consensus facade.
cardano-api/cardano-api.cabal Add resource-registry dependency needed for withRegistry / ResourceRegistry re-exports.
.changes/20260722_cardano_rpc_follow_tip.yml Changelog fragment for the FollowTip feature (needs kind update per review comment).
.changes/20260722_cardano_api_chaindb_follower_reexports.yml Changelog fragment for new cardano-api re-exports.

Comment thread .changes/20260722_cardano_rpc_follow_tip.yml
Comment thread cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs Outdated
Comment thread cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs Outdated

@Jimbo4350 Jimbo4350 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Some of the comments on your functions are difficult to decipher however I wouldn't block the PR on that. One comment for you to consider.

(Consensus.CardanoBlock Consensus.StandardCrypto)
(ByteString, BlockInMode)
component =
(,) <$> fmap BSL.toStrict Consensus.GetRawBlock <*> fmap fromConsensusBlock Consensus.GetBlock

@Jimbo4350 Jimbo4350 Jul 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about clients who just want the raw bytes and don't want to pay the decode cost at this point?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great question - and I'm glad you asked, because this is exactly where I want to take this next!

Right now they can't avoid it: the AnyChainBlock response message carries both native_bytes and the parsed cardano block, so the handler has to decode every block just to populate the response, regardless of what the client wants.

The exciting part: the spec already has the answer built in - the field_mask in FollowTipRequest/FetchBlockRequest, which cardano-rpc currently ignores (like every other UTxO RPC method here).
I've just opened #1273 to fix that properly: the mask will drive response construction itself, so a mask excluding the parsed cardano.* fields selects a raw-bytes-only BlockComponent and the decode never happens at all.
Raw-bytes-only followers get dramatically cheaper streaming, practically for free.
Out of scope for this PR, but very much on the roadmap!

@carbolymer
carbolymer force-pushed the mgalazyn/feature/grpc-follow-tip branch from 0e5a9c1 to 35f8d03 Compare July 30, 2026 13:50
@carbolymer
carbolymer enabled auto-merge July 30, 2026 13:50
@carbolymer
carbolymer added this pull request to the merge queue Jul 30, 2026
Merged via the queue into master with commit 5de6ff6 Jul 30, 2026
31 checks passed
@carbolymer
carbolymer deleted the mgalazyn/feature/grpc-follow-tip branch July 30, 2026 15:14
@github-project-automation github-project-automation Bot moved this from In Progress to Done in DevTools roadmap Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

gRPC: Implement FollowTip (UTxO RPC SyncService) in cardano-rpc

3 participants