fix(worker): use stored block length for replication reads (#1634) - #1637
Open
gangump82 wants to merge 1 commit into
Open
fix(worker): use stored block length for replication reads (#1634)#1637gangump82 wants to merge 1 commit into
gangump82 wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes Curvine worker block replication failures by ensuring the source reader uses the finalized block’s stored logical length (instead of an explicit zero), aligning replication reads with the strict logical-boundary semantics used elsewhere.
Changes:
- Add
BlockStore::open_reader_by_id_at_stored_lenfor internal read paths that should derive the logical boundary from storedBlockMeta::len(). - Refactor
open_reader_by_idto share the dataset-lock-held reader-open logic via an internal helper. - Update
WorkerReplicationManager::replicate_blockto open replication source readers using the stored metadata length, and add/adjust unit tests to cover the new API.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| curvine-worker/src/worker/replication/worker_replication_manager.rs | Use a stored-metadata-length reader for replication source reads to avoid zero-length logical boundaries. |
| curvine-worker/src/worker/block/block_store.rs | Introduce a stored-length reader API, factor common reader-open logic, and add regression coverage validating finalized reads via stored length. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix block replication failures caused by opening the source block reader with a
zero logical length.
Replication now derives its read boundary from the finalized block metadata
while preserving the explicit logical-length behavior used by normal client
reads and sparse/truncated blocks.
Issue Describe / Design
Closes #1634
Root cause
WorkerReplicationManager::replicate_blockopened the source block with:The third argument is the reader's logical length. After the strict logical
boundary behavior introduced for sparse and truncated block reads, zero is a
valid zero-length boundary rather than a request to infer the physical block
length.
As a result, the first replication
read_regioncall failed with:The error happened while reading the source replica, before data could be
written to the target Worker.
Reproduction
Fix approach
Add a dedicated
BlockStore::open_reader_by_id_at_stored_lenAPI for internalblock-copy operations that do not have a client-provided logical length.
The API obtains the current readable
BlockMeta, derives the logical boundaryfrom
BlockMeta::len(), resolves the generation and layout, and opens the filedescriptor while holding the same dataset read lock.
The existing explicit-length API remains unchanged. In particular, this change
does not restore a
max(logical_len, physical_len)fallback, so sparse andtruncated reads retain their exact logical boundaries.
Changes
curvine-worker/src/worker/block/block_store.rscurvine-worker/src/worker/block/block_store.rscurvine-worker/src/worker/block/block_store.rscurvine-worker/src/worker/replication/worker_replication_manager.rsTest verified
cargo test -p curvine-workercargo test -p curvine-tests --test replication_test test_replication_honors_source_block_capacity -- --nocapturelength=0; source capacity was preservedcargo test -p curvine-tests --test replication_test test_block_replication_e2e -- --nocapturecargo fmt --all -- --checkgit diff --checkDependencies