Snapshots feature refactor#292
Conversation
License Check Results🚀 The license check job ran with the Bazel command: bazel run --lockfile_mode=error //:license-checkStatus: Click to expand output |
|
The created documentation from the pull request is available at: docu-html |
| @@ -593,119 +593,141 @@ score::ResultBlank Kvs::flush() | |||
| result = score::MakeUnexpected(ErrorCode::JsonGeneratorError); | |||
| } | |||
| else | |||
There was a problem hiding this comment.
Please use clang-format with config available in the repo for C++ files.
| result = write_json_data(buf); | ||
| } | ||
| { | ||
| /* Write JSON Data */ |
There was a problem hiding this comment.
Please add a comment here that it writes to a file with snapshot ID 0.
Add this information to flush method docs.
| size_t count = 0; | ||
| bool error = false; | ||
| for (size_t idx = 0; idx < KVS_MAX_SNAPSHOTS; ++idx) | ||
| for (size_t idx = 1; idx <= KVS_MAX_SNAPSHOTS; ++idx) |
There was a problem hiding this comment.
Okay, so snapshot ID==0 is not considered snapshot? This might be fine, but consider documenting the approach to snapshot ID in score/kvs/docs/requirements/index.rst. It should be written somewhere that 0 is reserved for current storage, and snapshots are in <1; max snapshot> range.
| const score::filesystem::Path dst_json{filename_prefix.Native() + "_" + to_string(new_snapshot_id) + ".json"}; | ||
| const score::filesystem::Path dst_hash{filename_prefix.Native() + "_" + to_string(new_snapshot_id) + ".hash"}; | ||
|
|
||
| const auto copy_json_res = filesystem->standard->CopyFile(src_json, dst_json); |
There was a problem hiding this comment.
This approach stores new snapshot based on snapshot 0 file. This can lead to surprising behavior if flush was not done prior to snapshot creation.
I'd consider storing current memory state as less surprising variant, but nonetheless this should be clearly stated.
| return count; | ||
| } | ||
|
|
||
| score::Result<std::size_t> Kvs::snapshot_create() |
There was a problem hiding this comment.
Please rework the function to return quickly on error. This will reduce if-else pyramids occurring in the code, with easier tracking on what's the method result.
| } | ||
| else | ||
| { | ||
| if (snapshot_count_res.value() == 0 || snapshot_id.id >= snapshot_max_count()) |
There was a problem hiding this comment.
From previous methods I inferred that range is <1; max snapshot>, but this condition indicates that restoring for max snapshot is disallowed. Similar issue is observed in other methods.
| else if (snapshot_count_res.value() < snapshot_id.id) | ||
| else if (!json_exists_res.value()) | ||
| { | ||
| logger->LogError() << "Snapshot JSON file does not exist: '" << json_path << "'"; |
There was a problem hiding this comment.
Please check if it prints correctly, without added whitespaces surrouding path. If so, consider reworking those logs to logger->LogError() << "Snapshot JSON file does not exist:" << json_path;
| /* Check if files were created correctly */ | ||
| EXPECT_TRUE(std::filesystem::exists(kvs_prefix + ".json")); | ||
| EXPECT_TRUE(std::filesystem::exists(kvs_prefix + ".hash")); | ||
| EXPECT_FALSE(std::filesystem::exists(filename_prefix + "_1.json")); |
There was a problem hiding this comment.
Add a comment that having a rotated snapshot was previous expected behavior, or remove check for snapshot ID 1.
| } | ||
|
|
||
|
|
||
| /* Crea snapshot non contigui e verifica che il conteggio sia corretto */ |
There was a problem hiding this comment.
Please replace non-english comments.
The C++ implementation of the KVS module has been modified in terms of snapshot behavior.
Removal of
snapshot_rotateThe
snapshot_rotatefunction has been completely removed from the C++ implementation. Snapshot management is no longer automatic and requires an explicit call to the snapshot_create() function.The
flush()function no longer automatically creates snapshots.In the previous behavior (still valid for RUST)
flush()persisted the data and calledsnapshot_rotate()to automatically create a new snapshot. Nowflush()now only persists the current state of the KVS to disk.Implementation of snapshot_create API
The new snapshot_create() function has been implemented, allowing for the explicit creation of snapshots in the first available slot.
Implementation of snapshot_delete API
The snapshot_delete() function has been implemented to explicitly remove a specific snapshots by ID.
Update to snapshot_count and snapshot_restore.
A new component requirement has been introduced, according to the new behaviour of the snapshot_create.