feat(storage): add persistent volume snapshot/resize/persistence suite (DATASVC-XX-02/03/04)#439
Draft
abegnoche wants to merge 8 commits into
Draft
feat(storage): add persistent volume snapshot/resize/persistence suite (DATASVC-XX-02/03/04)#439abegnoche wants to merge 8 commits into
abegnoche wants to merge 8 commits into
Conversation
Adds a provider-agnostic block-storage validation suite plus the AWS reference implementation covering the block-storage trio that share one fixture (launch instance + create/attach EBS volume): - DATASVC-XX-02 (#321) snapshot_lifecycle: CreateSnapshot -> restore to a new volume -> attach/mount -> byte-compare the sentinel data. - DATASVC-XX-03 (#322) volume_resize: ModifyVolume + in-guest growpart/resize2fs, verifying the guest sees the larger filesystem. - DATASVC-XX-04 (#323) volume_persistence: stop/start the instance and verify the volume stays attached with its data intact. Reuses existing StepSuccessCheck + FieldExistsCheck + CrudOperationsCheck (no new validation classes, no released_tests.json change). common/ebs.py centralizes the boto3 volume/snapshot lifecycle and the Nitro NVMe by-id device mapping; create_volume/teardown_volume own the shared fixture. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
Adds the my-isv living-example scaffold (TODO blocks + ISVCTL_DEMO_MODE dummy success) for the five block-storage scripts and the provider config that imports the suite. Adds block-storage to MY_ISV_DOMAINS so 'make demo-test' exercises the whole pipeline end-to-end with no cloud. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
Adds the block-storage suite to the contracts list and per-step field table in suites/README.md, and a block-storage/ row to the my-isv scaffold domains table. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
Covers the common/ebs Nitro device mapping and modification waiter, plus happy-path and failure cases (guest setup failure, snapshot content mismatch, cleanup leak, growpart failure, volume detached after restart) for the five AWS block-storage scripts using boto3/ssh fakes. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
boto3 waiters raise WaiterError (a BotoCoreError, not a ClientError), so a waiter timeout would have propagated uncaught and skipped the JSON contract the validations depend on - the same failure mode #438 fixed for S3. Broaden the AWS script except clauses to (ClientError, BotoCoreError) and make the ebs cleanup helpers swallow BotoCoreError so best-effort teardown never raises out of a finally block. Also pass the snapshot ID to cleanup when CreateSnapshot succeeded but the completion wait failed. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
…new suite Move the DATASVC-XX-02/03/04 block-storage validations out of the standalone suites/block-storage.yaml and into suites/vm.yaml, so block storage is validated under the existing VM contract rather than adding another suite file. - vm.yaml: add fixture_volume / snapshot_lifecycle / volume_resize / volume_persistence / volume_teardown_checks. These activate only when a provider config supplies the matching volume steps; a standard VM run skips them automatically. Renamed the volume teardown check to volume_teardown_checks to avoid colliding with the VM teardown_checks, and dropped the redundant instance_launched (covered by setup_checks). - providers/aws|my-isv/config/block-storage.yaml: import suites/vm.yaml, rename the command group to vm (must match the inherited platform), and exclude the ssh-labelled VM host checks (cloud-init/GPU/OS/NIM are not block-storage concerns). - delete suites/block-storage.yaml. - docs: drop block-storage from the suites list and point the domain's contract at vm.yaml. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
Replace the earlier approach of folding block-storage validations into suites/vm.yaml. Instead, add a dedicated suites/storage.yaml umbrella contract (platform: storage) that today covers persistent block storage (DATASVC-XX-02/03/04) and is the future home for object/file storage checks - avoiding both suite proliferation and the skip-noise of sharing the VM contract. - suites/storage.yaml: new umbrella contract (was the block-storage suite). - suites/vm.yaml: reverted to its original VM-only validations. - providers/aws|my-isv/config/block-storage.yaml: import suites/storage.yaml and use the 'storage' command group (matches the suite platform). - docs: list the 'storage' suite and point the block-storage domain's contract at storage.yaml. The block-storage provider domain (config/block-storage.yaml + scripts/block-storage/) stays as the concrete block-volume implementation of the storage contract. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
Complete the storage-suite rename by aligning the runnable provider domain with the suite name. - providers/aws|my-isv/config/storage.yaml replace config/block-storage.yaml. - providers/aws|my-isv/scripts/storage/ replace scripts/block-storage/. - Makefile demo domain is now storage, so the focused smoke test is make demo-storage. - Tests and docs now refer to storage paths and the AWS unit test file is renamed to test_aws_storage_scripts.py. Signed-off-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Alexandre Begnoche <abegnoche@users.noreply.github.com>
abegnoche
changed the title
cursor
Bot
changed the title
Member
Author
|
Note: currently in draft because it adds a new storage.yaml file |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes #321
Closes #322
Closes #323
Adds a new
storagevalidation suite covering the persistent volume trio:snapshot_lifecycle: snapshot a volume, restore it, and verify sentinel data.volume_resize: grow the volume and in-guest filesystem, then verify size.volume_persistence: stop/start the instance and verify volume/data persistence.storage.yamlis intentionally broad so future object/file storage checks can land in the same suite.Changes
suites/storage.yamlprovider-agnostic contract.providers/aws/config/storage.yamlandproviders/aws/scripts/storage/.my-isvscaffold underproviders/my-isv/config/storage.yamlandproviders/my-isv/scripts/storage/.storagetoMY_ISV_DOMAINS(make demo-storage).isvctl/tests/test_aws_storage_scripts.py.Validation
ISVTEST_INCLUDE_UNRELEASED=1 AWS_PROFILE=ncp-isv-lab uv run isvctl test run -f isvctl/configs/providers/aws/config/storage.yaml -- -v -s— AWS live run passed setup, all three storage tests, and teardown.make demo-storagemake demo-testcd isvctl && uv run pytest tests/test_aws_storage_scripts.py tests/test_stub_contracts.py -qmake lintuvx pre-commit run