feat(ice): add validated Planar BC tooling - #4
Closed
ElmoPA wants to merge 10 commits into
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the reusable ICE/PACE foundation onto the clean EgoVerse stack: externally owned Slurm requeue support, immutable runtime capture, GPU probing, strict checkpoint validation, the generic requeue runner, one-GPU Planar BC launcher, and independent checkpoint mirroring.
The latest commit adds a bounded multi-node
ice-cpumirror pool. Array workers use shared short state locks and nonblocking per-checkpoint locks so validation, hashing, transfer, and remote SHA verification run concurrently without duplicate uploads. Array element zero alone owns whole-scratch pressure reporting, opt-in guarded pruning, and terminal completion. Pruning still retains at least two recent local checkpoints and requires a fresh local validation/hash, exact remote SHA-256, and an unchanged final local stat. GPU jobs remain independent.Verification: Ruff and shell syntax passed; 43 focused mirror/runner tests passed; the full branch suite passed 288 tests. No ICE allocation, transfer, pruning, or training job was submitted during this audit.
This PR is portable tooling, not live launch authority. Site identities and exact source/data/runtime/run identities remain explicit inputs, and the live ICE/Skynet gates still apply.