[codex] Persist GraphImpl4 reload metadata#88
Draft
tpn wants to merge 9 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
GraphImpl4 was already create-capable and had JIT/codegen smoke coverage, but file-backed reload was still incomplete for tables whose runtime lookup behavior depends on compacted key metadata. The risky gap was downsized 64-bit GraphImpl4 tables: without persisting and reconstructing the composed outer-downsize plus inner GraphImpl4 compact-key state, a reloaded table could not reliably reproduce the create-time lookup domain, especially for loaded-table JIT/codegen scenarios.
This PR makes GraphImpl4 reload semantics real for the covered CPU/LLVM paths while keeping unsupported paths explicit. Assigned8 JIT remains intentionally rejected, and loaded RawDog JIT remains not implemented.
What Changed
GRAPH_INFO_ON_DISKruntime metadata with persisted graph implementation, downsize metadata, and GraphImpl4 compact-key fields.Developer Impact
After this change, GraphImpl4 is no longer create-only for the covered persisted-table scenarios. File-backed GraphImpl4 sparse32 and downsized64 tables can be reloaded and exercised through the existing table interface, and loaded GraphImpl4 LLVM JIT/codegen smoke coverage now protects the compact-key metadata path.
The reload path still deliberately fails closed for unsupported or ambiguous metadata rather than guessing. RawDog loaded JIT is still out of scope for this PR.
Validation
Ran focused local validation:
git diff --checkcmake --build build-graphimpl4 --target perfecthash_unit_tests PerfectHashCreateExe./build-graphimpl4/bin/perfecthash_unit_tests --gtest_filter='PerfectHashOnlineJitTests.CreateTable64AndIndex64:PerfectHashOnlineJitTests.Index64OnNonGraphImpl64BitTableUsesMetadata:GraphImpl4BitUtils.ContiguousBitmapDetection:GraphImpl4BitUtils.ComposedDownsizeMetadataUsesComposedBitmap:GraphImpl4BitUtils.ComposedDownsizeMetadataRejectsInnerContiguityMismatch:GraphImpl4BitUtils.ComposedExtractionMatchesTwoStepExtraction:GraphImpl4BitUtils.Downsized64OuterBitmapProducesIdentityInnerBitmap:PerfectHashOnlineTests.GraphImpl4Assigned8RequiresOptIn:PerfectHashOnlineTests.GraphImpl4SupportsDownsized64BitInputs:PerfectHashOnlineTests.GraphImpl4RejectsNonGoodHashes:PerfectHashOnlineTests.GraphImpl4Assigned8JitRejected:PerfectHashOnlineTests.GraphImpl4RawDogJitMatchesIndexAssigned32:PerfectHashOnlineTests.GraphImpl4RawDogJitMatchesIndexSparse32:PerfectHashOnlineTests.GraphImpl4RawDogIndex32x4MatchesIndexAssigned16:PerfectHashOnlineTests.GraphImpl4LlvmJitMatchesIndexAssigned32:PerfectHashOnlineTests.GraphImpl4LlvmJitMatchesIndexSparse32:PerfectHashOnlineTests.GraphImpl4LlvmJitMulshrolate3RXMatchesIndexAssigned32:PerfectHashOnlineTests.GraphImpl4LlvmIndex32x4MatchesIndexAssigned16:PerfectHashOnlineTests.GraphImpl4LlvmIndex64x4MatchesDownsizedIndex:PerfectHashOnlineTests.GraphImpl4FileBackedReloadPreservesSparse32Compaction:PerfectHashOnlineTests.GraphImpl4FileBackedReloadPreservesDownsized64Compaction:PerfectHashOnlineTests.NonGraphImplFileBackedReloadPreservesDownsized64Metadata:PerfectHashOnlineTests.GraphImpl4FileBackedReloadSparse32JitIndex32:PerfectHashOnlineTests.GraphImpl4FileBackedReloadDownsized64JitIndex64'ctest --test-dir build-graphimpl4 --output-on-failure -R 'graphimpl4'roborev waiton the final commit: no issues foundscripts/roborev-matrix-review.sh HEAD^..HEAD: all agents agreed no Medium/High/Critical issues; PR is clean