Support chunked libcudf extraction for native UDF builds

[nvliyuan]

Summary

• support both conventional libcudf.so jar entries and the chunk-manifest representation introduced for large native libraries
• reconstruct numbered chunks atomically while validating manifest structure, library size, chunk sizes, and CRC32 values
• use the same extraction implementation from Maven and the standalone helper, preserving the existing jar-matched dependency pin flow

Fixes #643

Test plan

• On deck, use rapids-4-spark_2.12-26.10.0-20260823.065604-20.jar: reconstruct its 43 chunks into a 1.4 GB libcudf.so
• Resolve the jar-recorded native pins and check out cuDF 1494926abf3088da957ef5178a0b2271f381c976
• Run mvn clean package -Pudf-native-examples -Drapids4spark.version=26.10.0-SNAPSHOT with CUDA 12.9 and GCC 13: BUILD SUCCESS
• Compare native UDF cuDF references with reconstructed libcudf.so: 10 required symbols, 0 missing
• Run a JVM native StringWordCount smoke test against the packaged UDF and the same plugin jar: native udf completed rows=3
• Extract a synthetic conventional jar containing libcudf.so and libnvcomp.so.1 and verify exact output bytes
• Verify a corrupted synthetic chunk fails CRC validation without leaving a partial libcudf.so
• Run Python syntax validation, bash -n, xmllint --noout pom.xml, and git diff --check

[greptile-apps]

Greptile Summary

The PR adds a shared extractor that supports conventional and chunk-manifest libcudf.so representations and integrates it into both native build paths.

• Validates chunk metadata, sizes, and CRC32 values before atomically installing the reconstructed library.
• Adds focused unit tests and a path-filtered GitHub Actions workflow for extraction behavior.
• Updates Maven, the standalone helper, and documentation to use the shared implementation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-native-libs.py	Adds conventional extraction and validated, atomic reconstruction of chunked libcudf archives.
examples/UDF-Examples/RAPIDS-accelerated-UDFs/test_extract_native_libs.py	Adds focused regression coverage for successful extraction, malformed metadata, corruption cleanup, and missing entries.
.github/workflows/native-lib-extraction-test.yml	Runs the focused extraction unit suite when the helper, tests, or workflow change.
examples/UDF-Examples/RAPIDS-accelerated-UDFs/extract-cudf-libs.sh	Replaces duplicated unzip and flattening logic with the shared Python extractor.
examples/UDF-Examples/RAPIDS-accelerated-UDFs/pom.xml	Routes Maven native dependency extraction through the shared Python implementation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  J[RAPIDS Accelerator JAR] --> D{libcudf representation}
  D -->|Conventional entry| E[Copy to temporary file]
  D -->|Chunk manifest| V[Validate manifest and chunk metadata]
  V --> R[Reconstruct chunks and verify CRC32]
  E --> A[Atomic install into native-deps]
  R --> A
  A --> B[Native Maven or standalone build]

Loading

_{Reviews (3): Last reviewed commit: "Limit native extraction to libcudf" | Re-trigger Greptile}