Support verified on-demand Windows CUDA sidecar downloads at runtime

[leehack]

Goal

Allow Windows x64 applications to download a compatible CUDA backend sidecar at runtime instead of embedding every supported CUDA runtime in the application package.

This is a follow-up to the initial build-time packaging design:

• CUDA 13 is the default bundled variant.
• Users can opt into CUDA 12 or both variants.
• Runtime download remains optional and must never happen silently.

Producer-side experiment: leehack/llamadart-native#37
Related backend-selection issue: #7

Motivation

The current experimental compressed sidecars are approximately:

• CUDA 13: 496 MB
• CUDA 12: 605 MB
• Both: 1.1 GB

Bundling both provides broad compatibility but substantially increases application size. Runtime acquisition would let an application ship without CUDA or with one preferred variant, then obtain the compatible sidecar only when the user enables CUDA.

Runtime selection also avoids treating the build machine's GPU and driver as representative of the eventual target machine.

Proposed behavior

Consent and policy

• Runtime downloading is explicitly enabled by the application or confirmed by the user.
• No network request occurs merely because GpuBackend.auto is used.
• Applications can disable runtime acquisition and remain fully offline.
• Provide progress, cancellation, and actionable failure reporting.

Compatibility selection

Probe the installed NVIDIA driver through the Windows CUDA Driver API before loading any CUDA backend.

Initial policy, derived from the verified sidecar contracts:

• Driver family 580+ and compute capability 7.5+ may select CUDA 13.
• Driver family 525+ and compute capability 5.0+ may select CUDA 12.
• Otherwise report CUDA as unavailable and preserve CPU/Vulkan behavior.

Allow an explicit CUDA 12/13 override for troubleshooting and mixed-GPU systems. When both variants are cached, load exactly one CUDA backend and only its matching runtime dependencies per process.

Acquisition and verification

• Resolve assets for the exact pinned llamadart-native tag.
• Verify the release-asset SHA-256 digest before extraction.
• Validate cuda-pack.json, including contract version, llama.cpp tag and commit, platform/architecture, CUDA variant, compatibility bounds, and per-file digests.
• Reject path traversal, unexpected files, wrong architecture, stale/version-skewed packs, and incomplete dependency sets.
• Extract into a temporary directory and atomically promote a fully verified cache entry.
• Use a per-pack lock so concurrent engines/processes do not corrupt the cache.
• Do not log credentials, signed URLs, or secret-bearing cache paths.

Cache lifecycle

• Cache by native tag, llama.cpp commit, platform, architecture, and CUDA variant.
• Reuse only completely verified entries.
• Define retry/resume behavior for interrupted downloads.
• Expose cache inspection and removal without deleting unrelated native assets.
• Document disk requirements and offline behavior.

Acceptance criteria

• No silent runtime download; explicit application/user consent is required.
• CUDA 12 and CUDA 13 compatibility selection is covered by boundary tests.
• Explicit variant override works for mixed-GPU and troubleshooting cases.
• Only one CUDA backend and matching dependency family is loaded per process.
• Corrupt, truncated, wrong-tag, wrong-commit, wrong-architecture, and unsupported-contract packs fail safely.
• Concurrent acquisition is atomic and leaves no accepted partial cache.
• Cancellation and retry do not poison later loads.
• Offline/no-driver/unsupported-GPU paths preserve documented CPU/Vulkan behavior with actionable diagnostics.
• Unit tests use fixture packs and a controlled HTTP source; default CI does not depend on large remote assets.
• A GPU-backed validation matrix covers CUDA 12 and CUDA 13 before the feature is documented as supported.

Out of scope

• Publishing the CUDA sidecars; that remains owned by llamadart-native.
• Loading CUDA 12 and CUDA 13 simultaneously in one process.
• Treating runtime download as a substitute for the initial build-hook options.
• Claiming CUDA kernel/inference support from archive or loader-only validation.

The open MTP acceptance, CUDA lockup, DFlash concurrency, DSpark VRAM-leak, EAGLE, and failed-state-restore reports remain publication gates for affected runtime paths.