Skip to content

Speed up Windows ARM64 Vulkan builds - #38

Draft
leehack wants to merge 1 commit into
mainfrom
optimize/windows-arm64-vulkan-build
Draft

Speed up Windows ARM64 Vulkan builds#38
leehack wants to merge 1 commit into
mainfrom
optimize/windows-arm64-vulkan-build

Conversation

@leehack

@leehack leehack commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • cross-build the Windows ARM64 Vulkan lane on windows-2022 with the x64-hosted Visual Studio Clang toolchain and Ninja
  • compile the large host-side ggml-vulkan target with -Os while retaining Release ThinLTO; embedded SPIR-V is unchanged
  • keep the existing native ARM64 BLAS lane and every non-ARM64 build path unchanged
  • add an explicit CMake opt-out plus focused preset-routing tests and ARM64 package/ABI checks

Why

At the current b10453 baseline, ggml-vulkan.cpp accounted for roughly 25 minutes of a 33m12s Windows ARM64 Vulkan source build under Clang -O3. Disabling IPO globally or only for the backend did not materially fix that bottleneck.

The final -Os + Release IPO variant completed the full source build in 6m12s and the job in 7m34s. The generated ggml-vulkan.dll remained about 37 MiB, matching the original optimized package range.

Validation

  • python3 -m unittest discover -s tools/tests -v — 3/3 pass
  • python3 -m json.tool CMakePresets.json — pass
  • python3 -m py_compile tools/build.py tools/tests/test_windows_build_config.py — pass
  • actionlint .github/workflows/validate_wrapper.yml — pass
  • exact b10453 nonpublishing build: https://github.com/leehack/llamadart-native/actions/runs/32130472582 — pass
    • all 9 DLLs verified as PE AArch64
    • wrapper exports, including llama_dart_speculative_need_embd*, verified
    • Vulkan dependency on ggml-base.dll, KleidiAI, and fast-compile cache contract verified
    • independent local artifact inspection confirmed ARM64 architecture, exports, imports, and ~37 MiB Vulkan DLL

Negative/control experiments:

Publication gate

Before publishing a release built with this path, Windows ARM64 Vulkan hardware must validate model load, generation correctness, and representative throughput. The change lowers optimization only for CPU-side Vulkan host code, so compile/package evidence cannot exclude a runtime performance regression.

Merging this build configuration does not publish artifacts; the native release workflow remains a separate approval boundary.

No llamadart change is required: artifact names, package layout, public headers, and exported ABI are unchanged.

Tracked by #65.

@leehack
leehack marked this pull request as ready for review August 18, 2026 14:10
@leehack
leehack marked this pull request as draft August 18, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant