Skip to content

Rearchitect vcpkg port for official submission#1

Open
bmehta001 wants to merge 21 commits into
mainfrom
bhamehta/vcpkg-port-rearchitect
Open

Rearchitect vcpkg port for official submission#1
bmehta001 wants to merge 21 commits into
mainfrom
bhamehta/vcpkg-port-rearchitect

Conversation

@bmehta001
Copy link
Copy Markdown
Owner

@bmehta001 bmehta001 commented Mar 4, 2026

  • Rearchitects the vcpkg port from a shell-script-based build with different behavior for different platforms, into a pure CMake workflow for official submission to the vcpkg registry that should works on all supported platforms
  • Have created vcpkg tests and a sample project where this library was imported into ONNXRuntime and successfully tested these and all existing unit/functional tests where applicable, on Windows, WSL, Android Simulator on WSL with NDK, macOS, and iOS Simulator on macOS

Changes:

vcpkg

  • For portfile.cmake, use vcpkg_from_github() + vcpkg_cmake_configure() + vcpkg_cmake_install() + vcpkg_cmake_config_fixup()
  • Added vcpkg.json manifest with dependency declarations (sqlite3, zlib, nlohmann-json, curl), platform constraints (!uwp), and optional feature flags (azmon, privacyguard, cds, signals, sanitizer, liveeventinspector).
  • Added usage file (vcpkg usage instructions shown after install)
  • Removed: CONTROL, TODO.md, get_repo_name.sh, response_file_linux.txt, response_file_mac.txt, v142-build.patch, since they are now redundant or outdated

CMake (had to modify to fix some behavior in legacy builds and for vcpkg)

  • Use MATSDK_USE_VCPKG_DEPS option to detect vcpkg toolchain and thus
    • Resolve dependencies with find_package() ; when OFF, use existing vendored/system-installed workflow
    • Skip manual compiler/platform flag configuration (architecture, paths, etc.) for vcpkg builds
    • Added CMake package config (cmake/MSTelemetryConfig.cmake.in) to generate find_package(MSTelemetry CONFIG) support with transitive dependency re-finding (find_dependency() for sqlite3, zlib, nlohmann-json, curl, pthreads as needed).
    • Added install/export targets — install(TARGETS mat EXPORT MSTelemetryTargets ...) with GNUInstallDirs, write_basic_package_version_file(), and configure_package_config_file() to specify paths in standardized way
    • Deduplicated and simplified dependency linking — replaced the tangled shared/static forking with a clear three-way branch: vcpkg mode (imported targets), Android legacy (bundled source compilation), and Linux/macOS legacy
      (-lsqlite3 -lz).
    • Added Android HTTP client selection (HttpClient_Android.cpp is now compiled on Android instead of HttpClient_Curl.cpp)
    • Link Apple frameworks like CoreFoundation, Foundation, CFNetwork, Network, SystemConfiguration, UIKit (iOS) / IOKit (macOS) via target_link_libraries() with -framework syntax.
    • Used message(STATUS ...) consistently throughout (or FATAL_ERROR) to use official CMake keywords
    • Quote EXISTS path arguments to prevent CMP0012 warnings.

Build flags

  • Added -fno-finite-math-only (helps fix issues for ORT builds consuming this library where -ffast-math is enabled), since sqlite3 uses the INFINITY macro, which is undefined under -ffinite-math-only
  • Move -Wno-reorder to apply only to CMAKE_CXX_FLAGS (since it's a C++-only flag)
  • Split GCC vs Clang warning flags (GCC doesn't support -Wno-unknown-warning-option)
  • Removed ZLIB_WINAPI from vcpkg builds since vcpkg's zlib does not use WINAPI/STDCALL.
  • Updated MATSDK_API_VERSION from 3.4 to 3.10 (which I assume is current SDK version)

Obj-C/Swift wrapper resilience

  • Added compile-time feature detection to ODWLogger.mm to conditionally compile Privacy Guard and Sanitizer integration. When modules aren't available (e.g., submodule not cloned), calls log a warning instead of failing to compile.
  • Updated Package.swift to add Privacy Guard and Sanitizer compilation conditions.
  • Updated Swift wrapper to use #if canImport() for optional module imports.

Updated samples

  • Added shared MSTelemetrySample.cmake for ease of updating all sample CMakeLists.txt files now include this shared module that handles both vcpkg (find_package) and legacy (vendored) dependency resolution.
  • Removed hardcoded install paths from all sample CMakeLists.txt files, since it was causing local builds to fail

Test infrastructure (tests/vcpkg/)

  • Added standalone vcpkg consumer test (main.cpp, CMakeLists.txt, vcpkg.json) using core APIs (LogManager, EventProperties, PII annotations, event priority) and verifies find_package(MSTelemetry CONFIG) works correctly.
  • Added platform-specific test scripts:
    - test-vcpkg-windows.ps1 — auto-detects host architecture (x64/ARM64), finds VS via vswhere, builds with x64-windows-static or arm64-windows-static, runs 8 integration tests.
    - test-vcpkg-linux.sh — detects host triplet, builds and runs tests.
    - test-vcpkg-macos.sh — builds with host-native triplet, runs tests.
    - test-vcpkg-ios.sh — builds for both arm64-ios (device) and simulator (arm64-ios-simulator), verifies Mach-O binary architecture.
    - test-vcpkg-android.sh — cross-compiles for configurable ABI (arm64-v8a, armeabi-v7a, x86_64, x86), verifies ELF binary production.
  • Added README.md with usage instructions for all platforms.

Other changes

  • Added .gitattributes to marks tools/ports/, tests/vcpkg/, and docs/building-with-vcpkg.md with export-ignore so they are excluded from GitHub tarballs and so vcpkg files don't affect SHA needed for vcpkg, since vcpkg doesn't need the port files themselves when downloading the GitHub source tarball
  • Updated .gitignore to ignores vcpkg test build artifacts.
  • Fixed Logger.cpp destructor — removed LOG_TRACE call from ~Logger() to prevent static-destruction-order crash on iOS simulator (where debugLogMutex may be destroyed before the Logger instance). If this is an issue, I can revert it, as I am not sure if it would be an actual issue in a real iOS deployment scenario.
  • Removed stale header references from .vcxproj and .vcxproj.filters files (referenced files that no longer exist). If I was in doubt, I kept it in.
  • Updated docs (docs/building-with-vcpkg.md) to cover all platforms, overlay port usage, feature flags, and troubleshooting.

After merging

  • Need a separate commit to update the SHA and point REF at a release tag (I assume will be 3.10.42.1)
  • Then will officially submit port to vcpkg registry
    • May be possible to set it so that future releases will automatically result in updated vcpkg ports

@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch 6 times, most recently from 8c1d2b5 to 0999317 Compare March 19, 2026 04:34
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch 6 times, most recently from 7a5f685 to 0d9bb76 Compare April 1, 2026 16:55
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch 8 times, most recently from 6f874e4 to 5405100 Compare April 29, 2026 21:37
- Modernize CMakeLists.txt: flatten nesting, deduplicate, use
  consistent quoting and variable patterns
- Remove stale header references from vcxproj and vcxitems files
- Simplify test CMakeLists.txt files
- Fix CMake conventions in packaging scripts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch from ffc37ed to ed30a8a Compare April 29, 2026 21:42
bmehta001 and others added 5 commits April 30, 2026 06:57
Keep the before.targets toolset selection deterministic on newer
Visual Studio hosts, but only set the MIP props fallback when a
consumer has not already chosen a toolset. While addressing the
MSBuild review comments, also point the optional module test
conditions and source paths at the real lib/modules locations and
use CPACK_PACKAGE_FILE_NAME for the RPM status message.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch from 6d9b5c4 to c857e4b Compare May 2, 2026 01:58
bmehta001 and others added 13 commits May 11, 2026 12:34
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Quote the optional ECS test.json configure_file input and output paths so CMake handles source or build directories that contain spaces.

Files changed:

- tests/functests/CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
PR microsoft#1416 inadvertently regressed PR microsoft#1415 by removing
-fno-finite-math-only from the three Unix REL_FLAGS branches and adding
-Wno-nan-infinity-disabled to silence the Clang diagnostic. That left
release builds using -ffast-math without preserving the NaN/Inf
semantics needed by nlohmann::json and SQLite paths.

Remove -ffast-math entirely from the GCC, AppleClang, and generic Clang
release flags rather than relying on -fno-finite-math-only to partially
undo it. This SDK is not floating-point compute-bound; its hot paths are
string, Bond/JSON serialization, HTTP I/O, and SQLite reads/writes.

Avoiding -ffast-math:
- preserves std::isnan/std::isinf behavior for JSON and storage code,
- avoids compiler/runtime fast-math side effects such as x86 GCC's
  crtfastmath.o changing MXCSR FTZ/DAZ behavior process-wide, and
- aligns with SQLite's guidance to avoid fast-math.

This subsumes microsoft#1415's partial mitigation and aligns with microsoft#1392's intent,
extended to GCC because GCC fast-math has the same broad assumptions and
runtime side effects. Also remove -Wno-nan-infinity-disabled because the
warning should not be suppressed once the cause is gone.

Validation:
- CMake Release build of UnitTests on macOS arm64 (AppleClang 21).
- UnitTests passed on macOS arm64.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Avoid carrying a confusing note about warning flags that are not used, and keep CMakeLists.txt focused on active compiler settings.

Files changed:

- CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update vendored zlib to 1.3.2

Refresh vendored zlib sources, headers, documentation, and minizip files to upstream v1.3.2 so the SDK tracks the current supported zlib release.

Preserve the SDK's symbol-prefix shim and legacy Windows project wiring, and update the zlib CMake shim so out-of-source builds no longer mutate zconf.h or define Windows macros on non-Windows hosts.

Files changed:

- cgmanifest.json

- zlib core sources, headers, docs, and minizip support files

- zlib/names.h and zlib/CMakeLists.txt repo-specific shims

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Move zlib MSBuild projects outside vendored tree

Keep the vendored zlib folder closer to the upstream 1.3.2 layout while preserving the legacy MSBuild project used by the Windows solution.

Files changed:

- Move vc14 zlib Visual Studio projects to third_party/Solutions/zlib/vc14

- Update solution, sample, and test ProjectReference paths

- Remove unused vc9/vc10/vc11 zlib project directories

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use a real ZLIB package in the non-vcpkg Linux/macOS path instead of a bare imported z target with no location. This restores the regular debug test path on Ubuntu/macOS while keeping the vendored-vs-vcpkg dependency split intact.

Files changed:
- lib/CMakeLists.txt
- tests/functests/CMakeLists.txt
- tests/unittests/CMakeLists.txt

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@bmehta001 bmehta001 force-pushed the bhamehta/vcpkg-port-rearchitect branch from e6fd7e3 to 9e2b8fe Compare May 22, 2026 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant