Skip to content

macOS Apple Silicon (arm64) native port - #8

Open
bhodgens wants to merge 2 commits into
stgatilov:trunkfrom
bhodgens:macos-apple-silicon-port
Open

macOS Apple Silicon (arm64) native port#8
bhodgens wants to merge 2 commits into
stgatilov:trunkfrom
bhodgens:macos-apple-silicon-port

Conversation

@bhodgens

Copy link
Copy Markdown

macOS Apple Silicon (arm64) native port

Native build of The Dark Mod engine for macOS on Apple Silicon (arm64), plus a
zipsync fix so the official asset installer tool builds on macOS.

Status: working. The engine runs natively on an Apple M2 Ultra (macOS 26):
GL 4.1 Metal context, OpenAL soft with HRTF, all 64 official pk4s of the TDM
2.14 asset set mounted, main menu and in-game render loop confirmed.

What changed

All changes are gated to APPLE / MACOS_X / __APPLE__ and do not affect
Windows or Linux builds.

Engine / build system

  • CMakeLists.txt: macOS branches — sys/posix/platform_osx.cpp instead of
    platform_linux.cpp, no GLX/WGL, Cocoa/IOKit/CoreVideo frameworks, PIE,
    C-only vs C++ flags split, roqvq tool excluded + stubbed.
  • sys/cmake/homebrew_shims.cmake (new): maps conan-style imported targets
    (zlib, minizip, curl, vorbis, jpeg, png, ffmpeg) to Homebrew packages via
    pkg-config, plus a shim for a local Tracy package.
  • sys/cmake/setup_macos_tracy.sh (new): reproducible Tracy 0.13.1 bootstrap —
    downloads, SHA256-verifies, applies the same patches as the conan recipe
    (RecreateQueries + qgl* renaming), and removes the upstream
    #if !defined TRACY_ENABLE || defined __APPLE__ gate so the OpenGL backend
    compiles on Apple.
  • COMPILING_MACOS.md (new): step-by-step build/run guide.

Core fixes (required for any Apple toolchain)

  • idlib/sys/sys_defines.h: force-include <sys/param.h> before ALIGN is
    defined — Darwin's one-arg ALIGN(p) macro would otherwise shadow idlib's
    two-arg ALIGN(x, a).
  • sys/sys_public.h: MACOS_X defined as 1 (was empty, which broke
    #if MACOS_X checks in KeyInput.cpp / posix_net.cpp).
  • sys/posix/platform_osx.cpp: rewritten (main(), sysctl CPU count,
    monotonic ticks, process spawn, preferences, localization stub). Removes a
    stale duplicate main and an obsolete clock_gettime shim.
  • INFINITY macro handling: Math.h undefs it after the public header (idMath
    uses the internal value); FileSystem.cpp / Heap_Embedded.cpp restore it
    around <random> includes where libc++ needs it.

zipsync (asset installer tool)

  • tdm_installer/zipsync/StdFilesystem.cpp: use std::filesystem on Apple
    (libc++ has no <experimental/filesystem>); last_write_time via
    duration_cast (no file_time_type::clock::to_time_t).
  • tdm_installer/zipsync/CMakeLists.txt: don't link stdc++fs on Darwin.

Tests / tools

  • tests/TestRun.cpp: doctest 2.5+ moved getRegisteredTests into
    doctest::detail (version-gated).
  • tools/compilers/roqvq_stub.cpp (new): stub for RoQFileEncode_f — the
    real roqvq needs libjpeg's private jpegint.h, which Homebrew jpeg-turbo
    does not ship. Engine path still uses the .roq decoder.

How to reproduce

# 1. deps (Homebrew)
brew install zlib minizip curl libvorbis libjpeg-turbo libpng ffmpeg
# 2. tracy (installs to ~/tdm-deps)
./sys/cmake/setup_macos_tracy.sh
# 3. build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
# 4. assets + run (see COMPILING_MACOS.md)

The zipsync-built zipsync CLI was used to install the full TDM 2.14 asset
set (64 pk4s, ~5 GB) from the official mirrors with hash verification, then
thedarkmod.x64 was run successfully.

Notes / limitations

  • Engine reports "unsupported CPU" on arm64 and falls back to
    idSIMD_Generic — intentional, NEON SIMD paths are not implemented.
  • macOS GL 4.1 lacks some optional extensions (GL_ARB_buffer_storage,
    GL_ARB_multi_draw_indirect, GL_KHR_debug, ...); the engine has fallback
    paths for all of them.
  • roqvq (RoQ video encoder) tool is stubbed on macOS (libjpeg private header
    unavailable via Homebrew).
  • Tested on arm64 / macOS 26 only. Intel Macs untested.

Happy to adjust anything to fit project conventions.

Engine changes:
- CMakeLists.txt: APPLE branches for sources (platform_osx instead of
  platform_linux, no GLX/WGL), Homebrew deps via pkg-config shims,
  Cocoa/IOKit/CoreVideo link flags, PIE enabled, C-only flags split
  from C++ (-std=c++17 generator expression), roqvq excluded + stubbed.
- sys/cmake/homebrew_shims.cmake: conan-style imported targets mapped
  to Homebrew packages (zlib minizip curl vorbis vorbisfile jpeg png
  ffmpeg) + local tracy 0.13.1 shim.
- idlib/sys/sys_defines.h: force-include <sys/param.h> before defining
  ALIGN so Darwin's one-arg ALIGN(p) macro can never shadow idlib's
  two-arg ALIGN(x,a); arm64 CPUSTRING.
- idlib/sys/sys_includes.h: __APPLE__ branch (signal/pthread).
- sys/sys_public.h: MACOS_X defined as 1 (was empty; #if MACOS_X in
  KeyInput.cpp/posix_net.cpp needs a value).
- sys/posix/platform_osx.cpp: rewritten (main(), sysctl CPU count,
  monotonic-clock ticks, Sys_DoStartProcess, Sys_DoPreferences,
  OSX_GetLocalizedString stub); removes stale duplicate main and
  obsolete clock_gettime shim.
- INFINITY macro: keep #undef in Math.h (protects idMath::INFINITY),
  restore locally around <random> includes (FileSystem.cpp,
  Heap_Embedded.cpp) where libc++ needs it; Math.cpp undefs again for
  the member definition.
- tests/TestRun.cpp: doctest 2.5+ moved getRegisteredTests to
  doctest::detail (version-gated, decimal compare).
- tools/compilers/roqvq_stub.cpp: RoQFileEncode_f stub (real roqvq
  needs libjpeg private header jpegint.h, not shipped by Homebrew
  jpeg-turbo).

zipsync tool (used to install assets):
- StdFilesystem.cpp: std::filesystem on Apple (no experimental/fs in
  libc++); last_write_time via duration_cast (no clock::to_time_t).
- CMakeLists.txt: skip stdc++fs link on Darwin.

Verified: engine builds and runs natively on Apple M2 Ultra
(macos 26, arm64), GL 4.1 Metal context, OpenAL soft with HRTF,
full TDM 2.14 asset set (64 pk4s) installed via zipsync from
official mirrors.
- sys/cmake/setup_macos_tracy.sh: downloads Tracy 0.13.1, verifies
  SHA256, applies the same patches as the conan recipe (RecreateQueries,
  qgl* renaming), removes the upstream __APPLE__ gate that disables the
  OpenGL backend, and installs a CMake shim into ~/tdm-deps.
- COMPILING_MACOS.md: step-by-step build/run guide for Apple Silicon.

The script's output was verified byte-identical to the manually
prepared Tracy used for the arm64 build.
@stgatilov

Copy link
Copy Markdown
Owner

Hello!

Thank you for the PR!
I have started reviewing it...

@stgatilov stgatilov left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe I have reviewed everything.
Unfortunately, I don't have Mac and can't test things work fine, so I have to push a lot of the cleanup to your side...

To begin with, please tell me if you are okay with mass-renaming ALIGN and INFINITY macros as I proposed? That could cause some trouble merging I'm afraid...

And one general question I wanted to ask you: I see tons of code in sys/osx. Does your build use any of it? Of yes, then which part exactly? Is there anything useful there? That code is very old and 100% dead. I think think that if you have properly working build on modern Mac, wouldn't it be better for me to delete the whole directory after your PR is merged?

P.S. I'm going on vacation soon, will return back on 26-th.

Comment thread framework/FileSystem.cpp
// mac-port: idlib's Math.h #undef-ines the INFINITY macro (to protect idMath::INFINITY),
// which breaks libc++'s <random> on Darwin — restore it just for this include
#if defined(__APPLE__) && !defined(INFINITY)
#define INFINITY __builtin_inff()

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know what... I think I'll simply carpet-rename this macro to something else on SVN trunk.
It is a much better decision than to dance the colliding name all the time.
And you'll be able to delete all these changes in your PR.

Comment thread idlib/math/Math.h

#define DEG2RAD(a) ( (a) * idMath::M_DEG2RAD )
#define RAD2DEG(a) ( (a) * idMath::M_RAD2DEG )
#define DEG2RAD(a) ( (a) * idMath::M_DEG2RAD )

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please revert meaningless modifications like these.

Comment thread idlib/sys/sys_defines.h

#elif defined(__linux__) || defined(__FreeBSD__) || defined(__APPLE__) || defined(__GNUC__) || defined(__clang__)

// stgatilov/mac-port: on Darwin, <sys/param.h> -> <arm/param.h> defines a one-argument

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just recently the same happened on my daily job.
AI agent started using the name of a programmer (who is not working at the company anymore) as the magic prefix in its comments =)

Comment thread idlib/sys/sys_defines.h
// ensures the Darwin macro can never be (re)defined later.
#if defined(__APPLE__)
#include <sys/param.h>
#undef ALIGN

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try to do the same as with infinity: try to carper-rename the macro so that it does not collide anymore.

Comment thread idlib/sys/sys_includes.h
#include <signal.h>
#include <pthread.h>

#elif defined(__APPLE__)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can merge it into the previous case.

Comment thread CMakeLists.txt
if (APPLE)
ucm_remove_files(
"renderer/backend/qgl/glad_wgl.c"
"renderer/backend/qgl/glad_glx.c"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it excludes both glx and wgl versions, then how do all the function calls like qglClear compile?
How initialized all the static variables aka "this GL extension is supported", who initialized global function pointers?

In any case, I think the difference between Linux and Mac exclusion is not big enough yet to justify two separate lists. I'd say better merge them.

Comment thread CMakeLists.txt
find_package(doctest REQUIRED CONFIG)
find_package(tracy REQUIRED CONFIG)
endif()
target_link_libraries(TheDarkMod

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say empty line is missing just before this one...

Comment thread CMakeLists.txt
# note: must be after project: https://gitlab.kitware.com/cmake/cmake/-/issues/14983
check_pie_supported()
if(TDM_THIRDPARTY_ARTEFACTS)
include(ThirdParty/use_artefacts.cmake)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLDR: Would it be hard to convert homebrew_shims.cmake into a toolchain file that would allow CMake to find packages exactly the way how CMake finds them with TDM_THIRDPARTY_ARTEFACTS = OFF ?

Something went wrong here, I think you broke the main build.
I think the original if(TDM_THIRDPARTY_ARTEFACTS) should not be touched at all. Your if(APPLE) is just a separate case. Maybe you can simply block TDM_THIRDPARTY_ARTEFACTS on APPLE, or make it default to OFF there.

But I think there is more important issue to discuss here.
In the modern world, CMakeLists.txt should specify which third-party packages are needed, but should not specify how to find them. How to find packages should be passed from outside or taken from the system's CMake files.

There are two ways to build TDM on Linux today:

  1. conan build sets TDM_THIRDPARTY_ARTEFACTS = OFF and passes information about how to find dependencies directly, I think using CMAKE_TOOLCHAIN_FILE.
  2. Conanless build uses TDM_THIRDPARTY_ARTEFACTS = ON and thus includes the use_artefacts.cmake that points to prebuilt binaries. This is more or a simplification for the main case, in principle we can drop TDM_THIRDPARTY_ARTEFACTS variable and write in the readme that toolchain file must be passed...

In principle, CMake can also find system-installed libraries. It is possible to set TDM_THIRDPARTY_ARTEFACTS = OFF and try to use this approach, although it won't work because it is always necessary to patch some of them unfortunately...

I think in your case it would be better to also use TDM_THIRDPARTY_ARTEFACTS = OFF and pass toolchain file.


// TODO: switch to std::filesystem when it includes something like file_time_type::clock::to_time_t
// mac-port: libc++ has no <experimental/filesystem>; std::filesystem works there
#if defined(__APPLE__) || __cplusplus >= 201703L

@stgatilov stgatilov Aug 19, 2026

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the check for C++17.
The reason for using experimental version of filesystem is exactly that it allows to get proper timestamps for file modification dates, while standard version does not have.

#include "../../idlib/precompiled.h"
#include "../posix/posix_public.h"
//#include "../sys_local.h"
#include "../sys_local.h"

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented include just to write exactly the same on the following line?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants