macOS Apple Silicon (arm64) native port - #8
Conversation
Engine changes: - CMakeLists.txt: APPLE branches for sources (platform_osx instead of platform_linux, no GLX/WGL), Homebrew deps via pkg-config shims, Cocoa/IOKit/CoreVideo link flags, PIE enabled, C-only flags split from C++ (-std=c++17 generator expression), roqvq excluded + stubbed. - sys/cmake/homebrew_shims.cmake: conan-style imported targets mapped to Homebrew packages (zlib minizip curl vorbis vorbisfile jpeg png ffmpeg) + local tracy 0.13.1 shim. - idlib/sys/sys_defines.h: force-include <sys/param.h> before defining ALIGN so Darwin's one-arg ALIGN(p) macro can never shadow idlib's two-arg ALIGN(x,a); arm64 CPUSTRING. - idlib/sys/sys_includes.h: __APPLE__ branch (signal/pthread). - sys/sys_public.h: MACOS_X defined as 1 (was empty; #if MACOS_X in KeyInput.cpp/posix_net.cpp needs a value). - sys/posix/platform_osx.cpp: rewritten (main(), sysctl CPU count, monotonic-clock ticks, Sys_DoStartProcess, Sys_DoPreferences, OSX_GetLocalizedString stub); removes stale duplicate main and obsolete clock_gettime shim. - INFINITY macro: keep #undef in Math.h (protects idMath::INFINITY), restore locally around <random> includes (FileSystem.cpp, Heap_Embedded.cpp) where libc++ needs it; Math.cpp undefs again for the member definition. - tests/TestRun.cpp: doctest 2.5+ moved getRegisteredTests to doctest::detail (version-gated, decimal compare). - tools/compilers/roqvq_stub.cpp: RoQFileEncode_f stub (real roqvq needs libjpeg private header jpegint.h, not shipped by Homebrew jpeg-turbo). zipsync tool (used to install assets): - StdFilesystem.cpp: std::filesystem on Apple (no experimental/fs in libc++); last_write_time via duration_cast (no clock::to_time_t). - CMakeLists.txt: skip stdc++fs link on Darwin. Verified: engine builds and runs natively on Apple M2 Ultra (macos 26, arm64), GL 4.1 Metal context, OpenAL soft with HRTF, full TDM 2.14 asset set (64 pk4s) installed via zipsync from official mirrors.
- sys/cmake/setup_macos_tracy.sh: downloads Tracy 0.13.1, verifies SHA256, applies the same patches as the conan recipe (RecreateQueries, qgl* renaming), removes the upstream __APPLE__ gate that disables the OpenGL backend, and installs a CMake shim into ~/tdm-deps. - COMPILING_MACOS.md: step-by-step build/run guide for Apple Silicon. The script's output was verified byte-identical to the manually prepared Tracy used for the arm64 build.
|
Hello! Thank you for the PR! |
stgatilov
left a comment
There was a problem hiding this comment.
I believe I have reviewed everything.
Unfortunately, I don't have Mac and can't test things work fine, so I have to push a lot of the cleanup to your side...
To begin with, please tell me if you are okay with mass-renaming ALIGN and INFINITY macros as I proposed? That could cause some trouble merging I'm afraid...
And one general question I wanted to ask you: I see tons of code in sys/osx. Does your build use any of it? Of yes, then which part exactly? Is there anything useful there? That code is very old and 100% dead. I think think that if you have properly working build on modern Mac, wouldn't it be better for me to delete the whole directory after your PR is merged?
P.S. I'm going on vacation soon, will return back on 26-th.
| // mac-port: idlib's Math.h #undef-ines the INFINITY macro (to protect idMath::INFINITY), | ||
| // which breaks libc++'s <random> on Darwin — restore it just for this include | ||
| #if defined(__APPLE__) && !defined(INFINITY) | ||
| #define INFINITY __builtin_inff() |
There was a problem hiding this comment.
You know what... I think I'll simply carpet-rename this macro to something else on SVN trunk.
It is a much better decision than to dance the colliding name all the time.
And you'll be able to delete all these changes in your PR.
|
|
||
| #define DEG2RAD(a) ( (a) * idMath::M_DEG2RAD ) | ||
| #define RAD2DEG(a) ( (a) * idMath::M_RAD2DEG ) | ||
| #define DEG2RAD(a) ( (a) * idMath::M_DEG2RAD ) |
There was a problem hiding this comment.
Please revert meaningless modifications like these.
|
|
||
| #elif defined(__linux__) || defined(__FreeBSD__) || defined(__APPLE__) || defined(__GNUC__) || defined(__clang__) | ||
|
|
||
| // stgatilov/mac-port: on Darwin, <sys/param.h> -> <arm/param.h> defines a one-argument |
There was a problem hiding this comment.
Just recently the same happened on my daily job.
AI agent started using the name of a programmer (who is not working at the company anymore) as the magic prefix in its comments =)
| // ensures the Darwin macro can never be (re)defined later. | ||
| #if defined(__APPLE__) | ||
| #include <sys/param.h> | ||
| #undef ALIGN |
There was a problem hiding this comment.
I'll try to do the same as with infinity: try to carper-rename the macro so that it does not collide anymore.
| #include <signal.h> | ||
| #include <pthread.h> | ||
|
|
||
| #elif defined(__APPLE__) |
There was a problem hiding this comment.
I think you can merge it into the previous case.
| if (APPLE) | ||
| ucm_remove_files( | ||
| "renderer/backend/qgl/glad_wgl.c" | ||
| "renderer/backend/qgl/glad_glx.c" |
There was a problem hiding this comment.
If it excludes both glx and wgl versions, then how do all the function calls like qglClear compile?
How initialized all the static variables aka "this GL extension is supported", who initialized global function pointers?
In any case, I think the difference between Linux and Mac exclusion is not big enough yet to justify two separate lists. I'd say better merge them.
| find_package(doctest REQUIRED CONFIG) | ||
| find_package(tracy REQUIRED CONFIG) | ||
| endif() | ||
| target_link_libraries(TheDarkMod |
There was a problem hiding this comment.
I'd say empty line is missing just before this one...
| # note: must be after project: https://gitlab.kitware.com/cmake/cmake/-/issues/14983 | ||
| check_pie_supported() | ||
| if(TDM_THIRDPARTY_ARTEFACTS) | ||
| include(ThirdParty/use_artefacts.cmake) |
There was a problem hiding this comment.
TLDR: Would it be hard to convert homebrew_shims.cmake into a toolchain file that would allow CMake to find packages exactly the way how CMake finds them with TDM_THIRDPARTY_ARTEFACTS = OFF ?
Something went wrong here, I think you broke the main build.
I think the original if(TDM_THIRDPARTY_ARTEFACTS) should not be touched at all. Your if(APPLE) is just a separate case. Maybe you can simply block TDM_THIRDPARTY_ARTEFACTS on APPLE, or make it default to OFF there.
But I think there is more important issue to discuss here.
In the modern world, CMakeLists.txt should specify which third-party packages are needed, but should not specify how to find them. How to find packages should be passed from outside or taken from the system's CMake files.
There are two ways to build TDM on Linux today:
conan buildsets TDM_THIRDPARTY_ARTEFACTS = OFF and passes information about how to find dependencies directly, I think using CMAKE_TOOLCHAIN_FILE.- Conanless build uses TDM_THIRDPARTY_ARTEFACTS = ON and thus includes the
use_artefacts.cmakethat points to prebuilt binaries. This is more or a simplification for the main case, in principle we can drop TDM_THIRDPARTY_ARTEFACTS variable and write in the readme that toolchain file must be passed...
In principle, CMake can also find system-installed libraries. It is possible to set TDM_THIRDPARTY_ARTEFACTS = OFF and try to use this approach, although it won't work because it is always necessary to patch some of them unfortunately...
I think in your case it would be better to also use TDM_THIRDPARTY_ARTEFACTS = OFF and pass toolchain file.
|
|
||
| // TODO: switch to std::filesystem when it includes something like file_time_type::clock::to_time_t | ||
| // mac-port: libc++ has no <experimental/filesystem>; std::filesystem works there | ||
| #if defined(__APPLE__) || __cplusplus >= 201703L |
There was a problem hiding this comment.
Please remove the check for C++17.
The reason for using experimental version of filesystem is exactly that it allows to get proper timestamps for file modification dates, while standard version does not have.
| #include "../../idlib/precompiled.h" | ||
| #include "../posix/posix_public.h" | ||
| //#include "../sys_local.h" | ||
| #include "../sys_local.h" |
There was a problem hiding this comment.
Commented include just to write exactly the same on the following line?
macOS Apple Silicon (arm64) native port
Native build of The Dark Mod engine for macOS on Apple Silicon (arm64), plus a
zipsyncfix so the official asset installer tool builds on macOS.Status: working. The engine runs natively on an Apple M2 Ultra (macOS 26):
GL 4.1 Metal context, OpenAL soft with HRTF, all 64 official pk4s of the TDM
2.14 asset set mounted, main menu and in-game render loop confirmed.
What changed
All changes are gated to
APPLE/MACOS_X/__APPLE__and do not affectWindows or Linux builds.
Engine / build system
CMakeLists.txt: macOS branches —sys/posix/platform_osx.cppinstead ofplatform_linux.cpp, no GLX/WGL, Cocoa/IOKit/CoreVideo frameworks, PIE,C-only vs C++ flags split, roqvq tool excluded + stubbed.
sys/cmake/homebrew_shims.cmake(new): maps conan-style imported targets(zlib, minizip, curl, vorbis, jpeg, png, ffmpeg) to Homebrew packages via
pkg-config, plus a shim for a local Tracy package.
sys/cmake/setup_macos_tracy.sh(new): reproducible Tracy 0.13.1 bootstrap —downloads, SHA256-verifies, applies the same patches as the conan recipe
(RecreateQueries + qgl* renaming), and removes the upstream
#if !defined TRACY_ENABLE || defined __APPLE__gate so the OpenGL backendcompiles on Apple.
COMPILING_MACOS.md(new): step-by-step build/run guide.Core fixes (required for any Apple toolchain)
idlib/sys/sys_defines.h: force-include<sys/param.h>beforeALIGNisdefined — Darwin's one-arg
ALIGN(p)macro would otherwise shadow idlib'stwo-arg
ALIGN(x, a).sys/sys_public.h:MACOS_Xdefined as1(was empty, which broke#if MACOS_Xchecks in KeyInput.cpp / posix_net.cpp).sys/posix/platform_osx.cpp: rewritten (main(), sysctl CPU count,monotonic ticks, process spawn, preferences, localization stub). Removes a
stale duplicate
mainand an obsoleteclock_gettimeshim.Math.hundefs it after the public header (idMathuses the internal value);
FileSystem.cpp/Heap_Embedded.cpprestore itaround
<random>includes where libc++ needs it.zipsync (asset installer tool)
tdm_installer/zipsync/StdFilesystem.cpp: usestd::filesystemon Apple(libc++ has no
<experimental/filesystem>);last_write_timeviaduration_cast(nofile_time_type::clock::to_time_t).tdm_installer/zipsync/CMakeLists.txt: don't linkstdc++fson Darwin.Tests / tools
tests/TestRun.cpp: doctest 2.5+ movedgetRegisteredTestsintodoctest::detail(version-gated).tools/compilers/roqvq_stub.cpp(new): stub forRoQFileEncode_f— thereal roqvq needs libjpeg's private
jpegint.h, which Homebrew jpeg-turbodoes not ship. Engine path still uses the .roq decoder.
How to reproduce
The zipsync-built
zipsyncCLI was used to install the full TDM 2.14 assetset (64 pk4s, ~5 GB) from the official mirrors with hash verification, then
thedarkmod.x64was run successfully.Notes / limitations
idSIMD_Generic— intentional, NEON SIMD paths are not implemented.GL_ARB_multi_draw_indirect, GL_KHR_debug, ...); the engine has fallback
paths for all of them.
unavailable via Homebrew).
Happy to adjust anything to fit project conventions.