Skip to content

development_setup.md

Latest commit

 

History

History
385 lines (305 loc) · 14.3 KB

development_setup.md

File metadata and controls

385 lines (305 loc) · 14.3 KB
Raw

Development setup

Zero to "all tests pass" on a fresh Mac. Or on a Linux box if you only want the portable CPU backend.

Apple Silicon (the primary path)

1. Xcode command-line tools

xcode-select --install
xcrun --show-sdk-version   # should be 14.0+ at minimum; 26.0+ unlocks Metal 4

If you're on macOS 26 (Tahoe) and the SDK version reports 14.x, run sudo xcode-select -s /Applications/Xcode.app so the full Xcode is the selected toolchain — the CLT-only path doesn't ship the macOS 26 SDK.

2. CMake (3.20+)

brew install cmake
cmake --version

3. Python (3.10+) and NumPy

For the python_basic CTest target and the Python binding:

brew install python@3.13   # or whatever 3.10+ your Brew has
python3 -m pip install --upgrade pip
python3 -m pip install numpy

If you prefer pyenv / asdf / system Python, any 3.10+ works. NumPy is the only runtime dependency.

4. Clone + build + test

git clone https://github.com/tsotchke/tensorcore.git
cd tensorcore
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j8
ctest --test-dir build --output-on-failure

Expected: 31/31 tests pass in roughly 5-15 seconds on a local Apple build.

If ctest shows fewer than 31 tests, your Python or NumPy may not be visible to CMake -- python_basic skips, dropping the count by one.

5. Quick smoke

./build/examples/hello_gemm

Expected first line: [tensorcore] loaded metallib: …/build/tensorcore.metallib. Then [tensorcore] device="Apple M2 Ultra" (or whatever chip), then tc_gemm: ok backend=simdgroup_matrix. If you see backend=simdgroup_matrix, your fast path is wired correctly.

6. The bench (optional)

./build/bench/bench_gemm
./build/bench/bench_attention
./build/bench/bench_inference_7b

See benchmarks.md for expected numbers per chip.

Non-Apple platforms (portable CPU only)

For Linux CPU nodes or Intel Macs participating in a hybrid mesh:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DTC_ENABLE_METAL=OFF
cmake --build build -j8
ctest --test-dir build --output-on-failure

This builds only the portable CPU backend: pure C/C++17, no Metal, no Apple frameworks. The C ABI is identical to the Metal-enabled build; ops that still depend on an external backend return TC_ERR_UNSUPPORTED_FAMILY cleanly.

What works on the portable CPU build:

  • tc_init / tc_shutdown / tc_device_info_get
  • tc_buffer_alloc / tc_buffer_free / tc_buffer_map / tc_buffer_size
  • tc_stream_create / tc_stream_sync / tc_stream_destroy
  • tc_gemm (all dtypes, transpose flags, batched, async)
  • tc_attention_forward / tc_attention_backward
  • tc_rmsnorm_*, tc_layernorm_*, tc_rope_*, tc_swiglu_*, tc_softmax_*, tc_adamw_step, tc_fused_rmsnorm_gemv, tc_fused_layernorm_gemv
  • tc_conv2d_forward, tc_conv2d_backward_input, tc_conv2d_backward_weight
  • tc_quantize_weights / tc_gemv_quantized (Q4_0, Q8_0)
  • tc_gguf_* (full reader surface)
  • tc_dist_* with TC_DIST_SINGLE backend (world_size=1 no-ops)
  • tc_dist_* with TC_DIST_GLOO on default Apple and portable CPU builds: TCP rendezvous, fp32 SUM/AVG/MIN/MAX all-reduce, fp16 SUM/AVG all-reduce, byte-level broadcast, allgather, and barrier
  • DiLoCo single-rank, dense multi-rank, and sparse TOPK multi-rank outer steps over TC_DIST_GLOO
  • opt-in CPU GEMM variants via TC_USE_AVX2_GEMM=1, TC_USE_AVX2_GEMM=1 TC_AVX2_THREADS=1, TC_USE_NEON_GEMM=1, and TC_USE_AMX_GEMM=1; the portable CI script smokes these in isolated Python subprocesses. Direct AMX C regressions also build in portable CPU mode but skip unless TC_RUN_AMX_GEMM_TEST=1 is set on known-good Apple-Silicon hardware. test_amx_probe runs everywhere because it checks only non-trapping AMX metadata/stub paths.
  • shared-library AVX2 throughput via build/bench/bench_gemm_shared, which links libtensorcore.so so OpenMP tile fanout is present without making the static SDK archive depend on OpenMP.
  • sparse top-k compression helpers
  • memory-tier stub baseline and portable CPU activation-checkpointing discard/realize
  • HIP and CUDA backend diagnostics with deterministic unsupported stubs
  • tc_status_string / tc_dtype_name / tc_backend_name

What doesn't (returns TC_ERR_UNSUPPORTED_FAMILY):

  • tc_dist_* with TC_DIST_RING
  • tc_dist_* with TC_DIST_GLOO bf16/int8 reductions and public generic sparse packed wire-format APIs
  • HIP/chipStar execution
  • CUDA execution
  • DiLoCo dropout tolerance and non-shipped compression modes

tc_last_backend() reports portable_cpu for every served call on this build.

Windows x86 portable CPU setup

Windows is a portable CPU target today. It builds the same public C ABI and Python ctypes surface, installs tensorcore.dll or libtensorcore.dll depending on the generator, and runs a Winsock-backed TC_DIST_GLOO loopback split-rank smoke. POSIX-only forked GLOO tests remain Unix-only.

Jack's Windows machine is reachable over Tailscale/SSH; use tailscale status to confirm its current MagicDNS name or 100.x address. The expected toolchain on that machine is:

  • Visual Studio Build Tools 2022 with Desktop development with C++ (MSVC, Windows SDK, and CMake tools).
  • Git for Windows.
  • Python 3.11 or 3.12 for the Python binding smoke.
  • Optional Ninja; otherwise use the Visual Studio generator from a Developer PowerShell or x64 Native Tools prompt.

The bootstrap script checks this state, installs Python for the current user when requested, installs Visual Studio Build Tools when run from an elevated PowerShell, and then runs the same Windows smoke used in CI:

cd "$env:USERPROFILE\src\tensorcore"
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\bootstrap_windows_cpu.ps1

# First-time toolchain install, from an Administrator PowerShell:
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\bootstrap_windows_cpu.ps1 -Install

On Jack's machine:

git clone https://github.com/tsotchke/tensorcore.git
cd tensorcore
powershell -NoProfile -ExecutionPolicy Bypass -File scripts\ci_windows_cpu.ps1

From a Unix controller on the same tailnet, use the SSH orchestrator to keep the Windows host reproducible:

mkdir -p ~/.config/tensorcore
printf '%s\n' 'TC_WINDOWS_SSH=builder@windows-host.example' \
  > ~/.config/tensorcore/windows-host.env
TC_WINDOWS_SSH_KEY="$HOME/.ssh/id_ed25519_jack" scripts/run_windows_host_smoke.sh
TC_WINDOWS_EVIDENCE_PATH=/tmp/windows-host.json \
  TC_WINDOWS_SSH_KEY="$HOME/.ssh/id_ed25519_jack" \
  scripts/run_windows_host_smoke.sh

By default the script clones the repo if missing, then uses git pull --ff-only on master before running bootstrap_windows_cpu.ps1. Set TC_WINDOWS_RESET=1 only for a clean operational gate where it is acceptable to hard-reset the remote checkout to origin/master. Set TC_WINDOWS_EVIDENCE_PATH to capture JSON evidence for scripts/check_windows_host_smoke_evidence.py and scripts/check_operational_evidence.py.

For an explicit Visual Studio x64 build:

powershell -NoProfile -ExecutionPolicy Bypass -File scripts\ci_windows_cpu.ps1 `
  -Generator "Visual Studio 17 2022" -Platform x64 `
  -BuildDir build-windows-cpu -Prefix "$env:TEMP\tensorcore-install"

The script configures TC_ENABLE_METAL=OFF, builds the portable CPU library and CTest suite, installs the native SDK, verifies the installed public headers, then imports Python against the produced tensorcore.dll with TENSORCORE_LIB set explicitly.

Optional CUDA / HIP Scaffolding

The direct NVIDIA CUDA and chipStar HIP backends are opt-in build paths:

cmake -S . -B build-cuda -DTC_ENABLE_METAL=OFF -DTC_ENABLE_CUDA=ON
cmake -S . -B build-hip  -DTC_ENABLE_METAL=OFF -DTC_ENABLE_HIP=ON

TC_ENABLE_CUDA=ON requires CMake's CUDAToolkit package with CUDA::cudart and CUDA::cublas. TC_ENABLE_HIP=ON requires HIP runtime targets for device diagnostics; hipBLAS is detected separately and enables HIP GEMM when present. If runtime dependencies are missing, configure prints a warning and falls back to the deterministic unsupported stubs used by default builds. Installed CMake packages preserve the effective backend flags and rediscover CUDA/HIP dependencies before loading exported targets. Use scripts/ci_cuda_smoke.sh and scripts/ci_hip_smoke.sh for the runtime proof on machines that have those backends installed.

The portable build does not build or install tensorcore.metallib. Consumers should treat the metallib as a Metal-backend artifact and skip that lookup when TC_ENABLE_METAL=OFF.

IDE integration

-DCMAKE_EXPORT_COMPILE_COMMANDS=ON is on by default (see CMakeLists.txt:41). After cmake -B build, a compile_commands.json sits at build/compile_commands.json. Symlink or copy to the repo root if your editor needs it there:

ln -sf build/compile_commands.json .

clangd, VSCode's C/C++ extension, and Xcode's bridge support all read this file. Metal files use clangd's metal mode (the file extension is enough).

For Python development, point your IDE at python/tensorcore/__init__.py (the binding) and python/tests/test_basic.py (the test suite).

Running the deep smoke (release_smoke.sh)

The deepest pre-release check. ~30 seconds on M2 Ultra:

cmake --install build --prefix .scratch/release-smoke/install
REQUIRE_GPU=1 scripts/release_smoke.sh

On macOS this builds the wheel and native SDK archive, verifies the install via CMake find_package and pkg-config, runs the full ctest, and writes a JSON evidence file to build/release_smoke_runtime_evidence.json. See ci_and_scripts.md § release_smoke.sh for the schema. Use scripts/ci_portable_cpu.sh plus the shell archive scripts on Linux, and scripts/ci_windows_cpu.ps1 plus the PowerShell archive scripts on Windows; the release workflow requires all three platform gates.

Common environment knobs

Build-time:

cmake -B build \
  -DCMAKE_BUILD_TYPE=Release \
  -DTC_ENABLE_METAL=ON       # OFF for portable CPU
  -DTC_ENABLE_TENSOROPS=ON   # M5 mpp::tensor_ops dispatch (SDK 26+)
  -DTC_BUILD_TESTS=ON
  -DTC_BUILD_BENCH=ON
  -DTC_BUILD_EXAMPLES=ON

Runtime:

export TENSORCORE_LIB=/opt/tensorcore/lib/libtensorcore.dylib
export TC_METALLIB=/opt/tensorcore/lib/tensorcore.metallib
export TC_USE_128_TILE=1     # opt into 128×128 GEMM tile (regresses on M2)
export TC_Q4_USE_V1=1        # revert to original Q4_0 GEMV kernel

See observability.md for the complete list.

When make is enough

The repo ships a developer Makefile. From the repo root:

make build         # cmake -B build && cmake --build build -j
make test          # the 22 ctest cases
make bench         # GEMM + attention + 7B Q4_0 benches
make hello         # ./build/examples/hello_gemm
make decode        # ./build/examples/decode_step
make train         # ./build/examples/training_step
make smoke         # release_smoke.sh with REQUIRE_GPU=1
make install       # cmake --install to .scratch/dev/install
make wheel         # build + install + reimport the wheel
make check-version # version triple consistency
make docs-check    # docs/ link integrity
make icc-audit     # ICC index + doc-coverage + shell-hardening
make all           # build + test

See make help for the full menu.

What you don't need

  • MLX. Optional; tensorcore is independent. If you want to compare fp16 GEMM TFLOPS against MLX, install separately: pip install mlx.
  • PyTorch / TensorFlow. Tensorcore's core build only needs the C ABI. An experimental PyTorch bridge lives in bindings/pytorch for fp32/bf16 CPU and PrivateUse1 host-memory matmul plus opt-in torch.matmul dispatch experiments. It uses zero-copy tensor wrappers when the runtime accepts the tensor pointer and falls back to staged buffers otherwise. Importing tensorcore_torch registers PyTorch's PrivateUse1 name as tensorcore, exposes a small torch.tensorcore runtime module, and enables direct host-memory tensor allocation on device="tensorcore". Use tensorcore_torch.pytorch_backend_state() or torch.tensorcore.backend_state() to inspect that capability state from training/deployment scripts; tensorcore_torch.matmul_eligibility() reports the exact dispatcher reason for tensorcore versus ATen fallback. tensorcore_torch.execution_state() reports which native backend served each high-value operation, whether the binding copied buffers or used zero-copy memory, and any intentional fallback reason. The AMP policy is available through tensorcore_torch.pytorch_amp_supported_dtypes() and torch.tensorcore.get_amp_supported_dtype(). When PyTorch is installed, run REQUIRE_PYTORCH=1 REQUIRE_PYTORCH_BACKEND=1 scripts/ci_pytorch_smoke.sh to force-build the bridge and validate fp32/bf16, empty matmul, dispatcher, autograd fallback, PrivateUse1 allocation, and registration/backend-state behavior.
  • CUDA. Obviously.
  • Real GGUF model. The bench harness uses synthetic Q4_0 weights; full inference against a real GGUF is a v0.2 deliverable.
  • GitHub auth. Cloning is anonymous; only the gh CLI in release_process.md needs auth.

Troubleshooting first build

If cmake -B build fails with find_library(METAL_FRAMEWORK) errors, your Xcode CLT install is incomplete:

sudo xcode-select -s /Applications/Xcode.app
xcrun --show-sdk-path   # should print a path under /Applications/Xcode.app

If cmake --build build fails on .metal compilation with error: unknown intrinsic 'air.…', your Xcode is 17+ and you have stale CMake cache from an earlier 16.x configure. Wipe and reconfigure:

rm -rf build
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j8

If ctest shows python_basic failing with a NumPy error, run with PYTHONPATH=$PWD/python TENSORCORE_LIB=$PWD/build/libtensorcore.dylib python3 python/tests/test_basic.py directly to see the error.

See troubleshooting.md for runtime issues after the build succeeds.

See also