Add a macOS Apple Silicon CI job - #247
Conversation
Every job was ubuntu-latest, so pamica/tests/mlx_tests/ had never run in CI and the Accelerate BLAS path was untested. The second gap is not theoretical: a test_ng_convergence.py assertion stayed green on Linux through #241 and failed immediately on macOS, because the ~1e-6 trajectory shift moved the stopping iteration off the multiple of mir_step the assertion assumed. macos-26 is the current arm64 image and standard runners are free for public repositories. The job installs the mlx extra and asserts the import rather than letting mlx_tests self-skip into a green no-op. Refs #246. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review findings. Assert mx.default_device().type == gpu, not just that mlx imports. The mlx_tests module is gated on both (test_mlx_backend.py:28-33), so a runner that imported mlx and fell back to CPU would skip all 11 tests and still report success. The hosted macos-26 image does expose the GPU -- all 11 passed on the first run -- but nothing in the job would have caught it if a future image stopped. Add PYTORCH_ENABLE_MPS_FALLBACK=1. The two Linux jobs already set it defensively; this is the only job whose hardware can actually auto-select MPS. Add EXTRA_LDFLAGS=-static-libquadmath so the build really is the release-binaries.yml macos-arm64 recipe the comment claims. Correct the -m 'not slow' rationale: it was attributed to Rosetta and the x86_64 amica15mac, but most slow tests never invoke a binary, and the two that do resolve through PAMICA_NATIVE_BINARY (native/resolver.py:112), which this job sets to the arm64 binary it just built. They are excluded for wall-clock, and enabling them is a real follow-up rather than blocked work. Refs #246. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Findings addressed in da41ebd. One correction on the Critical, with evidence. Critical 1 — "the MLX tests will very likely still self-skip": empirically not the case. I checked the completed run's log rather than reasoning about it. On the hosted The 10 skips are That said, the finding is right as insurance and I have implemented it. You correctly spotted the second gate at Important 2 — the Important 3 — Important 4 — Suggestion — stale coverage comment. Updated to state why the omission is still required (this job runs Suggestion — |
Closes #246 (first half: the MLX/Accelerate gap).
Why
Every job in
ci.ymlwasubuntu-latest. Two things were therefore untested:pamica/tests/mlx_tests/had never run in CI. MLX is the recommendedApple GPU backend per
AGENTS.md;pyproject.tomleven excludespamica/mlx_impl/*from coverage with the comment that CI "cannot exerciseit and always measures 0%".
test_mir_history_survives_keep_best_restoreassertion was green on Weight nd by pre-update model weights #241'sPR CI and failed immediately after merge on macOS. The cause was not a stale
base — the ~1e-6 trajectory shift moved the stopping iteration from 55 to 54,
and the assertion only held when the stop landed on a multiple of
mir_step.Linux never saw it. Adding CI on
devpushes would not have caught thiseither; only a macOS runner would.
What
One
test-macosjob onmacos-26(current arm64 image;macos-14is arm64 toobut deprecated). Standard runners are free and unmetered for public repos, so
this costs nothing — deliberately not a
-large/-xlargevariant, which arebilled even here.
Verified locally on macOS 26.6.1 arm64 before pushing:
mlx.core.__version__->0.32.0, so the import assertion is realLAPACK_LIBS="-framework Accelerate" bash native/build.shbuilds cleanlypamica/tests/mlx_tests/collects 11 tests and runsThe MLX import is asserted explicitly rather than left to
mlx_tests'self-skip: a silent skip would leave this job green while testing nothing,
which is the exact failure mode it exists to prevent.
--no-covbecause the Linux job owns the coverage gate and the Codecov upload;a second partial run would report a different total for the same commit.
Deliberately not in scope
slowFortran-parity tests on macOS. They drive the bundledx86_64
amica15mac, which needs Rosetta on an arm64 runner. Pointing them atthe arm64 binary this job already builds is the natural follow-up; left in
CI runs Linux only: MLX backend and the Apple Silicon numerical path are never tested #246.
push: branches: [dev]. Worth doing, but a separate concern, and note itwould not have caught the failure described above.
sccn/pAMICAis public and GitHub advisesself-hosted runners only for private repos (a fork PR can run arbitrary code
on an unsandboxed runner). Hosted
macos-26covers the same ground with noexposure. Self-hosted remains the only option for CUDA.