From 08f8e9bbdc083c3030ed4bbe31fd5ef9c9fbe93b Mon Sep 17 00:00:00 2001 From: Seyed Yahya Shirazi Date: Sat, 15 Aug 2026 23:32:07 -0700 Subject: [PATCH 1/2] Record the MLX convergence-stop divergence The backend table listed Newton, PDF families, sharing, rejection, precision and rank detection, but not the convergence stops -- and MLX implements neither. It has no min_dll/use_min_dll/maxincs and no min_nd/use_grad_norm/ndtmpsum, so its only outcomes are max_iter, lrate_floor and the degenerate reasons: an MLX fit always runs the full budget while the other backends stop on likelihood stagnation. This is the one MLX limitation that does not fail loudly. Unsupported parameters are absent from the constructor (TypeError) and do_newton / non-GG pdftype raise NotImplementedError, but there is no parameter to reject for a stop that simply does not exist. Also correct the keep_best / MIR / persistence rows, which differ per backend rather than tracking MLX. Refs #248. Co-Authored-By: Claude Opus 5 --- docs/guides/amica-differences.md | 35 +++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/docs/guides/amica-differences.md b/docs/guides/amica-differences.md index df75575..43bba4d 100644 --- a/docs/guides/amica-differences.md +++ b/docs/guides/amica-differences.md @@ -120,6 +120,35 @@ Separate from reference divergences: the optional MLX backend is a subset. | Outlier rejection | yes | yes | no | yes | | Precision | f64/f32 | f64 | f32 only | f64 | | Rank detection | yes | yes | yes | yes (absolute floor) | - -MLX limitations raise `NotImplementedError` rather than differing silently, per -`.rules/backend_parity.md`. +| `min_dll` stop | yes | yes | **no** | yes | +| `min_nd` stop / gradient norm | yes | yes | **no** | yes | +| `keep_best` best-iterate restore | yes | no | no | n/a | +| MIR diagnostic | yes | no | no | n/a | +| Persistence | `state_dict` | EEGLAB `amicaout` | none | EEGLAB `amicaout` | + +Most MLX limitations fail loudly: `do_newton=True` and non-GG `pdftype` raise +`NotImplementedError`, and every unsupported parameter is simply absent from the +constructor, so passing it raises `TypeError`. + +**The convergence stops are the exception, and the one difference to plan +around.** `pamica/mlx_impl/core.py` implements neither stop — it has no +`min_dll`/`use_min_dll`/`maxincs` and no `min_nd`/`use_grad_norm`/`ndtmpsum`, so +its only outcomes are `max_iter`, `lrate_floor`, and the degenerate reasons +(`nan_ll`, `singular_ll`, `nan_params`). **An MLX fit therefore always runs the +full iteration budget**, while the other backends stop on likelihood stagnation +(`min_dll`, on by default at `1e-9`) — measured at iteration 326-1076 on the +bundled sample, depending on the BLAS build. + +Nothing warns you about this, because there is no parameter to reject. Two +consequences: + +- Moving a working configuration from the PyTorch backend to MLX for the + Apple-GPU speedup does more work than the same `max_iter` implies. Choose + `max_iter` for MLX from where the other backends actually converge on your + data, not from their default budget. +- MLX cannot be evaluated on gradient-norm behaviour at all. Where this + documentation says `min_nd` is unreachable "in all three implementations" + (see the convergence section of [validation](validation.md)), that means + Fortran, PyTorch and NumPy; MLX never computes the quantity. + +Tracked in issue #248. From f2ca8d7114f987ee51a53646cb46b130be7735ba Mon Sep 17 00:00:00 2001 From: Seyed Yahya Shirazi Date: Sat, 15 Aug 2026 23:38:39 -0700 Subject: [PATCH 2/2] Make the MLX stop divergence self-contained and accurate Two review findings, same root cause: the section was written against PR #244's validation.md rewrite, which is not on dev. It cross-referenced a claim ('in all three implementations') that exists only on that branch, so merging in the wrong order would have made the reference false. Dropped; the point is now stated without asserting what another page says. It also framed 'always runs the full budget' as unique to MLX, citing the iteration 326-1076 min_dll measurement. But PyTorch reaches its own max_iter=100 default first, so that contrast was wrong in the same way the validation.md table was before #244 corrected it. The real distinction is what raising max_iter buys: on the other backends a larger budget lets the likelihood stop end the fit, on MLX it is simply spent. Also name numpy_impl's min_grad_norm in the row and the prose. Co-Authored-By: Claude Opus 5 --- docs/guides/amica-differences.md | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/docs/guides/amica-differences.md b/docs/guides/amica-differences.md index 43bba4d..50cdc00 100644 --- a/docs/guides/amica-differences.md +++ b/docs/guides/amica-differences.md @@ -121,7 +121,7 @@ Separate from reference divergences: the optional MLX backend is a subset. | Precision | f64/f32 | f64 | f32 only | f64 | | Rank detection | yes | yes | yes | yes (absolute floor) | | `min_dll` stop | yes | yes | **no** | yes | -| `min_nd` stop / gradient norm | yes | yes | **no** | yes | +| `min_nd` stop / gradient norm | yes | yes (as `min_grad_norm`) | **no** | yes | | `keep_best` best-iterate restore | yes | no | no | n/a | | MIR diagnostic | yes | no | no | n/a | | Persistence | `state_dict` | EEGLAB `amicaout` | none | EEGLAB `amicaout` | @@ -134,21 +134,25 @@ constructor, so passing it raises `TypeError`. around.** `pamica/mlx_impl/core.py` implements neither stop — it has no `min_dll`/`use_min_dll`/`maxincs` and no `min_nd`/`use_grad_norm`/`ndtmpsum`, so its only outcomes are `max_iter`, `lrate_floor`, and the degenerate reasons -(`nan_ll`, `singular_ll`, `nan_params`). **An MLX fit therefore always runs the -full iteration budget**, while the other backends stop on likelihood stagnation -(`min_dll`, on by default at `1e-9`) — measured at iteration 326-1076 on the -bundled sample, depending on the BLAS build. +(`nan_ll`, `singular_ll`, `nan_params`). + +The distinction is not that an MLX fit runs to `max_iter` — a PyTorch fit does +too at its own `max_iter=100` default, because `min_dll` needs several hundred +iterations to trigger on a recording this size. The distinction is what raising +`max_iter` buys you. On the other backends, a larger budget lets the likelihood +stop end the fit when it stops improving. **On MLX a larger budget is simply +spent**, however long the fit has stopped making progress. Nothing warns you about this, because there is no parameter to reject. Two consequences: - Moving a working configuration from the PyTorch backend to MLX for the - Apple-GPU speedup does more work than the same `max_iter` implies. Choose - `max_iter` for MLX from where the other backends actually converge on your - data, not from their default budget. -- MLX cannot be evaluated on gradient-norm behaviour at all. Where this - documentation says `min_nd` is unreachable "in all three implementations" - (see the convergence section of [validation](validation.md)), that means - Fortran, PyTorch and NumPy; MLX never computes the quantity. + Apple-GPU speedup does more work than the same `max_iter` implies. Size + `max_iter` for MLX from where the other backends actually stop improving on + your data. +- MLX cannot be evaluated on gradient-norm behaviour at all, because it never + computes the quantity. Any statement elsewhere in these guides about whether + the `min_nd` threshold is reachable covers Fortran, PyTorch and NumPy only. + (`numpy_impl` spells that same threshold `min_grad_norm`.) Tracked in issue #248.