Skip to content

fix(optim): ship the CMA-ES distribution mean, not the noisy xbest (#69)#112

Open
minion1227 wants to merge 1 commit into
mini-router:mainfrom
minion1227:sn74-minion1227-69
Open

fix(optim): ship the CMA-ES distribution mean, not the noisy xbest (#69)#112
minion1227 wants to merge 1 commit into
mini-router:mainfrom
minion1227:sn74-minion1227-69

Conversation

@minion1227

Copy link
Copy Markdown
Contributor

Fixes #69

Re-submitted from my personal sn74-minion1227-69 branch per CONTRIBUTOIN.md (previous PR #85 used a non-conforming branch name).

Problem

SepCMAES.best() returned pycma’s result.xbest — the single candidate with the best evaluated fitness across the run. TRINITY’s training fitness J(θ) = E[R(τ)] is a stochastic signal: each candidate is scored on a freshly sampled minibatch with a sampled policy (sample=True). Under a noisy objective, xbest is selection-on-noise — the luckiest evaluation, not the policy with the best expected reward. Since train.py saves best()[0] as best_theta.npy, the shipped router was chosen by minibatch luck and overfits sampling noise (the same noise the README notes swung random routing 0.792 → 0.733 between runs).

Fix

Return the distribution mean (result.xfavorite) as the solution vector. pycma documents xfavorite as the recommended estimate of the optimum for noisy problems, and it is the maximum-likelihood point of the search distribution. This is the issue author’s first proposed option ("ship the mean"), kept minimal and self-contained.

Because train.py already saves whatever best() returns, no change to train.py is needed — best_theta.npy now holds the mean automatically.

Kept stable on purpose

  • The reported fitness stays the best observed training fitness (self._best_f), so run()’s history["best_fitness"] and the S7 smoke contract remain monotone; only the shipped vector changes.
  • The "no evaluation yet" RuntimeError before the first tell() is preserved.
  • Falls back to the locally tracked incumbent if a backend has no xfavorite.

Tests

Adds tests/test_sep_cmaes.py (CPU-only, torch-free, importorskip("cma")):

  • best() returns xfavorite, not xbest
  • best() raises before the first tell()
  • reported fitness is the running max of observed fitness
  • run()’s logged best_fitness stays non-decreasing (S7 regression guard)

Full root suite: 165 passed. ruff check clean on the changed files.

…ini-router#69)

SepCMAES.best() returned pycma's xbest — the single best-*evaluated*
candidate. TRINITY's fitness J(theta)=E[R(tau)] is a stochastic minibatch
reward with a sampled policy, so xbest is selection-on-noise: the luckiest
evaluation of the run, not the policy with the best expected reward. train.py
saves best()[0] as best_theta.npy, so the shipped router was chosen by
minibatch luck and overfits sampling noise.

Return the distribution mean (result.xfavorite) instead — pycma's recommended
optimum estimate for noisy objectives and the maximum-likelihood point of the
search distribution. The reported fitness stays the best *observed* training
fitness (self._best_f), so run()'s history and the S7 smoke contract remain
monotone; only the shipped vector changes. The no-eval-before-tell RuntimeError
contract is preserved.

Adds tests/test_sep_cmaes.py: best() returns xfavorite (not xbest), raises
before the first tell, reports the running-max fitness, and run()'s logged
best_fitness stays non-decreasing.

Fixes mini-router#69
@github-actions github-actions Bot added miner Miner contribution PR train Training or routing changes labels Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

miner Miner contribution PR train Training or routing changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sep-CMA-ES ships the noisy per-eval xbest, not the distribution mean

1 participant