Skip to content

feat(memory): add safe-point SessionBank budget governor - #357

Open
PhilipJohnBasile wants to merge 2 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/sessionbank-memory-governor
Open

feat(memory): add safe-point SessionBank budget governor#357
PhilipJohnBasile wants to merge 2 commits into
youssofal:mainfrom
PhilipJohnBasile:feat/sessionbank-memory-governor

Conversation

@PhilipJohnBasile

Copy link
Copy Markdown
Contributor

This extracts the SessionBank memory governor from #336 as one reviewable system.

It adds an opt-in controller behind MTPLX_MEMORY_GOVERNOR=1. The controller samples process memory, applies hysteresis to SessionBank budget changes, and mutates the bank only after acquiring the model lock nonblocking and confirming that foreground, scheduler, restore, commit, MTP, and postcommit work are idle. Sampling happens before the lock is acquired.

Measurement

Command:

python3 scripts/bench_memory_governor.py --observe-iterations 200000 --apply-iterations 50000 --batches 40 --output docs/benchmarks/sessionbank-memory-governor-no-model-20260826.json

Apple Silicon arm64, Python 3.14.7:

  • observe: 1,594 ns/op mean, 1,679 ns/op p95, 627,176 ops/s
  • safe apply: 1,032 ns/op mean, 1,169 ns/op p95, 968,946 ops/s
  • unsafe rejection: 651 ns/op mean, 729 ns/op p95, 1,536,454 ops/s
  • correctness: 50,000 safe applications and 50,000 rebalances; zero unsafe applications and zero unsafe rebalances

This is a synthetic controller-overhead measurement. It does not load a model, import MLX, or simulate a live macOS memory-pressure event. The tracked JSON receipt includes the exact command, host, configuration, and SHA-256 hashes of the measured inputs.

Validation

/Users/pjb/git/MTPLX/.venv/bin/python -m pytest -q tests/test_memory_governor.py tests/test_memory_pressure_guard.py tests/test_session_bank.py tests/test_session_bank_env_caps.py tests/test_server_openai.py tests/test_no_mlx_imports.py
# 435 passed

python3 -m compileall -q mtplx/memory_governor.py mtplx/server/openai.py tests/test_memory_governor.py scripts/bench_memory_governor.py
# passed

/Users/pjb/git/MTPLX/.venv/bin/python -m ruff check mtplx/memory_governor.py tests/test_memory_governor.py scripts/bench_memory_governor.py
# passed

/Users/pjb/git/MTPLX/.venv/bin/python -m ruff check --ignore F841 mtplx/memory_governor.py mtplx/server/openai.py tests/test_memory_governor.py scripts/bench_memory_governor.py
# passed

git diff --check
# passed

A full Ruff run without the exclusion still reports the existing last_history_role F841 in mtplx/server/openai.py; the same failure reproduces on untouched upstream/main.

Excluded from this PR

  • native adaptive tuning and policy hooks
  • prefill, concurrency, and speculative decoding controls
  • unified-memory and expert-residency systems
  • OTLP export
  • replay, semantic-anchor, and expert-locality systems
  • systems API, dashboard, README, changelog, and workflow changes

@PhilipJohnBasile
PhilipJohnBasile force-pushed the feat/sessionbank-memory-governor branch from bc6f9bc to 4adb2fa Compare September 1, 2026 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant