feat: out-of-tree handler plugin system (maru.handler_plugins)#58
Open
hyunyul-XCENA wants to merge 5 commits into
Open
feat: out-of-tree handler plugin system (maru.handler_plugins)#58hyunyul-XCENA wants to merge 5 commits into
hyunyul-XCENA wants to merge 5 commits into
Conversation
Let vendor-/hardware-specific behaviour live outside maru core instead of in a separate package. MaruHandler discovers plugins registered under the `maru.handler_plugins` entry-point group at construction and calls them at four seams: on_init, on_batch_retrieve, on_close, contribute_stats. Design follows vLLM's soft-fail plugin model (not PyTorch's hard-fail autoload): a Maru plugin is an optional optimization, so a load/hook failure is logged and skipped, never raised. MARU_PLUGINS filters by name. Plugins couple to core only through a small stable accessor surface: - MaruHandler.is_region_mapped(region_id) - MaruHandler.get_region_dax_path(region_id) (via mapper/client get_dax_path) Everything a plugin needs about a batch arrives via the on_batch_retrieve hook args. Empty-plugin hot path stays free (guarded dispatch). Adds maru_handler/plugin.py (MaruHandlerPlugin protocol + soft-fail loader), tests/unit/test_plugin_loader.py, and a README Plugins section.
Mark MaruHandler.is_region_mapped / get_region_dax_path and the four MaruHandlerPlugin hook signatures as the public, stable plugin contract that out-of-tree plugins depend on across independent release cycles. - docs/source/api_reference/plugins.md: entry-point group, hook table with timing, stable accessor surface, soft-fail + version-skew warning, MARU_PLUGINS. - handler.py: strengthen accessor docstrings/comment to flag them as stable API. - test_plugin_loader.py: contract test pinning the accessor signatures, hook names, and entry-point group name so CI fails if the surface drifts.
…allowlist Review-driven robustness for configuration mistakes (soft-fail already covers runtime exceptions): - a factory returning None is skipped (was retained as a dead "plugin") - duplicate entry-point names: first wins + warn (running both would double every hook — matches vLLM's loader) - an allowlist (MARU_PLUGINS) name matching no installed plugin now warns instead of silently loading nothing Adds unit tests for all three (16 passed).
Main's get_stats now returns stats.stats_manager; update the plugin get_stats test's RPC stub so it doesn't AttributeError after the rebase.
hyunyul-XCENA
changed the title
hyunyul-XCENA
force-pushed
the
feat/oot-plugin-hooks
branch
from
ea88784 to
c68c29c
Compare
Public maru docs shouldn't name a specific vendor plugin. Replace the named example with a generic 'writing a plugin' description and genericize the region-mapping note.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an out-of-tree (OOT) handler plugin system so vendor-/hardware-specific behaviour can live in separate packages instead of Maru core. Maru stays vendor-neutral: a plugin package registers a
maru.handler_pluginsentry point andMaruHandlerdiscovers it at construction and calls it at defined seams — core never imports it. Loading is soft-fail (vLLM-style), so a missing or broken plugin never breaks KV-cache operation.Key Changes
maru_handler/plugin.py(new):MaruHandlerPluginprotocol (4 optional hooks) +load_handler_plugins()soft-fail loader — filters byMARU_PLUGINS, dedups duplicate names (first wins), and warns on aNonefactory return / typo'd allowlist name.maru_handler/handler.py: dispatch at four seams (on_init/on_batch_retrieve/on_close/contribute_stats) via_dispatch_plugins; the empty-plugin case is guarded so the retrieval hot path stays free.on_batch_retrievefires after the region-mapping loop (a plugin acts on regions mapped in the current batch); plugin stats are namespaced understats["plugins"][<PluginClassName>].MaruHandler.is_region_mapped()andget_region_dax_path()(backed by newmapper/clientget_dax_path). A contract test fails CI if the accessor signatures, hook names, or entry-point group drift.docs/source/api_reference/plugins.md.kv_offset + kv_lengthbefore device ops; access-dumper flush/logging tuning.Test Plan
tests/unit/test_plugin_loader.py(loader soft-fail / allowlist / dedup / None-return / dispatch isolation / API contract): 16 passed, ruff cleanpytest -v) — note: the full unit suite hits a pre-existing, environment-specific native crash in the mockedconnect()coverage tests on CUDA-enabled hosts (reproduces onmainwithout this branch; unrelated to this change)Related Issues