Skip to content

refactor(common): centralize cross-feature and architecture gating#558

Open
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:feat/feature-gate
Open

refactor(common): centralize cross-feature and architecture gating#558
Graffioh wants to merge 1 commit into
Luce-Org:mainfrom
Graffioh:feat/feature-gate

Conversation

@Graffioh

@Graffioh Graffioh commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Summary

Cross-feature rules were duplicated between server_main and backend_factory, each layer seeing a different subset of the launch configuration — no authoritative admission path, and the GGUF architecture read three or four times.

The factory now owns the decision. prepare_backend() resolves model metadata and target placement once into an opaque ResolvedBackendPlan, applies all compatibility policy, and returns a categorized result; server_main only maps that to the existing exit convention. create_backend(args) stays source-compatible.

Architecture capability table

Arch-dependent facts were previously implicit — an option is unsupported because the factory's dispatch simply doesn't forward it, an absence you can't test or document. model_capabilities.h makes them statements:

enum class FeatureSupport { Never, Monolithic, Both };

inline constexpr ArchCapabilities kArchCapabilities[] = {
//   arch          split  rdraft pflash offload  draft  ddtree vwidth fa_win dswa
    {"qwen35",     true,  true,  true,  false,   kBoth, kBoth, kNever, kBoth, kBoth},
    {"qwen35moe",  false, false, false, true,    kMono, kMono, kNever, kMono, kMono},
    {"laguna",     true,  false, false, true,    kMono, kMono, kMono,  kNever, kNever},
    {"qwen3",      false, false, true,  false,   kNever, kNever, kNever, kNever, kNever},
    {"gemma4",     true,  false, false, false,   kMono, kNever, kNever, kBoth, kNever},
    {"deepseek4",  true,  false, false, false,   kNever, kNever, kNever, kNever, kNever},
};

A row says what a model can do; a column says which architectures honor a flag — the dispatch chain can only be read one way. Row order matches that chain so the two diff by eye. Placement-sensitive entries are FeatureSupport because laguna and gemma4 forward a draft model only when monolithic.

It stays in the binary on purpose: it can only ever describe the dispatch chain, since a new architecture needs a backend before it needs a row. External config would add an artifact to keep in sync and a parse that can fail at startup, and buy nothing.

Compile-time checking

static_assert in model_capabilities.h checks the table is internally coherent (rows named and unique; no Both without a layer-split adapter; remote draft implies the arch accepts a draft model). backend_factory.cpp then checks it against the structs it actually feeds — each option arrives through a named config field, so detecting that field verifies the table against the code:

DFLASH_CHECK_ARCH("laguna", LagunaBackendArgs, LagunaLayerSplitAdapterConfig);

A row claiming support an architecture has nowhere to store fails the build. All four assert classes were mutation-tested to confirm they actually fire.

Not checkable: whether a dispatch branch assigns a field it has — a statement about statements, which C++ can't reflect over, and exactly the gemma4 bug below. That gap is covered by the tests and the row ordering.

Errors vs warnings

Errors say why a configuration cannot run; warnings say what silently will not happen. The test is whether the launch still yields a working server:

[server] warning: --draft ignored: architecture 'qwen3' has no speculative decode support
[server] warning: --draft ignored: architecture 'laguna' provides speculative decode only on single-device placement
[server] warning: --freq/--collect-routing ignored: architecture 'gemma4' has no expert routing to record

Degradations that rewrite config on the way through (--spark, --draft-residency) stay with the setting they mutate.

Behavior and compatibility

  • One new rejection: --target-devices on qwen35moe or qwen3. Neither has a layer-split adapter, so the factory handed the multi-device placement to a monolithic backend reading only the primary GPU — the launch appeared to work, then failed as an OOM far from its cause. This is the only way this PR turns a working launch into an exit 2.
  • Everything else previously dropped in silence is now a warning, so nothing that works today stops working.
  • Bug fixed: --fa-window was dropped on single-device gemma4. Gemma4BackendConfig carries the field and Gemma4Backend reads it, but the factory populated it only on the layer-split path — the same flag changed attention behavior or not depending on placement, with no diagnostic. Note this changes output for that configuration.
  • supports_remote_draft() is wired back in as a post-init cross-check, so a table that drifts from the backends fails loudly.
  • Invalid feature combinations still exit 2, backend/model failures exit 1. An undetectable or unbuildable architecture is now rejected on every launch, naming the architecture rather than reporting a generic backend-creation failure.

Validation

  • New test_feature_gate target: the gate and table are pure functions, so it compiles feature_gate.cpp and placement_config.cpp alone — no dflash_common, ggml or GPU toolkit. Builds and runs in under a second, which is why these tests left test_server_unit.
  • Full suite green on CUDA (sm86): server_unit 2059 + feature_gate 112 = 2171 assertions, matching the pre-split total exactly; deepseek4_unit also passing.
  • A 3M-configuration differential against the previous scattered rules, run for both compiled backends, reports the layer-split rule as the only admission change, and nothing that was rejected before and is accepted now.
  • -Wall -Wextra clean on every modified TU; git diff --check passes.

Follow-up (not in this PR)

Replacing the arch string with an enum class Arch parsed once in prepare_backend() would make the dispatch a switch, so -Werror=switch fails the build until a new architecture is handled at every site, and arch typos become compile errors rather than silently-false predicates — qwen36 would have been a build failure instead of an unreachable branch in model_card.cpp. Out of scope here because it touches every arch == "..." site (chat_template, model_card, moe_expert_compute_ipc, Spark, test_dflash): mechanical, but wide enough to deserve its own review.

Review in cubic

@Graffioh
Graffioh marked this pull request as ready for review July 25, 2026 08:05

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread server/src/server/server_main.cpp Outdated
Comment thread server/src/common/backend_args.h
@Graffioh
Graffioh force-pushed the feat/feature-gate branch from cd8c5f5 to 25c374d Compare July 25, 2026 08:29
@Graffioh Graffioh changed the title refactor(common): centralize cross-feature compatibility in backend factory refactor(common): centralize cross-feature compatibility Jul 25, 2026
@Graffioh
Graffioh force-pushed the feat/feature-gate branch from 25c374d to 558405a Compare July 25, 2026 11:45
@Graffioh Graffioh changed the title refactor(common): centralize cross-feature compatibility refactor(common): centralize cross-feature and architecture gating Jul 25, 2026
@Graffioh
Graffioh force-pushed the feat/feature-gate branch 2 times, most recently from 3e4c0b3 to e44e94f Compare July 25, 2026 12:06
Cross-feature rules were split between server_main and backend_factory, with each layer seeing only part of the launch configuration. Centralize those rules in the pure check_feature_compatibility() helper and make the backend factory the single admission owner.

prepare_backend() now resolves GGUF model metadata and the compiled target placement once into an opaque ResolvedBackendPlan, applies compatibility policy, and returns a categorized result. server_main forwards BackendArgs to that factory preflight and only maps its result to the existing exit-code convention. create_backend(args) remains source-compatible, while the prepared overload verifies the model path and target placement and rechecks compatibility at the dispatch boundary.

Keep raw requested configuration in backend_args.h, architecture capabilities in model_capabilities.h, and the pure gate independent of the factory header. Reuse the same GGUF metadata read for backend dispatch, PFlash/Spark decisions, and model-card resolution.

Architecture capabilities become one constexpr table whose rows mirror create_backend()'s dispatch chain, so a row states what a model can do and a column states which architectures honor a flag. Placement-sensitive entries carry a FeatureSupport value because laguna and gemma4 forward a draft model only when monolithic. The table stays in the binary deliberately: it can only ever describe the dispatch chain, since a new architecture needs a backend before it needs a row.

Two layers of static_assert keep the table honest. model_capabilities.h checks it is internally coherent: rows are named and unique, an architecture without a layer-split adapter cannot claim support on 'Both' placements, and remote draft execution requires an architecture that accepts a draft model. backend_factory.cpp then checks it against the structs it actually feeds, detecting whether each backend config carries the field an option arrives through, so a row claiming support an architecture has nowhere to store fails the build. Neither can see whether a dispatch branch assigns a field it has; that remains the job of the tests and of the table's dispatch-matching row order.

Admission gains one rule from that table. qwen35moe and qwen3 have no layer-split adapter, so --target-devices previously handed a multi-device placement to a monolithic backend that reads only the primary GPU and failed later as an out-of-memory; it is now rejected up front. Options that parse cleanly but never reach the backend are reported through collect_feature_warnings() instead, which never blocks a launch. Errors still describe why a configuration cannot run; warnings describe what silently will not happen.

Fix --fa-window being dropped on single-device gemma4: Gemma4BackendConfig carries the field and Gemma4Backend reads it, but the factory only populated it on the layer-split path. Restore the post-init supports_remote_draft() cross-check so a stale capability table fails loudly rather than routing draft work to a backend that cannot serve it.

Move the gate tests to their own test_feature_gate target. The gate and the capability table are pure functions over resolved facts, so the target compiles feature_gate.cpp and placement_config.cpp alone, without dflash_common, ggml or a GPU toolkit; a gate rule is testable in under a second instead of behind a full backend build. Build the check dependency list per target so that target still runs when the server unit tests are skipped for a missing CURL.

Behavior is otherwise preserved. A 3M-configuration differential against the previous scattered rules, run for both compiled backends, reports no other admission change; an undetectable or unbuildable architecture is now rejected consistently on every launch rather than only when remote draft is enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Claude-Session: https://claude.ai/code/session_01GeWuszoJoUnaUYmTxXE62Z
@Graffioh
Graffioh force-pushed the feat/feature-gate branch from e44e94f to e42adbf Compare July 25, 2026 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant