Skip to content

Distributed pipeline crash / layer routing limitation: coordinator crashes without output layer, but --layers cannot express non-contiguous spans #610

Description

@michele-cei

Environment

ds4 commit: efdadd4 (and also tested after fetch showing newer 0a7ad77, please confirm if fix is expected there)
Platform: Linux
GPU: NVIDIA GB10 (sm_121)
2 nodes (DGX1 + DGX2), testing distributed pipeline
Backend: CUDA
What I’m trying to do Split model across two machines:

coordinator on DGX1 for early layers
worker on DGX2 for later layers
Example intent:

coordinator: 0:20
worker: 21:output
Observed behavior

Coordinator with early-layer span crashes at startup:
bash
/home/administrator/ds4/bin/ds4-server
--role coordinator
--layers 0:20
--listen 0.0.0.0 9000
--host 0.0.0.0
--port 8000
--ctx 8192
Logs end with:

model map restricted to 0:20
memory planning completes
Segmentation fault (core dumped)
If trying to include output with non-contiguous syntax, parser rejects it:
bash
--layers 0:20,output
Error:

Text
invalid --layers 0:20,output: invalid end layer in 0:20,output
Worker rejects coordinator-only prefill flags (expected, just noting):
Text
--dist-prefill-chunk requires --role coordinator
When testing API during broken routing, sometimes get:
JSON
{"error":{"message":"coordinator route does not start at layer 0","type":"invalid_request_error"}}
Why this seems problematic If coordinator runtime effectively requires owning output/final stage in this configuration, but --layers only accepts contiguous spans and cannot express 0:K + output, then a valid split topology may be impossible to express via CLI.

Expected behavior At least one of:

Coordinator should work with contiguous 0:K while worker handles K+1:output (no segfault), or
CLI should support disjoint layer assignment (e.g. 0:20,output) if coordinator needs output stage, or
A clear documented constraint and recommended valid topology for multi-node split on CUDA/GB10.
Request

Please confirm whether this is a bug or intended limitation.
If bug: any patch/workaround?
If limitation: can disjoint --layers support be added, or can coordinator ownership rules be relaxed?
Additional notes

Standalone single-node tensor-parallel mode works on both nodes.
Problem appears specific to distributed coordinator path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions