Environment
ds4 commit: efdadd4 (and also tested after fetch showing newer 0a7ad77, please confirm if fix is expected there)
Platform: Linux
GPU: NVIDIA GB10 (sm_121)
2 nodes (DGX1 + DGX2), testing distributed pipeline
Backend: CUDA
What I’m trying to do Split model across two machines:
coordinator on DGX1 for early layers
worker on DGX2 for later layers
Example intent:
coordinator: 0:20
worker: 21:output
Observed behavior
Coordinator with early-layer span crashes at startup:
bash
/home/administrator/ds4/bin/ds4-server
--role coordinator
--layers 0:20
--listen 0.0.0.0 9000
--host 0.0.0.0
--port 8000
--ctx 8192
Logs end with:
model map restricted to 0:20
memory planning completes
Segmentation fault (core dumped)
If trying to include output with non-contiguous syntax, parser rejects it:
bash
--layers 0:20,output
Error:
Text
invalid --layers 0:20,output: invalid end layer in 0:20,output
Worker rejects coordinator-only prefill flags (expected, just noting):
Text
--dist-prefill-chunk requires --role coordinator
When testing API during broken routing, sometimes get:
JSON
{"error":{"message":"coordinator route does not start at layer 0","type":"invalid_request_error"}}
Why this seems problematic If coordinator runtime effectively requires owning output/final stage in this configuration, but --layers only accepts contiguous spans and cannot express 0:K + output, then a valid split topology may be impossible to express via CLI.
Expected behavior At least one of:
Coordinator should work with contiguous 0:K while worker handles K+1:output (no segfault), or
CLI should support disjoint layer assignment (e.g. 0:20,output) if coordinator needs output stage, or
A clear documented constraint and recommended valid topology for multi-node split on CUDA/GB10.
Request
Please confirm whether this is a bug or intended limitation.
If bug: any patch/workaround?
If limitation: can disjoint --layers support be added, or can coordinator ownership rules be relaxed?
Additional notes
Standalone single-node tensor-parallel mode works on both nodes.
Problem appears specific to distributed coordinator path.
Environment
ds4 commit: efdadd4 (and also tested after fetch showing newer 0a7ad77, please confirm if fix is expected there)
Platform: Linux
GPU: NVIDIA GB10 (sm_121)
2 nodes (DGX1 + DGX2), testing distributed pipeline
Backend: CUDA
What I’m trying to do Split model across two machines:
coordinator on DGX1 for early layers
worker on DGX2 for later layers
Example intent:
coordinator: 0:20
worker: 21:output
Observed behavior
Coordinator with early-layer span crashes at startup:
bash
/home/administrator/ds4/bin/ds4-server
--role coordinator
--layers 0:20
--listen 0.0.0.0 9000
--host 0.0.0.0
--port 8000
--ctx 8192
Logs end with:
model map restricted to 0:20
memory planning completes
Segmentation fault (core dumped)
If trying to include output with non-contiguous syntax, parser rejects it:
bash
--layers 0:20,output
Error:
Text
invalid --layers 0:20,output: invalid end layer in 0:20,output
Worker rejects coordinator-only prefill flags (expected, just noting):
Text
--dist-prefill-chunk requires --role coordinator
When testing API during broken routing, sometimes get:
JSON
{"error":{"message":"coordinator route does not start at layer 0","type":"invalid_request_error"}}
Why this seems problematic If coordinator runtime effectively requires owning output/final stage in this configuration, but --layers only accepts contiguous spans and cannot express 0:K + output, then a valid split topology may be impossible to express via CLI.
Expected behavior At least one of:
Coordinator should work with contiguous 0:K while worker handles K+1:output (no segfault), or
CLI should support disjoint layer assignment (e.g. 0:20,output) if coordinator needs output stage, or
A clear documented constraint and recommended valid topology for multi-node split on CUDA/GB10.
Request
Please confirm whether this is a bug or intended limitation.
If bug: any patch/workaround?
If limitation: can disjoint --layers support be added, or can coordinator ownership rules be relaxed?
Additional notes
Standalone single-node tensor-parallel mode works on both nodes.
Problem appears specific to distributed coordinator path.