fix(cuda): avoid null output-head dereference in distributed coordinators

[hy-sde]

Summary

Fix a startup segmentation fault when CUDA distributed inference assigns the output head to a worker, for example a two-DGX-Spark pipeline with coordinator --layers 0:19 and worker --layers 20:output.

metal_graph_alloc_raw_cap() sized the logits buffer through weights->output->dim[1]. A sliced coordinator intentionally does not map the worker-owned output tensor, so weights->output is NULL during ds4_session_create().

Use the model shape metadata (DS4_N_VOCAB) instead. The allocator no longer needs ds4_weights, so this also removes that unused parameter and updates its call sites.

Reproduction

Hardware:

• Two NVIDIA DGX Sparks, one GB10 GPU per node
• Direct ConnectX-7/DAC link, pipeline transport over TCP
• DeepSeek V4 Flash Q2 imatrix GGUF

Start the worker first:

./ds4 --cuda \
  -m "$MODEL" \
  --role worker \
  --layers 20:output \
  --coordinator 192.168.100.10 9911 \
  --ctx 8192 \
  --debug

Then start the coordinator:

./ds4 --cuda \
  -m "$MODEL" \
  --role coordinator \
  --layers 0:19 \
  --listen 192.168.100.10 9911 \
  --ctx 8192 \
  --tokens 64 \
  --temp 0 \
  --nothink \
  --debug \
  -p "In two sentences, explain why the sky appears blue."

Before this change, the coordinator completed model preparation and context planning, then exited with SIGSEGV before opening the listener.

GDB showed:

#0 metal_graph_alloc_raw_cap  ds4.c:16826
#1 ds4_session_create         ds4.c:56578
#2 run_sampled_generation     ds4_cli.c:523
#3 main                       ds4_cli.c:2129

The faulting expression was:

const uint64_t vocab_dim = weights->output->dim[1];

Verification

After the fix, the same two-node command completes route registration and generation:

distributed coordinator API: listening on 192.168.100.10:9911
distributed coordinator: registered worker ... layers=20:output
distributed coordinator: complete route ready: local 0:19 -> ... 20:output
distributed route ready
...
prefill: 17.14 t/s, generation: 14.68 t/s

The generated response was coherent and all 64 requested decode steps completed.

Additional checks on the DGX Spark coordinator:

make ds4
make cuda-regression
# cuda long-context regression: OK
make cpu

The regression requires a CUDA accelerator, a sliced distributed route, and the model artifacts; no source-text test was added.

[jessa0]

i think i hit this bug as well on rocm on efdadd4:

$ gdb --args /usr/bin/ds4-server --listen <redacted> 8001 --model /var/cache/models/ds4/model.gguf --role coordinator --layers 0:10
GNU gdb (Gentoo 17.2 vanilla) 17.2

...

(gdb) r
Starting program: /usr/bin/ds4-server --listen ipv6.sho102.bolt.internal 8001 --model /var/cache/models/ds4/model.gguf --role coordinator --layers 0:10
ds4: Linux rocm backend set oom_score_adj=1000
[New LWP 470760]
[New LWP 470761]
[New LWP 470762]
[LWP 470762 exited]
ds4: ROCm backend initialized on AMD Radeon 8060S Graphics (sm_115)
ds4: restricting rocm model map to layers 0:10 (23 spans, 39.81 GiB tensor span)
ds4: ROCm preparing model tensor mappings: 38.12 GiB
[New LWP 470805]
[LWP 470805 exited]
ds4: ROCm startup model preparation covered 39.81 GiB of tensor spans in 23.790s
ds4: rocm backend initialized for graph diagnostics
ds4: memory: KV 0.78 GiB (raw 0.36 + compressed 0.42) + buffers 0.25 GiB + resident model 39.81 GiB = 40.84 GiB planned
ds4: memory detail: ctx=32768 prefill_cap=4096 raw_kv_rows=4352 compressed_kv_rows=8194 backend=rocm
0722 13:14:24 ds4-server: context buffers 1053.75 MiB (ctx=32768, backend=rocm, prefill_chunk=4096, raw_kv_rows=4352, compressed_kv_rows=8194)
Thread 1 "ds4-server" received signal SIGSEGV, Segmentation fault.
0x00005555555e91ec in metal_graph_alloc_raw_cap (g=g@entry=0x555556b66bd8, weights=weights@entry=0x555555ebc578, layer=0x555555ebc5a8,
    raw_cap=<optimized out>, ctx_size=ctx_size@entry=32768, prefill_cap=4096, enable_mtp=false, placement=0x0, cuda_tensor_parallel=false,
    shared_prefill_workspace=0x0) at ds4.c:16826
(gdb) bt
#0  0x00005555555e91ec in metal_graph_alloc_raw_cap (g=g@entry=0x555556b66bd8, weights=weights@entry=0x555555ebc578, layer=0x555555ebc5a8,
    raw_cap=<optimized out>, ctx_size=ctx_size@entry=32768, prefill_cap=4096, enable_mtp=false, placement=0x0, cuda_tensor_parallel=false,
    shared_prefill_workspace=0x0) at ds4.c:16826
#1  0x000055555564822a in ds4_session_create (out=0x5555560b48b0, e=<optimized out>, ctx_size=32768) at ds4.c:56578
#2  0x000055555555c64a in main (argc=<optimized out>, argv=<optimized out>) at ds4_server.c:12984

Edit: confirmed this patch prevents the crash.