Skip to content

Fix VLLMServeLM ignoring a hyphenated tensor-parallel-size in model_kwargs#289

Merged
nayopu merged 1 commit into
mainfrom
fix/vllm-tp-key-normalization
Jun 19, 2026
Merged

Fix VLLMServeLM ignoring a hyphenated tensor-parallel-size in model_kwargs#289
nayopu merged 1 commit into
mainfrom
fix/vllm-tp-key-normalization

Conversation

@nayopu

@nayopu nayopu commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

関連する Issue / PR

なし

目的

VLLMServeLMmodel_kwargs={"tensor-parallel-size": 1} のようにハイフン区切りで tensor_parallel_size を渡すと、指定値が黙って torch.cuda.device_count() に上書きされる不具合を修正する。

tensor_parallel_sizemodel_kwargs に含まれない場合、 model_kwargs["tensor_parallel_size"] = device_count() が自動注入されるが、ユーザーが -を使う tensor-parallel-size (例えば1)を渡す場合、vllm serve には --tensor-parallel-size 1 --tensor-parallel-size <device_count()> が二重に渡り、argparse の後勝ちで ユーザー指定のtensor-parallel-sizeがサイレントに無視される。

実装の詳細

VLLMServeLM.__init__ の自動注入ガードを、tensor_parallel_sizetensor-parallel-size のどちらの表記も「指定済み」とみなすように修正する。

  • どちらの表記も含まれていなければ、従来どおり tensor_parallel_sizetorch.cuda.device_count() を注入する
  • どちらか一方でも指定されていれば、その値を尊重して自動注入しない(二重フラグを防ぐ)
model_kwargs = model_kwargs or {}
if "tensor_parallel_size" not in model_kwargs and "tensor-parallel-size" not in model_kwargs:
    model_kwargs["tensor_parallel_size"] = torch.cuda.device_count()

動作確認

  • テストに通ることを確認した
  • ruff check / ruff format --check 通過
  • 既存挙動への影響なし(underscore 形キーは従来どおり)

追記事項

なし

VLLMServeLM auto-injects tensor_parallel_size = torch.cuda.device_count()
when the key is absent, but it checked only the underscore spelling. Since
the command builder maps "_" -> "-", "tensor_parallel_size" and
"tensor-parallel-size" become the same --tensor-parallel-size flag, so a
caller passing the hyphenated key slipped past the guard: the default was
injected anyway, emitting a duplicate --tensor-parallel-size flag with
device_count() silently winning (and breaking models without TP support on
multi-GPU nodes).

Skip the default injection when either spelling is present.

Co-authored-by: Claude (Managed) <noreply@anthropic.com>
@nayopu nayopu merged commit c56031e into main Jun 19, 2026
18 of 22 checks passed
@nayopu nayopu deleted the fix/vllm-tp-key-normalization branch June 19, 2026 05:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants