fix(cuda): support Turing GPUs - #24
Conversation
|
Independent sm_75 validation from a Quadro RTX 4000 (Turing, 8 GB). I've been testing #131 (
The Without the gating,
After applying the backend capability gates from #24 to the #131 branch, the server correctly fell back to Turing-compatible paths and I was able to run prolonged Qwen3.6 benchmarks successfully. Models exercised successfully on sm_75 included:
The most useful observation from my side is that checking whether these optional packages are merely importable is not sufficient on Turing: they can import successfully while containing no runnable sm_75 kernel, producing a late runtime failure instead of selecting the fallback. So the architecture check in this PR fixes a real failure mode I could independently reproduce on another Turing card. For clarity: I specifically transplanted/tested the optional-backend sm80 gating from this PR onto #131. I did not independently validate every other change in #24 (e.g. the kernel-cache packaging changes), so I don't want to overstate the scope of this report. |
Adds NVIDIA Turing (
sm_75) support for RTX 20-series GPUs:sm_80+.sm_75to kernel-cache builds.Verified on RTX 2060 SUPER with the Qwen3.6 NVFP4 model. The API server starts and serves
requests successfully.
The full test suite ran with 1,313 passed, 25 skipped, and 39 GPU-specific failures caused by RTX 2060 SUPER (
sm_75) limitations and unavailable optional backends.