Profiling CUDA Graph vs torch.compile vs eager decode on RTX 2070 — measuring per-token latency reduction across batch sizes, sequence lengths, and GPT-2 model sizes.
python performance-engineering benchmarking gpu cuda pytorch transformer triton profiling gpt2 llm-inference cuda-graph torch-compile decode-latency staticcache
-
Updated
Jul 17, 2026 - Python