PINNs-Torch, Physics-informed Neural Networks (PINNs) implemented in PyTorch.

Foundry materializes CUDA graphs along with its execution context to disk to support fast cold start of serving engines.

A comprehensive guide to using CUDA Graphs effectively with PyTorch, covering CUDA fundamentals, PyTorch integration, Megatron-LM implementations, and practical troubleshooting.

A light, transparent, and modular inference & quantization engine for studying LLMs.

This repository contains the source code for Grape.

PyTorch CUDA Graph Capture Examples

Handwritten Qwen2.5-0.5B inference engine — no vLLM, no model.generate(); CUDA Graph + hand-written Triton flash kernels, 3.34x vLLM decode TPS on a single RTX 2080 Ti (96.15/100). Independent work at ECNU summer school.

A minimalist, production-ready framework for training small language models through 4 sequential stages (Pretraining, SFT, Alignment, Reasoning). Optimized for extreme performance with a hardware-aware inference engine featuring CUDA Graphs, Static KV Caching, and Topic-Aware Conversational Memory.

Enhancing CUDA Intra-Streaming-Multiprocessor Parallelism for Large Language Models via Fine-Grained Task Graph

Profiling CUDA Graph vs torch.compile vs eager decode on RTX 2070 — measuring per-token latency reduction across batch sizes, sequence lengths, and GPT-2 model sizes.