g2n — run models that don't fit, on the GPU you already have

A PyTorch compiler (custom FX fusion passes, a Triton LayerNorm kernel, a persistent compile cache) plus a license-gated runtime that serves models and runs local LLMs larger than your VRAM. Also ships g2n.quantum, a statevector quantum circuit simulator (classical simulation — not quantum hardware).

Measured on a single RTX 4050 Laptop (6 GB) with benchmarks/bench.py in this repo — run it on your own card and post what you get:

• EuroLLM-9B at int4 weights, generating at 2.89 tok/s on a 6 GB card. 33 of its 42 layers stay resident; the rest stream from pinned host RAM with the transfers overlapped against compute. The planner sizes that split automatically and lands within 0.3% of the best split found by sweeping every option by hand.
• 3.5× faster cold compile than stock torch.compile on a post-reboot run.

pip install g2n torch

import torch, g2n

compiled = g2n.compile(model)     # == torch.compile(model, backend="g2n")
y = compiled(x)

Free (Community) gives you the g2n fusion passes on stock Inductor and quantum circuits up to 24 qubits. A license key unlocks more — activation is one command and then fully offline:

export G2N_LICENSE_KEY=G2N-XXXX-XXXX-XXXX
g2n activate && g2n status

Unlock	Tier
Persistent compile cache (warmup once per machine, not per run)	Pro
Enhanced planner: epilogue fusion + custom Triton kernels	Pro
Serving platform (pip install g2n-enterprise): registry, HTTP node, quantization incl. weight-only int8, CUDA graphs	Pro
Quantum: unlimited qubits + circuit fusion	Pro
max-autotune, dynamic batching, batched quantum sweeps, model zoo	Enterprise

Quantum in 20 seconds

import g2n.quantum as qf
c = qf.Circuit(2).h(0).cnot(0, 1)      # Bell state
c.measure_all(shots=1000)              # {'00': ~500, '11': ~500}
c.expectation("ZZ")                    # tensor(1.)

Guarantees

• Never worse than eager: any compile failure returns your unmodified model with a one-line warning.
• Offline after activation: license tokens are Ed25519-verified locally; no phone-home during runs.
• Honest numbers: no benchmark claim without named hardware and a script to reproduce it — including on your own machine.

Docs, pricing, benchmarks: g2n.dev · full manual at g2n.dev/docs · questions: support@g2n.dev.

What's open and what isn't

This repository is the free core: the compiler, the fusion passes and the quantum simulator, Apache-2.0, no obfuscation. The LLM offload and serving numbers quoted above come from g2n-enterprise, which is proprietary and paid. Nothing in either package phones home during a run.