feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype - #103
Closed
lucaspirola wants to merge 2 commits into
Closed
feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype#103lucaspirola wants to merge 2 commits into
lucaspirola wants to merge 2 commits into
