RABIT-KV
Target-bit-aware KV-cache quantization and physically packed serving for large language model inference. The frozen K3/V2/G32/R4/META8g64 configuration increases observed paged KV capacity on the evaluated H100 system, with higher latency and lower throughput. Canonical logical quality is model-dependent and is evaluat...