Jul 2026
VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs
VarRate is presented, a training-free KV codec that assigns each token a variable low-rank budget by its query salience, keeping every token at a nonzero rank, and is the strongest matched-memory compressor.
Shahrzad Esmat, Dhawal Shah, Ali Jannesari
· arXiv.org · 0 citations