Jul 2026
KVpop - Key-Value Cache Compression with Predictive Online Pruning
KVpop is introduced, which learns a fixed-budget KV eviction policy by directly supervising the keep-or-drop decision, and introduces a delayed memory-based scorer that, uniquely among learned eviction methods, defers scoring for a fixed number of steps to exploit near-future context.
Lukas Hauzenberger, N. Schmidinger, A. Hartl et al.
· arXiv.org · 1 citation