DropKV: Decoupling Residual-Output Perturbation for Near-Optimal KV-Cache Eviction
Key-value (KV) cache management has become a critical bottleneck for scaling Large Language Models (LLMs) to long-context scenarios due to linear memory growth. While recent eviction methods have moved beyond simple attention-score heuristics to incorporate value representations, they typically face a fundamental trade...
Ao-Zhong Zhang, Selcuk Gurses, Yan-Xia Deng et al.
· 1 citation