Preprint
Oct 2026
Rethinking KV Cache Eviction for Agentic LLM Serving
Key-value (KV) caches impose substantial memory pressure on large language model serving systems, requiring cache eviction when capacity is insufficient. Existing mechanisms typically evict KV from the ends of cached sequences to preserve prefix sharing across requests. However, agent workloads repeatedly reuse long hi...
Jia-Kun Fan, Han Meng, Yu-Xin Xia et al.
· 0 citations