Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Random Attention keeps the prompt and evicts uniformly at random within each attention head, computing no score at all; across four models and six reasoning tasks, it matches the strongest baseline in task performance while delivering 32-43% higher throughput than that method when deployed with vLLM.
Heng-Yi Wang, Jie-Lin Qiu, Wenting Zhao et al.
· 0 citations