Book
Open access
Nov 2026
LSH-KV: Efficient KV Cache Retrieval for Long-Context LLM Inference via Locality-Sensitive Hashing
As large language models increasingly support long-context inference, the computation and memory access overhead of attention and KV caches have become critical bottlenecks for inference latency and system scalability. Existing approaches mitigate these costs through KV cache dropping, approximate attention, or system-...
Hao-Huan Feng, Shi-Yu Yang, Yuxuan Lu et al.
· Proceedings of the 35th ACM... · 0 citations