Comparative Characterization of KV Cache Management Strategies for LLM Inference
This work conducts an empirical study of three state-of-the-art KV cache management frameworks: vLLM, InfiniGen, and H2O, revealing the most suitable selection and configuration of KV cache strategies under memory and performance constraints.