Monte Carlo Estimation for KV Cache Eviction
This work casts fixed-budget future-aware eviction as distributional estimation over plausible model-conditional query trajectories and introduces LORE-KV (Lookahead Output-perturbation with Reliability-weighted Ensembles for Key-Value caches), a training-free method that samples short autoregressive continuations from...
Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer et al.
· 1 citation