RIFT: KV Cache Compression for Draft Models with Complementary Compressibility Training
This work introduces RIFT, a family of selection-based KV cache compression strategies that balance prefill information and recent generated tokens, and demonstrates how a compressibility-driven drafter training phase, based on sparse attention masks, can improve the drafter's quality under a compressed KV cache.
Dor Tsur, Maor Ashkenazi, Maximilian Golub et al.
· 0 citations