SGD-KV: Summarization Guided KV Cache Compression
SGD-KV (Summarization-Guided KV Cache Compression), a head-aware framework that leverages a novel chunk-summarization diagnostic task to systematically identify and prioritize attention heads specialized in hierarchical information aggregation, is presented.
Ze-Yu Liu, Woo-Min Song, Xuandi Fu et al.
· 0 citations