DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models
DCKV, a differentiable codec for the KV cache and the first KV-cache codec that is trained end to end, is presented, a differentiable codec for the KV cache and the first KV-cache codec that is trained end to end.
Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev et al.
· 0 citations