RACE: RoPE-Aligned Attention Compression Based on Query-Key Activation Energies
Head-dimension compression can reduce the memory required by the key-value (KV) cache, but existing methods often reconstruct full-dimensional representations during inference, adding significant decoding FLOPs overhead. We formalize the design considerations for converting pretrained attention into a natively reduced-...
Irene Tenison, Abdulrahman Alabdulkareem, D. Weidele et al.
· 0 citations