GQLSA: Grouped-Query Latent Sparse Attention: A Hardware-Native Architecture for Efficient and Effective Sequence Modeling
We introduce Grouped-Query Latent Sparse Attention (GQLSA), a novel attention mechanism that integrates latent compression, grouped-query sharing, and block-sparse attention into a unified hardware-native architecture. GQLSA achieves linear O(T) complexity in sequence length, a 3.8x speedup over multi-head attention at...