QUILT: Rethinking Sparse-Attention Prefill through Shared Query Execution
Sparse attention reduces the cost of long-context attention, but existing kernels typically process queries independently, repeatedly loading and dequantizing KV entries shared across queries. We observe substantial overlap in the KV entries selected by neighboring queries, creating opportunities for cross-query reuse....
Zhen-Duo Zhao, Qi-Hui Zhou, Ming-Cong Song et al.
· 0 citations