Preprint
Nov 2026
Attend at the Right Resolution: Head-Adaptive Sparse Attention via Query-Key Interaction Structure
The proposed Head-Adaptive Sparse Attention (HASA), a training-free sparse attention framework that jointly adapts selection resolution along these two dimensions, captures 88.1% of dense-attention mass and, compared with UNIQUE, improves accuracy and speedup over dense attention.
W. Raza, Ahsan Bilal, Yi-Dan Shen et al.
· 0 citations