#artificial intelligence
Sep 2026
MoSAR: Mixture of Semantic Attention Regimes for Learning Adaptive and Approximable Attention Geometries
Mixture of Semantic Attention Regimes (MoSAR) is introduced, which learns such an adaptive, controlled-decay geometry over query--key interactions, suggesting that it is not only adaptive, but also amenable to low-cost approximation at inference time.
Michele Paolicelli, Alessandro Petruzzelli, Alessandro Francesco Maria Martina et al.
· arXiv.org · 0 citations