A Coulomb Particle Model for Learning Kernel Attention in Transformers
A particle-based method that learns this distribution by optimizing kernel-target alignment while regularizing particles with a Riesz/Coulomb repulsive potential is proposed and instantiate the method in linearized Transformer attention.