#machine learning
May 2026
Quantifying Concentration Phenomena of Mean-Field Transformers in the Low-Temperature Regime
It is proved that the token distribution rapidly concentrates onto the push-forward of the initial distribution under a projection map induced by the key, query, and value matrices, and remains metastable for moderate times.
A. Alcalde, Leon Bungert, Konstantin Riedl et al.
· arXiv.org · 5 citations