Transformers have exhibited impressive empirical success across various domains, but their theoretical foundations remain less developed. This work constitutes a mathematical study of the measure-to-measure operators defined by transformers. We show that transformers map sub-Gaussian inputs to sub-Gaussian outputs; thi...
Any continuous measure-to-measure operator with outputs of finite $p-th moment can be uniformly approximated, in the $p-Wasserstein distance, by continuous measure-dependent pushforwards, using measure-theoretic transformers.
This work studies whether one masked transformer can approximate causal token-to-token maps uniformly over sequences of arbitrary length sampling a fixed normalized horizon, and characterizes the causal families admitting uniform approximation on these compact input classes by a single transformer with length-independe...
Takashi Furuya, Maarten V. de Hoop, Gabriel Peyré· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.