Skip to content

Author

Yury Polyanskiy

2 papers indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

Scaling Limits of Long-Context Transformers

It is shown that the critical scale at which selectivity emerges is determined by the local exponent of the distance-to-query distribution near zero rather than by global features of the context, and scales like $\beta_n^\ast \asymp n^{2/(d-1)}$ for uniform keys on $\mathbb{S}^{d-1}$.

Giuseppe Bruno, Shi Chen, Zheng-Jiang Lin et al. · 3 citations

Continuous First, Discrete Later: VQ-VAEs Without Dimensional Collapse

Theoretical analysis predicts downstream performance as a function of warm-up length, enabling an adaptive criterion for switching from AE Warm-up to VQ-VAE training and empirically characterize how warm-up duration governs the achievable final loss.

Xinyu Zhao, N. Karagodin, Hamed Hassani et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.