Hui-Tao Yang

Learning to Extrapolate in Depth with Looped Transformers

It is shown that under exchangeability and symmetric positive initialization, population training at depth K converges to an update rule governed by a single effective parameter that balances graph propagation against fitting structure unresolved at training depth.