Looped Transformers as Optimizers
Looped Transformers provide a parameter-efficient approach to depth scaling by repeatedly applying shared Transformer blocks. Recent reasoning models have likewise highlighted the value of scaling test-time computation through longer computation trajectories. However, the principles for designing effective loop transit...
Yu-Long Huang, Chen Jiang, Zhan-Peng Zhou et al.
· 0 citations