Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers
Looped Transformers offer a parameter-efficient route to test-time scaling by reusing shared layers for iterative latent reasoning. However, additional iterations can reduce support for a reference answer, leaving unclear whether an update's direction is locally unhelpful or its full displacement moves too far. We stud...
Zhi-Hao Guo, Zong-Han Wu, Hai-Zhou Du et al.
· 1 citation