Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers
It is shown that length generalization need not rely on faithful step-by-step reasoning, as Looped Transformers can exploit task structure without explicitly representing every intermediate state.
Jian Liang, Xi Jin, Liang-Ming Pan
· 0 citations