Preprint
Jul 2026
Frontier Language Models Struggle to Copy: Text Can Be Better Viewed in 2D
2D-RoPE is introduced, which organizes text into a 2D grid rather than a 1D sequence and assigns each token a row ID and a column ID, and suggests that viewing text in 2D can benefit language modeling.
Haodong Wen, Yiran Zhang, Yingfa Chen et al.
· 0 citations