#machine learning
May 2026
Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior
Latent Recurrent Transformer is studied, a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token, while retaining one-forward-per-token decoding with 9% latency overhead over the standard Transformer.
Zeyi Huang, Xuehai He, Liliang Ren et al.
· arXiv.org · 2 citations