Preprint
Aug 2026
LoopMTP: A looped transformer guided by latent multi-token prediction
Multi-token prediction (MTP) supplies exactly the dense, forward-looking supervision the loop is missing, by softly aligning the hidden state of loop $t$ with the embedding of the token $t$ steps ahead, while a lightweight gate preserves useful information across iterations.
Behzad Shomali, Markus Frey, D. Berghaus et al.
· 0 citations