LoopAhead: Speculating Ahead for Faster Decoding in Looped Language Models
Looped Transformers improve parameter efficiency by repeatedly applying the same Transformer layers. However, generating each token still involves sequential computation and repeated access to model weights, leading to high decoding latency. Recent pipelined self-speculative decoding methods reduce this delay by drafti...
Zi-Xuan Liu, Nan Xue, Rui Bao et al.
· 0 citations