SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
Results show that looping can improve Transformers even under budget matching, offering a practical recipe that turns depth reuse into measurable gains.
Shaowen Wang, Ge Zhang, Kai-Rong Luo et al.
· 4 citations