Juan-Andres Perez Rua

Scaling Muon for Diffusion Transformers

Periodic Row-wise Muon is introduced, which performs a full NS5 spectral update once every \(K\) steps and applies a low compute and communication cost row-wise constrained update based on the current momentum at the remaining steps to preserve Muon's generative quality advantage while translating it into end-to-end training efficiency for large DiTs.