Convergence of Practical Muon
Muon is emerging as a promising alternative to AdamW for large-scale neural network training, yet theoretical understanding of its practical implementation remains incomplete, as existing analyses often simplify or omit two key components: (i) practical Newton--Schulz iterations with empirically tuned polynomial coeffi...
Hao-Nan Wang, Yu Wu, Minghui Liwang et al.
· 0 citations