When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark
Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a reference single-device Muon implementation with AdamW across 17.7M- and 33.6M-parameter causal language models and 2.70M-parameter vision transformers. A common 23-candi...
M. Akhtar, H. Nounou, M. Mansouri et al.
· 0 citations