Preprint
Aug 2026
On the Principles Behind Neural Network Optimizers
This thesis develops a principled grounding for Adam and motivates new designs, and reveals new local structures in matrix-based nonconvex problems, and helps understand and improve recent NN optimizers, such as Muon.
Yu-Shun Zhang
· 0 citations