Arithmetic Simplicity in Stochastic Gradient Methods
This work shows how to transform AdaGrad, Adam, and AdamW into arithmetically simple, and the convergence analysis is given for a static Adam, which is also arithmetically simple.
Bin Fu, Peng-Fei Gu, J. Nuñez et al.
· 0 citations