Csaba Szepesvari

On the Two Faces of Adam in Separable Linear Classification

This work gives sufficient conditions under which an Adam-trained classifier nearly maximizes the max-norm margin before the updates become gradient-like, and shows that for polynomially decreasing stepsizes with exponent \(a), the updates become approximately proportional to the negative gradient after $\Theta(1/\epsi...