Understanding Stability of Outer Nesterov Momentum on Stochastic Pseudo-Gradients
An exact mean-square stability criterion in any dimension is derived, which reveals that the inner batch size determines whether increasing the number of local steps eventually stabilizes or destabilizes the outer iteration.