Why Clipping Matters in AdaGrad? Toward a High-Probability Theory under Generalized Smoothness
This work analyzes the original same-step coordinate-wise AdaGrad under generalized smoothness and heavy-tailed noise with bounded variance and shows that clipping is a structural stabilizer of the adaptive geometry rather than merely a robustness heuristic for AdaGrad under heavy-tailed noise.
Alokendu Mazumder, Ayaan Mohd, Harshit Rawat et al.
· 0 citations