B. K. Le

Stationarity Floors and Vanishing Perturbations in Sharpness-Aware Minimization

We study a deterministic family of sharpness-aware minimization methods for smooth nonconvex functions. The perturbation is $$ y_k=x_k+\rho\, \frac{\nabla f(x_k)}{\norm{\nabla f(x_k)}^\alpha}, \qquad 0\leq\alpha\leq 1, $$ so that its effective radius is $\rho\norm{\nabla f(x_k)}^{1-\alpha}$. For $0<\alpha\leq1$, we giv...