A global iteration complexity bound is established for the smallest gradient norm among the first $k$ iterates for the smallest gradient norm among the first $k$ iterates when the initial sublevel set is bounded.
Abstract
We study the BFGS method with an Armijo-Wolfe line search for minimizing convex functions with Lipschitz-continuous gradients, without assuming strong convexity. We establish a global iteration complexity bound of $\mathcal{O}(k^{-1/2})$ for the smallest gradient norm among the first $k$ iterates. Moreover, when the initial sublevel set is bounded, we show that the function value gap converges at a rate of $\mathcal{O}(k^{-1})$. Our analysis leverages the classical trace-log-determinant potential function and reveals that a key inequality underlying this potential function remains valid without strong convexity.
We consider the design of optimal fixed-step first-order methods for $M$-Lipschitz convex optimization given $\|x_0-x_\star\|\leq D$. Prior works have identified several distinct fixed-step methods, parameterized by a matrix of stepsizes $W$, with the (information-theoretic) minimax optimal rate $MD/\sqrt{N+1}$ of obje...
This work proposes a computationally efficient algorithm that achieves the optimal $O(1/\sqrt{T})$ convergence rate, matching the lower bound, and closes the existing gap in one dimension, providing the first sharp rate guarantee in this setting.
A. Carpentier, Chloé Rouyer, Alexandre B. Tsybakov et al.· 0 citations
We study the last iterate of the projected subGradient Method (sGM) for convex Lipschitz objectives defined on $\mathbb{R}^d$. We prove that, for a finite horizon $n$ and a constant stepsize $\eta=\Theta(1/\sqrt n)$, the last iterate achieves an optimization error of order $d/\sqrt n$, showing that the extra $\log n$ f...
This work presents a new lower bound of $\Omega(T^{-1.9319})$ for the last-iterate convergence rate of gradient descent with predetermined nonnegative stepsize schedules, and provides rigorous evidence that stepsize schedules alone cannot accelerate plain GD to the optimal $O(T^{-2})$ convergence rate.
This work gives a deterministic and parameter-free Frank-Wolfe-type method with a global ambient-dimension-independent linear convergence rate under quadratic growth.
We develop an argmax principle for analyzing sum-of-squares relaxations of optimization problems over the unit sphere. Given a feasible pseudo-expectation, we form a polynomial of high-order pseudo-moments, such as $\Phi_k(u)=\widetilde{\mathbb E}\langle x,u\rangle^{2k}$. Our guiding principle is that its maximizers ar...
F. Granha, Pei Wu, Hao-Chen Xu· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
