A randomized first-order algorithm to compute Goldstein approximate second-order stationary points of $L$-smooth functions, using tools from randomized smoothing and applications to bilevel optimization are presented.
Abstract
In this paper, we exhibit a randomized first-order algorithm to compute Goldstein approximate second-order stationary points of $L$-smooth functions, using tools from randomized smoothing. The algorithm has oracle complexity $\widetilde{O}({ n^2}/{\varepsilon^9}+{ n^3}/{\varepsilon^7})$, where $n=1,2,\ldots$ is the input dimension and $\varepsilon>0$ is the (common) tolerance. We also present extensions to weakly convex functions and applications to bilevel optimization.
A tight lower bound is proved of $\Omega(\lambda^{1/2}\epsilon^{-7/2})$ for finding points satisfying the recently proposed relaxed notion of $(\lambda,\epsilon)$-stationarity, which allows combining further-away gradients.
We study the sample complexity of finding a point with expected hypergradient norm at most $\epsilon>0$ in nonconvex--strongly-convex stochastic bilevel optimization using fresh, globally unbiased first-order samples with bounded variance. For lower-variable smoothness order $p\ge1$, F$^2$SA-$p$ achieves $\widetilde O(...
Wen-Dao Wu, Hai-Han Zhang, Chen-Heng Zhang et al.· 0 citations
In this paper we propose first-order methods for a class of nonsmooth composite strongly convex--strongly concave and nonconvex--concave minimax optimization. We first develop an inexact proximal method and an accumulative regularizated method for strongly convex--strongly concave problems. The latter achieves the opti...
A global iteration complexity bound is established for the smallest gradient norm among the first $k$ iterates for the smallest gradient norm among the first $k$ iterates when the initial sublevel set is bounded.
We study whole-sequence convergence of the smoothing-based full-splitting proximal subgradient method (S-FSPS) for structured nonconvex and nonsmooth fractional programs, introduced by Bo\c{t}, Li, and Tao (SIAM J. Optim., 35(4):2623--2653, 2025) as Algorithm~4.1. Existing theory guarantees only the existence of a subs...
We study nonconvex--strongly-convex bilevel optimization under a stochastic first-order oracle. We introduce MRT-FD, a single-loop first-order method that simultaneously tracks the upper-level variable, the lower-level solution, and the auxiliary response arising from implicit differentiation of the hyperobjective. MRT...
Lin-Xuan Pan, Jun-Chi Yang· 1 citation
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
