The Single-Loop Predictor--Corrector Extragradient Method with Damped Momentum (PCE-DM), which combines an extragradient prediction--correction scheme with a novel auxiliary feedback recursion for the weaker-curvature variable, closes the condition-number complexity gap between standard single-loop methods and near-optimal multi-loop methods.
Abstract
We study smooth strongly convex--strongly concave minimax optimization in the deterministic unconstrained setting, without assuming a bilinear or separable structure. Although existing multi-loop methods attain near-optimal condition-number dependence, standard single-loop methods generally exhibit a substantial complexity gap. To close this gap, we propose the Single-Loop Predictor--Corrector Extragradient Method with Damped Momentum (PCE-DM), which combines an extragradient prediction--correction scheme with a novel auxiliary feedback recursion for the weaker-curvature variable. PCE-DM uses fixed parameters and two new full-gradient evaluations per iteration after one initialization query, while requiring no inner solves, accuracy schedules, or staged restarts. We develop a Lyapunov analysis that controls the predictor--corrector mismatch through corrected-gradient increments and establish last-iterate linear convergence. Specifically, PCE-DM computes an $\varepsilon$-accurate relative solution, measured by the squared Euclidean distance to the saddle point, within $\mathcal{O}\!\left(\sqrt{\kappa_x\kappa_y} \log(2\kappa_x\kappa_y/\varepsilon)\right)$ full-gradient queries. This result closes the condition-number complexity gap between standard single-loop methods and near-optimal multi-loop methods, matching the known lower-bound order up to logarithmic factors while retaining fixed, explicit single-loop updates. Numerical experiments on regularized linear regression and AUC maximization demonstrate the computational efficiency of PCE-DM.
These OS guarantees match the best-known bounds achieved by multi-loop methods while preserving a single-loop implementation, and provide the best-known SFO complexity guarantees among single-loop stochastic first-order methods for the respective stationarity criteria and problem classes.
Hui-Ling Zhang, Min-hao Zhang, Zi Xu et al.· 2 citations
Recent advances in penalty-based methods for stochastic bilevel optimization (SBO) have eliminated the need for second-order derivative oracles. However, for stochastic nonconvex-strongly convex bilevel problems, existing first-order methods typically rely on nested loops and/or large batch sizes for attaining $O(\epsi...
Xing-Yu Chen, Ming Yang, Quan-Qi Hu et al.· 0 citations
A new single-loop algorithmic framework for smooth nonconvex--concave minimax optimization and a lower bound of $\Omega(L^2D_Y\Delta_\phi\varepsilon^{-3})$ for optimization stationarity over projected zero-respecting first-order methods is established.
This work proposes a smooth approximation of PBO through reformulation, penalization and regularization, and establishes convergence guarantees in terms of both minimizers and stationarity, and develops two single-loop algorithms for deterministic and stochastic PBOs, respectively.
Qi Cao, Bo Zeng, Shang-Zhi Zeng et al.· 0 citations
We study nonsmooth convex-concave saddle-point problems over compact convex sets, assuming access to stochastic subgradients of the payoff function. We develop single-loop projection-free algorithms that use linear minimization oracles over the primal and dual domains. Unlike prior projection-free approaches that rely...
We study convex optimization over compact convex sets for twice differentiable objective functions with positive definite Hessians, assuming access to the feasible set through a linear minimization oracle. Existing second-order Frank--Wolfe methods either provide only a local linear rate over general convex sets or ach...
Shota Takahashi, Akiko Takeda· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.