An Inexact Riemannian Proximal Momentum Variance-Reduced Method: Complexity Bounds and KL Sequential Convergence
TL;DR
An abstract KL principle for conditional expected descent with memory and summable tails using only the ordinary pointwise KL property is developed, which yields almost-sure finite length, whole-sequence convergence, and deterministic KL rates.
Abstract
We develop a unified analysis of inexact stochastic Riemannian proximal optimization for finite-sum nonsmooth composite problems over compact embedded submanifolds. The framework accommodates variance-reduced gradient estimators, projected momentum, and inexact tangent-space proximal solves under a single conditional error-dissipation condition, verified for projection-based SVRG, SARAH/SPIDER, SAGA, and SAG. A computable Fenchel-dual residual criterion, with tolerance prescribed before sampling and inner iterations, enables explicit control of the inner work. We establish conditional expected descent, subsequential stationarity, and an \(O(\epsilon^{-2})\) outer complexity. With SARAH/SPIDER and accumulative regularization, iRPMVR attains \(O(n+\sqrt n\,\epsilon^{-2})\) component-gradient and \(O(\epsilon^{-3})\) proximal-operator complexities. We further develop an abstract KL principle for conditional expected descent with memory and summable tails using only the ordinary pointwise KL property. A counterexample shows that a power-type expected-KL implication used in earlier stochastic analyses can fail. The principle yields almost-sure finite length, whole-sequence convergence, and deterministic KL rates.