Stochastic gradient methods have gained increasing attention for solving large-scale inverse problems due to their computational efficiency. However, their theoretical justification in the context of ill-posed problems remains underdeveloped, particularly regarding convergence rate analysis, where existing results typically yield only suboptimal rates. In this paper, we address this gap by establishing order-optimal convergence rates for a stochastic gradient method applied to linear ill-posed problems in Hilbert spaces. Under Hölder-type source conditions with smoothness parameter
$$\nu \in (0, 1/2]$$
ν
∈
(
0
,
1
/
2
]
, we derive convergence rates both in expectation and almost surely, accommodating a broad class of step-size sequences, including constant and polynomially decaying ones. Our analysis is based on a delicate Lyapunov-type argument and an application of the Robbins–Siegmund theorem. As a byproduct, we also establish new convergence results that do not rely on any source conditions.
Four groups of subspace methods for nonlinear monotone equations, with applications to large-scale machine learning problems, using Jacobian-free subspace directions of conjugate-gradient type combined with either fixed step sizes or variable step sizes generated by the projected method of Solodov and Svaiter are intro...
M. Kimiaei, Shima Shabani, Michael Breuß· 0 citations
Stochastic variance reduced gradient (SVRG) is a variant of stochastic gradient descent and is a promising iterative method for solving large-scale inverse problems. Nevertheless, the development of theoretically grounded a posteriori stopping rules for SVRG remains an open challenge. In this work, we provide a converg...
We investigate the convergence of the Levenberg–Marquardt (LM) iterative regularization method for linear ill-posed inverse problems in Hilbert spaces under general source conditions characterized by admissible index functions. We introduce admissibility conditions tailored to the spectral filter structure of the LM it...
P. Pornsawad, N. Chumchob, Wannapa Panitsupakamon· AppliedMath· 0 citations
For weak solutions to quasilinear degenerate parabolic equations of $p$-Laplace type, a central obstacle in applying the method of intrinsic scaling to prove their H\"{o}lder regularity is the derivation of forward-in-time propagation estimate for the spatial measure of level sets. In this paper, we revisit and overcom...
A globally convergent regularized Newton method with positive definite regularization for solving nonsmooth optimization problems that replaces the identity matrix in traditional algorithms with a general positive-definite symmetric matrix to regularize the generalized Hessian.
