This work analyzes a stochastic Newton optimization scheme for locating the unique global minimizer of a general nonconvex objective function and proves global almost sure convergence.
Abstract
We analyze a stochastic Newton optimization scheme for locating the unique global minimizer of a general nonconvex objective function. The method couples a Newton algorithm to additive Gaussian noise with state-dependent variance. In the bounded domain setting, we prove global almost sure convergence. The proof is based on two features of the algorithm: a nondegenerate exploratory property that ensures entrance into a neighborhood of the minimizer after a finite number of steps, and a decaying-noise property that yields contraction with high probability and prevents infinitely many exits from the neighborhood of the minimum.
We investigate the optimization problem of minimizing a nonsmooth function that satisfies a nonsmooth version of the descent lemma over a nonempty and closed but not necessarily convex set. The objective function belongs to the class of upper-$\mathcal{C}^2$ functions, whereas the constraints may promote a sparse or lo...
Christian Kanzow, Jannis Krüger, Leo Lehmann· 0 citations
We explore a new growth condition for Lipschitz functions around local minimizers, recently proposed in the context of Goldstein-subgradient-based algorithms and their behavior in practice. On generic examples, these algorithms are often observed to converge approximately linearly. We focus on objectives that are piece...
For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.~is an iteration-efficient approach, based on minimizing Fletcher's augmented Lagrangian function, for finding an approximate second-order stationary point from an arbitrary starting point. In this pa...
F. Curtis, Ling-Jun Guo, Daniel P. Robinson· 0 citations
A new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian that achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual, the first second-order method for this problem class attaining this rate.
