A new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian that achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual, the first second-order method for this problem class attaining this rate.
Abstract
We develop a new direct accelerated Newton method for minimizing convex functions with Lipschitz continuous Hessian. The algorithm uses only primal variables and performs just one linear solve per iteration. With a simple predetermined choice of parameters, it achieves the global convergence rate of $O(1/k^3)$ in terms of the functional residual. To the best of our knowledge, this is the first second-order method for this problem class attaining this rate while relying solely on one linear system solve per iteration (without solving auxiliary nonlinear regularized subproblems, such as cubic regularization, performing nonlinear parameter searches, or using dual extragradient corrections). Our method can be implemented in a Hessian-free way, using an inexact linear system solver, while preserving the fast global rate. We further extend our construction to arbitrary geometry through Bregman divergence, and to composite optimization problems.
This paper proposes a general line-search Newton framework for unconstrained optimization that avoids repeated Hessian regularization by exploiting the Newton direction only when it is well-defined and suitable and provides the first Newton-type algorithm together with a comprehensive convergence analysis for this impo...
The reduced linear system used to compute the Newton step is derived, the corresponding merit function is defined, and practical approaches for constructing the diagonal scaling matrix from derivative information are discussed.
E. Bertolazzi, Alberto De Marchi, Davide Stocco· 0 citations
As an extension of convex quadratic optimization (CQO) problems, the weighted convex quadratic optimization (WCQO) plays an important role in the domain of mathematical programming and engineering. In this paper, we propose a short-step primal-dual interior-point algorithm for solving WCQO based on the strategy of weig...
Rima Hamadouche, L. Derbal, M. Achache· Reserche operationelle· 0 citations
This work proposes a nonlinear-residual linearized augmented Lagrangian method (NR-LALM) that replaces this subproblem by a regularized Gauss-Newton-type step while retaining the classical multiplier update based on the nonlinear constraint residual.
Ben-Qi Liu, Kangkang Deng, Zichen Wang et al.· 0 citations
For solving nonconvex equality-constrained optimization problems, a recent Gradient-Eigenstep Algorithm by Goyens et al.~is an iteration-efficient approach, based on minimizing Fletcher's augmented Lagrangian function, for finding an approximate second-order stationary point from an arbitrary starting point. In this pa...
F. Curtis, Ling-Jun Guo, Daniel P. Robinson· 0 citations
It is shown that the augmented Lagrangian subproblems preserve the $(L_0,L_1)$-smooth structure, with parameters depending on the penalty coefficient, which allows us to employ recent accelerated first-order schemes designed for generalized smooth optimization instead of classical smooth optimization methods.
A. Vyguzov, F. Stonyakin· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
