A preconditioned proximal Barzilai--Borwein method for multiobjective composite optimization that combines objective-wise Barzilai--Borwein scaling with a common preconditioner that captures shared curvature information is proposed.
Abstract
Multiobjective composite optimization problems arise in sparse regularization, constrained multiobjective models, and multi-task learning, but their numerical solution remains challenging when the smooth components are ill-conditioned. Proximal gradient methods are inexpensive per iteration but may converge slowly, while proximal Newton and quasi-Newton methods exploit curvature information at the cost of evaluating expensive metric proximal mappings. To address these issues, we propose a preconditioned proximal Barzilai--Borwein method for multiobjective composite optimization. The method combines objective-wise Barzilai--Borwein scaling, which reduces imbalance among objectives, with a common preconditioner that captures shared curvature information. To avoid non-diagonal metric proximal mappings, we develop a subspace variant in which the search direction is computed in a two-dimensional subspace generated by a proximal-gradient-type direction and a projected historical direction. By constructing a conjugate basis with respect to the preconditioning metric, the subspace model decomposes into tractable one-dimensional subproblems. The framework is further extended to nonsmooth terms of the form $g_i(Ax)$ through a linear-operator-aware preconditioner, yielding explicit proximal evaluations via dual subproblems. We also analyze an inexact version based on relaxed descent conditions. We establish the global convergence of the inexact algorithm in the nonconvex setting and prove a linear convergence rate under an error-bound condition. Numerical experiments on ill-conditioned $\ell_1$-regularized, structured $\ell_1$-regularized, and linearly constrained problems demonstrate the effectiveness of the proposed method.
Many image-processing problems can be formulated as composite cardinality optimization (CCOP), whose objective is the sum of two convex terms and a cardinality function composed with a linear operator. The composite cardinality term creates major algorithmic challenges: the separability of the cardinality function is lost and convergence analysis often requires surjectivity-type assumptions on the linear operators. To overcome these challenges, we consider the stationary dual formulation of CCOP, which has more favorable structure consisting of two convex terms and a separable cardinality function. In this paper, we develop an efficient proximal point algorithm (PPA) to solve the stationary dual problem. The efficiency of our PPA stems from two aspects. Firstly, the key step of its subproblem solver minimizes a smooth convex function over a low-dimensional subspace by the classic semismooth Newton algorithm (SNA), which has global convergence and local superlinear rate under suitable conditions. Secondly, implementable inexact criteria are proposed for early termination of the SNA subroutine. These components form the basic framework of our inexact PPA. Under suitable conditions, it enjoys global convergence and local linear convergence rate. In particular, we provide examples in which the convergence assumptions are automatically satisfied. Finally, the SNA subroutine is incorporated into our inexact PPA to solve jump-sparse signal recovery and computed tomography (CT) image restoration. Numerical results demonstrate the time efficiency and solution accuracy of our proposed method.
This work proposes a class of Multi-Objective Moreau Envelope based Hessian-free Algorithms (MOMEHA) to solve the multi-objective bilevel learning problems with nonconvex lower level and proposes a momentum-based variant of MOMEHA (i.e., MB-MOMEHA) method to solve the stochastic multi-objective bilevel learning problems.
In this paper, we propose a balanced augmented Lagrangian method based on accelerated stochastic ADMM (b-ASADMM) to efficiently solve structured separable nonconvex optimization problems subject to linear constraints. The objective function in this problem comprises potentially nonsmooth and smooth functions, where the smooth function is an average of multiple nonconvex smooth functions. The involved smooth subproblem is tackled by an accelerated stochastic gradient method based on weighting of stochastic item and pre-variable. The involved nonsmooth subproblem is solved under incorporation of Bregman distance to avoid the case that subproblem does not have a closed-form solution due to the complicated quadratic term or other hindering. The involved balanced augmented Lagrangian method advances the original ALM by balancing its subproblems and improving its implementation. In contrast to most deterministic and stochastic ADMMs, our dual variable allows a more flexible and larger step-size region. By standard smoothness assumption, we establish the global convergence and iteration complexity of the generated sequence. Furthermore, we provide a linear convergence rate of b-ASADMM under a local error bound condition and the weakly convex property of the nonsmooth component. Numerical experiments on the graph-guided fused Lasso problem and the smooth clipped absolute deviation penalty problem are conducted to verify the effectiveness of b-ASADMM.
This paper studies a non-separable composite $\ell_0$-$\ell_2$ regularization model that simultaneously enforces sparsity and smoothness for inverse problems. The $\ell_0$ norm induces inherent nonconvexity and nonsmoothness, while linear transformations further introduce nonseparability, making the problem computationally challenging to solve. The existing inexact augmented Lagrangian method suffers from high computational complexity and unstable convergence. To overcome these difficulties, we develop two novel augmented Lagrangian algorithms with exact multipliers, designed respectively for the full row-rank case and the general matrix case, where all subproblems are globally optimized via closed-form solutions. Furthermore, we prove linear convergence of the proposed method when the transformation matrix is full row rank. In the general setting, all accumulation points of the generated sequence are KKT points for the original problem. Numerical experiments on synthetic data, trend filtering, and image smoothing demonstrate the superior efficiency and accuracy of the proposed methods over the existing method, confirming our theoretical analysis.
Empirical risk minimization is a standard and effective paradigm for learning predictive models by minimizing average loss. In high-stakes decision-making, however, an average-loss criterion may underrepresent rare but severe losses. Spectral risk measures (SRMs) provide a principled framework by incorporating weighted order statistics of losses, but the induced nonsmoothness and nonseparability from sorting make the resulting optimization problems challenging. We propose a relative inexact proximal augmented Lagrangian method with a semismooth Newton subproblem solver for solving SRM-based optimization problems. Exploiting a dual reformulation and properties of the Moreau envelope, we reduce the subproblems to structured dual-variable formulations, significantly simplifying computation. We provide explicit generalized Jacobian characterizations and tailor the pool adjacent violators algorithm for their efficient evaluation. Numerical results on synthetic and real-data instances show that the proposed method attains lower running times than the tested ADMM baseline while producing comparable stationarity residuals and sparse solutions.
Rufeng Xiao, Rujun Jiang, Xudong Li et al.· 0 citations