Aug 2026· SIAM Review· Vol 68, pp. 656-683· 0 citations· 47 references
Computer Science
TL;DR
The groundwork for MOO is established, focusing on well-known approaches such as the weighted sum method alongside more advanced techniques like the multiobjective gradient descent algorithm (MGDA), and it is demonstrated that difficulties can arise regardless of the specific MOO method used.
Abstract
Abstract.
Recently, there has been increasing interest in the application of multiobjective optimization (MOO) in machine learning (ML). This interest is driven by the numerous real-life situations in which multiple objectives must be optimized simultaneously. A key aspect of MOO is the existence of a Pareto set, rather than a single optimal solution, which represents the optimal trade-offs between different objectives. Despite its potential, there is a noticeable lack of satisfactory literature serving as an entry-level guide for ML practitioners aiming to apply MOO effectively. In this paper, our goal is to provide such a resource and highlight pitfalls to avoid. We begin by establishing the groundwork for MOO, focusing on well-known approaches such as the weighted sum (WS) method alongside more advanced techniques like the multiobjective gradient descent algorithm (MGDA). We critically review existing studies across various ML fields in which MOO has been applied and identify challenges that can lead to incorrect interpretations. One of these fields is physics informed neural networks (PINNs), which we use as a guiding example to carefully construct experiments illustrating these pitfalls. By comparing WS and MGDA with one of the most common evolutionary algorithms, NSGA-II, we demonstrate that difficulties can arise regardless of the specific MOO method used. We emphasize the importance of understanding the specific problem, the objective space, and the selected MOO method, while also noting that the neglect of factors such as convergence criteria can result in misleading experiments.
This work presents a new approach that leverages a multitask GP model to jointly learn all objectives and constraints, and proposes a total correlation metric that enables identifying an ${optimal}$ subset of objectives and constraints to be evaluated at every round, even under uniform evaluation costs.
Ashwin Renganathan, Peter Bachman· arXiv.org· 0 citations
Many decision-making tasks are inherently multi-objective. Interactive evolutionary multi-objective optimization algorithms (iEMOAs) are particularly well-suited to help the decision-maker find their most preferred solution. However, many of these algorithms employ black-box preference models. For high-stakes decision-...
Thomas Quadt, B. Čule, R. Lindelauf et al.· 0 citations
Multi-objective optimization is essential in complex systems, where multiple conflicting objectives must be balanced simultaneously. Traditional multi-objective evolutionary algorithms (MOEAs) are effective in approximating the Pareto front but typically rely on iterative population-based optimization, which may limit...
Yu Sun, Yu-Qing Chang, Fu-Li Wang et al.· IEEE Transactions on Automat...· 0 citations
To facilitate the solving of complex problems, evolutionary multitask optimization is regarded as a promising technique due to its capacity for cross-task knowledge transfer. In contrast to empirical evidences, no mathematical runtime support exists for this technique within the domain of multi-objective optimization....
Enhancing the classification performance is more challenging than reducing the number of features in multiobjective feature selection. Such imbalance might lead the population to converge to a region on the Pareto front with a small number of features. Meanwhile, increasing the classification performance is time consum...
The hybrid approach between machine learning and mixed-integer quadratic programming (MIQP) is proposed in this study to optimize a multi-objective portfolio that trades off between return rate and downside risk. This research is inspired by traditional approaches such as Markowitz variance and fuzzy-clustering, which...
Thanut Singhsathitsuk· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.