Decision-Driven Regularization: A Blended Model for Learning and Optimization
TL;DR
This paper proposes a biobjective formulation that balances prediction accuracy and cost minimization, termed decision-driven regularization, which is shown to be numerically superior to other benchmarks, such as ordinary least squares, random forest, XGBoost, SPO+, perturbation gradient, and learning and rank, in the authors' synthetic studies.
Abstract
In contextual optimization, the decision maker seeks optimal decisions to minimize a cost function that varies based on observed features. This context is common in many business applications ranging from on-demand delivery and retail operations to portfolio optimization and inventory management. In this paper, we study the learning and optimization approach, which first learns how outcomes result from the features and then selects optimal decisions based on these outcomes. We focus on the integrated learning and optimization literature and identify that a lack of control for prediction accuracy can lead to overfitting and a loss of decision effectiveness against simple separate learning and optimization models. Instead, we propose a biobjective formulation that balances prediction accuracy and cost minimization, termed decision-driven regularization. It also addresses ambiguity in the definition of the cost function via a surrogate that depends on a new hyperparameter. We additionally show that alternative perspectives for formulating the problem, namely robust optimization and regret minimization, lead to models that are closely related to our proposed model. As a consequence, our framework generalizes models such as SPO+. Our model is shown to be numerically superior to other benchmarks, such as ordinary least squares, random forest, XGBoost, SPO+, perturbation gradient, and learning and rank, in our synthetic studies. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: The research of Q. Tang is funded in part by the Ministry of Education, Singapore [Tier 1 Grant RG47/24] and the National Natural Science Foundation of China [Grant 72271147]. The research of X. Zhang is supported by the National Natural Science Foundation of China [Grant 72501273], the Anhui Provincial Natural Science Foundation [Grant 2408085QG222], and the Fundamental Research Funds for the Central Universities [Grant BJ2040160100]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2024.0930 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2024.0930 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .