Machine Learning Optimization and Applications

Abstract

Optimization is a critical research area with increasing relevance to machine learning. For a range of machine learning tasks—such as hyperparameter tuning, knowledge transfer, and feature selection—optimization techniques are essential. Hyperparameter tuning, for instance, relies on optimization to identify the ideal set of parameters to maximize model performance, while feature selection uses optimization to isolate the most relevant data attributes, improving accuracy and reducing model complexity. Knowledge transfer between domains also benefits from robust optimization methods, ensuring that models trained in one setting adapt effectively to new contexts. This talk will draw on current research in our lab to provide an overview of optimization in machine learning, highlighting two primary integrations: (1) using optimization to enhance machine learning, and (2) applying machine learning to support optimization. We’ll illustrate these approaches with two examples from our work. The first example demonstrates how distributionally robust optimization can improve knowledge transfer across different domains by developing a new loss function to account for shifts in data distributions, thereby enhancing model robustness. The second example leverages predictive machine learning models to provide critical parameter information in a planogram optimization problem, a task commonly encountered in retail and logistics to optimize product placements. We hope this talk offers valuable insights into the potential of these approaches, sparking discussions and inspiring new directions to further advance research in both optimization and machine learning.