MixDiBB: Distributed Black-Box Optimization for Mixed-Parameter Search Spaces
TL;DR
Mixed-Parameter DiBB (MixDiBB), a highly parallelizable and distributable system that scales state-of-the-art BBO algorithms to high-dimensional problems by leveraging partial separability, which is commonly found in real-world problems is presented.
Abstract
Many real-world black-box optimization problems involve a mix of continuous, integer, and categorical parameters, where gradients are unavailable and gradient-based methods are inapplicable. An alternative can be found within the family of Black-Box Optimization (BBO) methods, which includes broadly adopted approaches such as Genetic Algorithms and Evolution Strategies. The most successful implementations maintain full covariance information between parameters, yielding high sample-efficiency at the cost of quadratic performance, which restricts their applicability to low-dimensional applications. This paper presents Mixed-Parameter DiBB (MixDiBB), a highly parallelizable and distributable system that scales state-of-the-art BBO algorithms to high-dimensional problems by leveraging partial separability, which is commonly found in real-world problems. This is achieved by partitioning the parameter space based on inter-parameter relations, then applying a custom, potentially different optimization method to each partition. MixDiBB is particularly effective in applications with mixed parameter types, as it allows users to select the ideal algorithm for each parameter subset. Our results demonstrate high performance on standard benchmarks, but with three orders of magnitude more dimensions than typically showcased in the literature, and within shorter wall-clock time.