Jul 2026· Proceedings of the Genetic and Evolutionary Computation Conference Companion· pp. 17-18· 0 citations· 4 references
TL;DR
This work proposes an automated framework that generates and refines benchmark suites using large language models (LLMs) and evolutionary search and shows that this combined approach substantially increases discriminability, improving scores from approximately 2.04 to 2.58.
Abstract
The GECCO Benchmark Design Challenge formulates benchmark construction as the problem of maximizing performance differences between optimizers, measured via pairwise rank distances. We approach this as a meta-optimization task and propose an automated framework that generates and refines benchmark suites using large language models (LLMs) and evolutionary search. Candidate functions are synthesized through LLM-driven evolution and evaluated directly on their ability to discriminate between five standard optimizers under a fixed evaluation budget. We further optimise the suite by combinatorial optimization of all generated functions over all considered dimensionalities in the range of [2,80). Our results show that this combined approach substantially increases discriminability, improving scores from approximately 2.04 to 2.58. These findings highlight that effective benchmark design requires not only diverse function structures but also careful control of problem dimensionality, and demonstrate the potential of learning-driven methods for constructing targeted benchmarking suites.
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
LLM-driven evolutionary search can discover algorithm designs that achieve Pareto-efficient trade-offs difficult to reach through manual design, with SMAC hyperparameter optimization integrated into the evolutionary loop.
G. Laskaris, R. Brasher, Niki van Stein et al.· 0 citations
This study investigates whether the current generation of frontier Large Language Models can predict the relative performance of metaheuristic optimization algorithms from source code alone, potentially reducing the computational cost of benchmarking during automated algorithm design. We task three models (GPT 5.4 Thinking, Gemini 3.1 Pro, and Claude Opus 4.6) with predicting improvement or regression between consecutive iterations of iteratively refined differential evolution, and with ranking independently designed algorithms on standard benchmarks. Initial results are promising (88% accuracy), but accuracy collapses on a second algorithm sequence (22–44%). A controlled comparison experiment provides evidence that the models' predictions correlate with code complexity metrics rather than with actual algorithmic performance. Self-reported confidence scores are poorly calibrated across all models tested. These findings indicate that while LLM-based performance prediction is not yet a reliable substitute for benchmarking, it shows potential as a complementary pre-screening tool in iterative algorithm design, particularly in settings where code modifications follow predictable patterns.
Michal Pluháček, Paweł Kolendo, Krzysztof Tylka-Suleja et al.· Proceedings of the Genetic a...· 0 citations
Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the landscape of over one hundred methods remains fragmented. We therefore present OmniOpt, a unified survey and benchmark cookbook of optimizers for the research community. OmniOpt rests on four coupled components. First, we treat every optimizer update as a structured transformation through a five-stage meta-pipeline, and show that most methods engage only one or two of these stages. Second, we use norm-constrained linear minimization oracles (LMOs) to unify different optimizers. Third, these two views ground a dual-dimension taxonomy, one dimension assigning each method to a mechanism family and the other recording the measurable training objectives it aims to improve. Fourth, and at the core of this paper, we instantiate the full taxonomy in a unified cross-domain benchmark spanning representative optimizers, model scales, and training regimes from language model pretraining to image classification, systematically analyzing each method family across multiple effect objectives and laying out their trade-offs. OmniOpt thus supplies the research community with an operational coordinate system for selecting optimizers under explicit mechanism and objective assumptions, and charts a direction for the future development of the optimizer community.
Siyuan Li, Jiabao Pan, Yumou Liu et al.· 0 citations
The evaluation of heuristic optimizers on test problems, better known as benchmarking, is a cornerstone of research in multiobjective optimization. However, many frequently used test problems either feature a limited degree of optimization challenges or have poorly understood reference solutions. Here, we present an overview of BONO-Bench [5], a recently proposed problem generator and benchmark set for bi-objective numerical optimization. Building on convex-quadratic problems, it features diverse challenges ranging from different levels of conditioning, shapes of Pareto set and front as well as plateaus to different structured and unstructured multimodality patterns. Furthermore, we enable best practices for empirical runtime analysis of optimizers using reference solutions that can be approximated to an arbitrary degree, resulting in precise target values for the hypervolume and exact R2 indicators.
Lennart Schäpermeier, P. Kerschke· Proceedings of the Genetic a...· 0 citations
Learning-to-optimize (L2O) methods accelerate repeated optimization by training models to predict solutions, warm starts, branching decisions, or other forms of solver guidance. A critical yet largely overlooked component of these pipelines is the feature function that maps problem instances to inputs for machine learning models. Existing L2O methods typically rely on hand-crafted features, making representation design manual and largely fixed across domains. We introduce FunL2O, the first unified framework for automating feature design through LLM-driven program evolution for L2O. In a FunSearch-style loop, an LLM proposes executable feature functions, while a fixed evaluation process retrains the original L2O model and measures downstream optimization performance. We evaluate FunL2O on linear and quadratic programming tasks involving solution prediction and warm-starting, as well as on mixed-integer optimization tasks using GNN-guided backdoor branching and Predict-and-Search. Across continuous and discrete optimization tasks and four LLMs, the evolved features consistently outperform hand-crafted representations. These results establish LLM-driven feature evolution as a general and effective approach to automating representation design in L2O.
Bingheng Li, Junyang Cai, Yupeng Zhang et al.· 0 citations