Generative heuristics is introduced, a methodology that combines traditional metaheuristics with LLM-based semantic evaluation to address ‘soft optimization’ problems containing both hard quantitative constraints and soft qualitative objectives.
Abstract
In many real-world domains, optimization problems involve both quantitative objectives and qualitative criteria that cannot be directly formulated as mathematical functions. While recent research has integrated large language models (LLMs) into optimization algorithms, these approaches primarily use LLMs to guide the search process, generate heuristics, or tune optimization strategies, rather than to evaluate semantic objectives. This paper introduces generative heuristics, a methodology that combines traditional metaheuristics with LLM-based semantic evaluation to address ‘soft optimization’ problems containing both hard quantitative constraints and soft qualitative objectives. The proposed methodology first applies a metaheuristic algorithm to generate a shortlist of high-quality candidate solutions satisfying the quantitative objectives. These candidates are then evaluated by an LLM according to a user-defined qualitative rubric, and the resulting semantic scores are incorporated into the optimization process through a weighted soft objective. The methodology is illustrated through a constrained portfolio optimization problem in which financial risk is minimized while simultaneously promoting strategic objectives related to sector diversification, geographic exposure, environmental-social-governance quality, and overall portfolio coherence. Experimental results show that the proposed methodology consistently improves strategic alignment while producing solutions that remain close to those obtained from the hard optimization formulation.
Large Language Models (LLMs) are opening new directions for automated heuristic design (AHD), allowing evolutionary methods to create and enhance heuristics for constrained optimization problems (COPs). However, most existing approaches face the challenge of the exploration-exploitation balance, where the evolution needs to escape convergence to homogeneous populations and discover as large a heuristic landscape as possible. To address this challenge, we introduce Quality-Diversity Evolution (QDEvo), a multi-objective framework that integrates Quality-Diversity optimization with LLM-based AHD. At its core, QDEvo employs a semantic survival selection mechanism that clusters algorithms by functional similarity, then applies local Pareto competition. Evaluation on well-known COPs benchmarks and real-world problems shows that our method consistently outperforms the state-of-the-art baseline in both Hypervolume and Inverted Generational Distance metrics. These results facilitate further exploration of the algorithmic design space, while ensuring competitive solution quality and efficiency.
Nam Do Khanh, Nhat Nguyen Tran Minh, Dat Pham Vu Tuan et al.· Proceedings of the Genetic a...· 1 citation
Multi-objective optimization provides the mathematical foundation for reasoning about trade-offs in complex decision problems, from engineering design to resource allocation. Software testing exemplifies such problems: practitioners must simultaneously optimize for fault detection capability, code coverage, execution cost, and test suite diversity—objectives that are fundamentally incommensurable. Since the early 2000s, multi-objective evolutionary algorithms (MOEAs) such as NSGA-II, MOEA/D, and their many-objective extensions (MOSA; DynaMOSA) have served as the dominant mathematical framework for navigating these trade-offs through Pareto-front approximation with hand-crafted fitness functions. However, the recent emergence of reinforcement learning (RL) and large language models (LLMs) is shifting the optimization paradigm from numerical Pareto-front approximation toward neural, semantically aware decision making over learned representations. This paper presents a systematic mapping study of multi-objective optimization algorithms, tracing their evolution from classical Pareto-based methods toward AI-driven and hybrid approaches, with software testing as the primary application domain. We survey 120+ papers published from 2000 to 2025 and propose a novel five-level taxonomy (L1–L5) that classifies optimization approaches along the intelligence spectrum: classical MOEAs, ML-guided MOEAs, RL-driven optimization, LLM-driven optimization, and hybrid neuro-evolutionary systems. For each level, we analyze the mathematical problem formulations (Pareto optimality conditions, Markov decision processes, and neural loss landscapes), objective function design, algorithmic convergence properties, and computational complexity. We further conduct a cross-cutting mathematical analysis comparing these paradigms along dimensions of convergence, diversity, scalability, and interpretability. Our survey identifies critical open mathematical challenges: the lack of formal convergence guarantees for LLM-driven optimization, the under-exploration of many-objective (m≥4) formulations in AI-driven testing, the sample complexity of reinforcement learning for combinatorial test optimization, and the absence of standardized benchmarks with known Pareto-optimal frontiers. We conclude by outlining a research roadmap for the next generation of multi-objective optimization systems that combine the complementary mathematical strengths of neural function approximation and evolutionary diversity preservation.
This work investigates the hybridization of a Genetic Algorithm with Pareto Local Search to improve the exploration of non-dominated rulesets and provides a documented baseline for hybrid metaheuristics in rule mining.
Evgueni Blanquart, L. Jourdan, Nadarajen Veerapen· Proceedings of the Genetic a...· 0 citations
Structural optimization problems often involve a large number of decision variables and highly non-convex feasible regions, making convergence to the true Pareto front extremely challenging. Even when convergence is achievable, it typically requires thousands of function evaluations, resulting in significant computational cost. This highlights the need for efficient and robust optimization algorithms for real-world engineering applications. In this study, we introduce a novel constrained multi-objective evolutionary algorithm, termed DPCME. The algorithm employs two interacting populations that exchange information, enabling effective global exploration and reducing the risk of convergence to local optima. To further enhance performance, a recent repair-based constraint-handling technique is incorporated, and alternative repair approaches are proposed and systematically evaluated. The proposed algorithm is tested on three engineering problems: the 72-bar truss, the 120-bar truss, and a chemical tanker structure, each involving hundreds of nonlinear failure constraints. Its performance is evaluated against state-of-the-art constrained multi-objective optimization algorithms from the latest PlatEMO package. A total of 43 algorithms are initially tested, from which the 12 best-performing methods are selected for detailed comparison. The results demonstrate that DPCME achieves superior or competitive convergence and diversity across all test cases, and that the inclusion of repair-based constraint handling further improves its performance.
Mixed-integer optimization (MIO) involves solving a problem defined by both discrete and continuous decision variables. Although common in real-world applications, MIO is difficult to solve because traditional metaheuristics are primarily designed for a single variable type. This paper compares six different strategies identified in a recent survey, categorizing them as either global strategies (GS), which treat the decision vector as a unified type, or decomposition-based strategies (DS), which split the problem into continuous and discrete subproblems. We benchmark these strategies based on a genetic algorithm across the 24 functions of the COCO BBOB-mixint test suite. The results show that GS are more efficient at performing simple functions and are more robust in multi-modal landscapes. As expected, however, DS are vulnerable to variable interdependencies. A resolution limit has been identified for high conditioning functions, whereby all strategies reach a performance ceiling. This is partly because the landscape features are smaller than the minimum integer step size of 1, causing the algorithm to exceed the optimal level. These results suggest that future research should explore hybrid metaheuristics to overcome the inherent constraints of the integer grid.
Thomas Lang, Denis Pallez· Proceedings of the Genetic a...· 0 citations
We address the problem of achieving convergence and diversity in many-objective problems, focusing on continuous and unconstrained functions. It is known that with increasing numbers of objectives (say from 4 to 20) even modern many-objective Evolutionary Algorithms (EAs) may struggle to converge to, and fully distribute across the Pareto front. This paper presents a general and modular hybrid approach that integrates local search into reference-vector-based Many-Objective Evolutionary Algorithms (MaOEAs), addressing issues such as weakened selection pressure and the increasing complexity of exploring high-dimensional objective spaces. The hybrid approach employs Sequential Quadratic Programming (SQP) guided by achievement scalarizing directions, derived from either the Weighted Achievement Scalarizing Function (W-ASF) or the Penalty-based Boundary Intersection (PBI) schemes, depending on the decomposition strategy of the underlying MaOEA. It is designed to be broadly applicable with limited parameter tuning, facilitating integration with algorithms from the NSGA-III and MOEA-DD families. The effectiveness of the proposed approach is demonstrated through extensive experiments on standard continuous-variable many-objective benchmark problems as well as on representative real-world case studies. Results show that integrating local search significantly enhances performance, while a principled method for setting hybrid parameters ensures robustness and reproducibility. Although limited to an empirical study over a (large) test function suite, these findings highlight the potential of combining mathematical programming techniques with evolutionary algorithms for high-dimensional many-objective optimization problems.
Regina C. L. C. de Sousa, Dênis E. C. Vargas, Elizabeth F. Wanner et al.· Journal of Heuristics· 0 citations