Jul 2026· Proceedings of the Genetic and Evolutionary Computation Conference Companion· 0 citations· 23 references
Abstract
We investigate the use of large language models (LLMs) as evolutionary operators for optimizing the architecture of multi-agent systems (MAS) - a representative instance of LLM-driven optimization over attributed-graph artifacts that arise in generative design. Given a task description, an initial MAS configuration is generated and then refined on a per-task basis through two optimization strategies: (1) a population-based evolutionary approach where LLMs serve as mutation, crossover, and selection operators, with an LLM-as-a-Judge ensemble providing the surrogate fitness signal; and (2) a simpler iterative regeneration scheme that uses textual feedback from LLM judges to guide single-solution improvement without maintaining a population. We evaluate both approaches on the GAIA benchmark. Our experiments reveal a critical sensitivity to the judge model's quality: a capable judge (Claude 4 Sonnet) yields +14.3 and +6.67 percentage-point accuracy improvements for evolutionary and iterative approaches, respectively, whereas a weaker judge (Gemini-2.5-Flash) provides no benefit or even degrades performance below the unoptimized baseline. Both approaches incur substantial computational costs, raising questions about practical cost-effectiveness. These results highlight fundamental challenges of surrogate fitness functions in LLM-driven evolutionary optimization and offer practical insights for applying evolutionary methods to MAS architecture design.
Automated evolution of Multi-Agent Systems (MAS) holds significant potential for reducing the manual effort required to design and optimize LLM-based agent architectures. However, extant search-based paradigms face a fundamental trade-off, where an expanded optimization scope exacerbates evolutionary instability, while discrete branch-and-discard search isolates insights across lineages. To address these limitations, we propose a continuous, data-driven optimization paradigm built upon a unified ReAct-based infrastructure that reconciles a broad optimization scope with operational stability. Under this paradigm, we present OptiMAS, a task-agnostic agentic optimizer that leverages textual interaction trajectories and task feedback as loss signals for end-to-end MAS evolution. Equipped with a novel dual-track memory mechanism, OptiMAS sustains performance improvement over extended optimization horizons. Evaluation on four heterogeneous agentic benchmarks with three varying scale and accessibility LLM backbones, demonstrates that OptiMAS consistently achieves competitive or superior accuracy relative to both domain-specialized hand-crafted systems and existing evolutionary methods. Our work establishes a practical milestone toward robust, automated MAS evolution.
Yuxin Cheng, Chang Liu, Hanxin Yu et al.· 0 citations
In evolutionary robotics, island models promote exploration by evolving multiple subpopulations, though migration is typically governed by fixed heuristics. Meanwhile, quality-diversity (QD) methods like MAP-Elites excel in large-population regimes but struggle with small populations due to sparse descriptor space coverage. We propose an LLM-guided migration controller for island-model evolutionary design, where an LLM receives compact summaries of island states including fitness statistics and morphology information and outputs migration decisions. We use an LLM-based mutation operator as a component of the evolutionary pipeline, but focus on migration as a process-level control mechanism. Evaluated on three EvoGym benchmarks, our approach maintains substantially higher structural diversity than mutation-only evolution while achieving comparable fitness, and approaches the diversity of QD methods. These results demonstrate a favorable fitness-diversity trade-off in small-population settings, with task-dependent effects, and position LLMs as coordinators of inter-island information flow in evolutionary robotics. Our code is publicly available at: https://github.com/emirb0/evogym-migration
Emir Buğra Şahin, Ege Demir, Bilgehan Altıntaş et al.· Proceedings of the Genetic a...· 0 citations
Results show that EMAS can turn experience from new samples into reusable updates to MAS topology and prompts, and is best or tied in six of eight model--benchmark settings.
Chao Fei, Qingyi Si, Kaihua Liang et al.· 0 citations
We investigate whether agentic artificial intelligence can automate parts of the process of designing genetic programming systems by introducing an agentic framework that identifies and implements parent selection algorithms using large language model (LLM) reasoning and retrieval-augmented generation. Using symbolic regression as a test bed, we first conduct an ablation study across four LLM types to evaluate the effects of agentic reasoning and retrieval on generated algorithm categories, validity, implementation similarity, and downstream performance. Results show that these components substantially influence the types of algorithms generated, but their downstream performance largely depends on the underlying LLM. The strongest configuration, the full agentic setup with 5 mini (5 mini--AR), consistently generated established $\epsilon$-lexicase implementations while maintaining competitive downstream performance. We then benchmark this configuration against fixed implementations of tournament selection and semi-dynamic MAD $\epsilon$-lexicase. Across six symbolic regression problems, 5 mini--AR performed similarly to $\epsilon$-lexicase while generally outperforming tournament selection. These findings demonstrate the potential of agentic AI to translate domain knowledge into generating executable components, providing a step toward automated configuration and design of evolutionary systems.
Jose Guadalupe Hernandez, Jui-Hsuan Chang, A. Saini et al.· 0 citations
Improving the task performance of Large Language Models (LLMs) is essential, yet scaling these models faces significant challenges such as diminishing returns and high costs. Multi-Agent Systems (MAS) offer a promising solution by distributing tasks among specialized agents to improve the overall task performance. This can reduce training costs at the expense of increased test time due to the discussion and decision-making process. The decision protocol is a critical component of MAS because it specifies how multiple agents collaborate to create a final solution. This thesis introduces the Multi-Agent LLM (MALLM) framework, which implements and evaluates various decision protocols, namely voting, consensus, and judge decision mechanisms, to simulate multi-agent discussions for conversational task solving. Unlike previous work that used a single decision protocol or tested them on limited datasets, this study systematically examines their impact on a diverse set of tasks, ranging from knowledge-based datasets (MMLU, MMLU-Pro, GPQA) and logic-based datasets (StrategyQA, MuSR, Math-lvl-5, SQuAD 2.0). The results indicate that consensus protocols excel in knowledge-intensive domains while voting and judge protocols are more effective for logic-based tasks. Increasing response diversity through independent solution generation improves decision quality, while changes in information access during the decision process have minimal impact.
Large language model coding agents increasingly perform open-ended data modeling and analysis. These agents are stochastic and adaptive, and therefore their autonomous model discovery behavior cannot be adequately characterized by a single benchmark run. In this work, we propose an experimental design and analysis framework for systematically evaluating this discovery process, quantifying its variability, and identifying important factors. The proposed framework treats these agents as stochastic model-discovery operators, which map task-specific discovery data and an optimization target to a fitted model. Specifically, we investigate two such operators, Codex and Claude Code, under controlled experimental factors including agent's reasoning effort, task, optimization metric, and composition of training data. For each agent-task-metric combination, regression models and inference are conducted for multiple responses such as output quality, dollar cost, wall-clock time, and process complexity. Furthermore, we develop a utility-aligned canonical decomposition to characterize the dominant direction of the reasoning-effort effect and to assess whether that direction aligns with a performance-cost utility direction. The proposed framework is demonstrated on a testbed of networked word-forming games with insightful findings on reasoning effort with respect to cost and process complexity.
Hao He, Xueying Liu, C. Kuhlman et al.· 0 citations