EvoPrompt-ELO: automated large-scale prompt generation via ELO-guided evolutionary optimization

Abstract

Abstract High construction costs and unstable evaluation signals remain challenges in cross-task prompt engineering. Existing methods struggle with diversity maintenance at scale. To address these issues, this paper proposes EvoPrompt-ELO, an automated prompt-generation framework based on evolutionary strategies and ELO Rating System (ELO)-based dynamic adversarial evaluation. The framework employs Large Language Models (LLMs) as mutation operators to generate candidate prompts, incorporates Sentence-BERT (Sentence Bidirectional Encoder Representations from Transformers) for diversity constraints, and utilizes an LLM-as-a-Judge (treating an LLM as an evaluator) pairwise comparison mechanism combined with ELO scoring to achieve dynamic quality assessment and survival-of-the-fittest selection. Experiments show that the generated prompts achieve a 61.08 % win rate against the seed prompts in LLM-based blind testing, achieving broad semantic-mode coverage – doubling the number of distinct semantic clusters over the seed set (from 10 to 20) – while quality-guided selection concentrates the corpus on high-value regions, and successfully construct a task-agnostic, high-quality general-purpose prompt corpus Prompt-8K (8,261 prompts) efficiently. Furthermore, the ELO-based dynamic evaluation mechanism provides a trustworthy and scalable quality assessment framework that aligns with principles of adaptive trust management in evolving AI systems, enabling progressive refinement of knowledge repositories through principled adversarial evaluation and rating aggregation.