Comparing Genetic Algorithms and LLMs for Research-Hypothesis Generation: Mainstream Convergence of LLMs, Wider Hypothesis Exploration by Combinatorial Search, and the Limits of Literature-Distance Novelty Metrics

Abstract

Large language model (LLM)-based creative AI systems tend to produce plausible rather than novel research ideas. This study compares two ways of generating research hypotheses: a genetic algorithm (GA) that recombines Method, Problem, and Goal elements under a fitness function rewarding distance from the literature and literature support for each link, and free proposals by an LLM. Papers and hypotheses share one embedding space built from 2856 arXiv papers. Over 30 runs with budget-matched baselines and outcome measures on held-out and later-published papers, the selected hypotheses were more distant from the unseen literature than unsearched ones, but the GA matched rather than outperformed random search and hill climbing. Across two LLM families, two prompts, and two temperatures, and even when choosing from the GA’s concept vocabulary, LLM-proposed hypotheses were less diverse and closer to existing literature, whereas combinatorial search explored more widely. On 9755 ICLR submissions, the literature-distance novelty measure was practically unrelated to reviewers’ judgments (Spearman ρ ≤ 0.06), so it cannot establish originality. In controlled manuscript generation, LLM judges proved unreliable, and their order-consistent verdicts more often favored manuscripts built on LLM-proposed hypotheses, which blinded human raters tended to rank as least useful. Code and data are openly released.