Can LLM’s Pick Up Where GP Left off? Investigating LLM Mathematical Expression Simplification Capabilities

Abstract

Symbolic regression seeks to discover interpretable mathematical expressions that describe the relationship between variables in a dataset. Genetic Programming remains the dominant approach for SR, however it is well established that GP is prone to producing complex expressions due to bloat. Post-hoc simplification of GP-derived mathematical expressions is therefore a practically important objective. In this work, we investigate the use of Large Language Models as a post-hoc simplification layer for GP-derived mathematical expressions, examining two prompting strategies; Few-Shot Prompting and In-Context Learning across two contemporary LLMs: GPT-5.4-mini and Nemotron-3-Super. We evaluate our approach across 91 datasets drawn from the Penn Machine Learning Benchmark, assessing simplification performance in terms of expression size, mean squared error (MSE), and coefficient of determination (R2). Our results demonstrate that LLMs are capable of reducing expression complexity without catastrophic accuracy loss in the majority of cases. All LLM approaches produce significantly shorter expressions than the GP baseline, with node count reductions ranging from approximately 1 to 7 nodes on average. The accuracy cost is statistically significant but small in practice, with rank-biserial effect sizes between 0.01 and 0.16. These findings suggest that LLMs represent a promising and complementary component within larger SR pipelines.