Differentiable Genetic Programming for High-dimensional Symbolic Regression.

Abstract

Symbolic Regression (SR) aims to uncover hidden relationships within data by generating mathematical expressions, offering a pathway toward interpretable machine learning. Genetic Programming (GP) has traditionally dominated SR due to its flexibility in evolving expression trees. However, as the dimensionality of SR problems increases, the stochastic nature of GP leads to inefficiencies and poor scalability, particularly in high-dimensional real-world tasks. To address this, we propose a novel Differentiable Genetic Programming method, termed DGP, which enables gradient-based optimization for high-dimensional SR for the first time. Specifically, DGP proposes a differentiable symbolic tree that relaxes the discrete structure of GP trees to a continuous representation, thus allowing for efficient gradient based optimization. To bridge the gap between this continuous space and valid symbolic expressions, DGP designs a sampling strategy that ensures structural validity and introduces a di versification mechanism to escape local optima and enhance global search. Extensive experiments on complex regression and expression recovery high-dimensional benchmarks demonstrate that DGP consistently outperforms state-of-the-art GP and neural network-based SR methods, achieving superior accuracy and expression recovery across a range of noise levels. For example, DGP achieves the best R2 score of 0.32 on the DLBCL dataset with 7,400 dimensions, whereas most other methods yield nearly zero or even negative R2 scores. The results demonstrate that DGP offers a reliable and robust solution for tackling complex high-dimensional SR problems, positioning it as a compelling tool for interpretable machine learning. The source code is available at https://github.com/songxt3/DGP.