Skip to content
Preprint

Bayesian Symbolic Regression with Entropic Reinforcement Learning

Aug 2026 · 2 citations · 48 references
Computer Science

TL;DR

ERRLESS (Entropy-Regularized Reinforcement Learning for Expression Structure Sampling), a scalable approach for sampling from the posterior distribution over expressions given data using maximum-entropy reinforcement learning, is proposed and achieves competitive results on the Feynman benchmark.

Abstract

Symbolic regression is the problem of finding an algebraic expression describing a stochastic dependence of a target variable on a set of inputs. Unlike forms of regression that fit parameters assuming a fixed model structure, symbolic regression is a search problem over the space of expressions, represented, for example, as abstract syntax trees using a library of operators. Symbolic regression is typically used in settings with limited, noisy data in the natural sciences. However, searching for a single best-fitting expression fails to capture the epistemic uncertainty about the expression, which motivates a Bayesian perspective that enables uncertainty quantification and specification of natural priors to constrain the search space. In this work, we propose ERRLESS (Entropy-Regularized Reinforcement Learning for Expression Structure Sampling), a scalable approach for sampling from the posterior distribution over expressions given data using maximum-entropy reinforcement learning. ERRLESS learns a neural policy that constructs expressions sequentially by building up their abstract syntax trees. At convergence, the policy samples expressions from the posterior. At test time, expressions can be sampled by rollouts of this policy. We demonstrate that ERRLESS achieves competitive results on the Feynman benchmark while producing short and interpretable expressions. Additionally, we demonstrate that the mean of the posterior predictive approximated by ERRLESS achieves a high coefficient of determination ($R^2$) compared to an SMC baseline, highlighting the benefits of the Bayesian perspective in symbolic regression.

View source

Similar papers

Open access Oct 2025

Iterated agent for symbolic regression

Symbolic regression (SR), the automated discovery of mathematical expressions from data, is a cornerstone of scientific inquiry. However, it is often hindered by the combinatorial explosion of the search space and a tendency to overfit. Popular methods, rooted in genetic programming, explore this space syntactically, o...

Zhuo-Yang Song, Ze-Yu Cai, Shu-Tao Zhang et al. · 5 citations
Preprint Oct 2026

A Latent Space Optimization Approach for Symbolic Discovery of Dynamical Models

In this paper, we propose a hybrid latent-space Bayesian Optimization (BO) and global optimization framework for solving symbolic regression tasks to discover dynamical models from data. Symbolic regression can discover equations from data without fixing the functional form of the expression a priori. The proposed fram...

Tong Liu, Ilias Mitrai · 0 citations
#machine learning Preprint Aug 2026

Reinforcement Learning for Symbolic Equation Solving

A reinforcement-learning agent that solves symbolic equations step by step, covering both nonlinear closed equations and a controlled class of restricted-open families requiring a change of variables (CoV) such as completing the square is presented.

Kevin P. O. Keeffe · 0 citations
Preprint Oct 2026

Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs

In this paper we propose a method for estimating conditional densities from an iid sample using a language-model agent. The agent writes the estimator as a short program that, for each covariate value, returns the estimated conditional distribution of the response. The program is built from a library of probability dis...

Lorenzo Tedesco · 0 citations
#machine learning Preprint Oct 2026

Reinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers

Reinforcement learning (RL) has become a standard tool for post-training language models on reasoning tasks, where the policy is updated by reward feedback while exploring the space of responses. Despite its empirical success, theoretical understanding of RL post-training remains limited, in particular of why on-policy...

Naoki Nishikawa, Taiji Suzuki · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.