Skip to content
Preprint

Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions

Oct 2026 · 0 citations
Mathematics Computer Science

TL;DR

Under stability and regularity conditions on value function gradients, Bellman residual derivatives, and fitted critics, it is proved an $\mathcal{O}(\log K)$ regret bound with constants uniform in $\delta$.

Abstract

High-frequency control problems are often studied through time discretization, but refining the timestep $\delta$ increases the discrete horizon $H=T/\delta$ and can make standard finite-horizon regret guarantees deteriorate even when the physical horizon $T$ is fixed. We study episodic value-based reinforcement learning for Euler discretizations of controlled diffusions. The performance is measured as the cumulative regret over $K$ episodes relative to the optimal policy of the corresponding Euler model. Under stability and regularity conditions on value function gradients, Bellman residual derivatives, and fitted critics, we prove an $\mathcal{O}(\log K)$ regret bound with constants uniform in $\delta$. For dynamics with drift linear in unknown parameters and unknown constant diffusion covariance, we verify localized versions of these conditions for a constrained, model-based controlled-diffusion fitted-$Q$ algorithm (CD-FQ) and obtain high-probability $\mathcal{O}(\log K\log\log K)$ regret for fixed and finite $T$, again with constants uniform in $\delta$.

View source

Similar papers

#machine learning Preprint Oct 2026

Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed Features

We study the last iterate of stochastic gradient descent (SGD) on streaming least squares with discretely distributed features (Slock's model), which decouples the dynamics across eigendirections. Under power-law source and capacity conditions, we derive the exact finite-time risk and asymptotic equivalents of constant...

Noé Colomban, F. Bach, Fabian Schaipp · 0 citations

Sharp Critical Minimax Laws and No-Learning Thresholds in Continuous-Time Adaptive Control

We study episodic continuous-time control with an unknown vector control gain, scalar state, quadratic action cost, and smooth convex terminal cost. In the scalar Gaussian experiment, let $\Delta(H)$ denote the minimax improvement over zero control and set $\delta=\sqrt2H^2-1$. We prove the critical law $$ \Delta(H)\as...

Chen Jia · 0 citations
#machine learning Preprint Oct 2026

Reliable Scale-Free Bandits via Empirical Likelihood

The exact exponent of linear regret deviations on two-armed Gaussian instances with common variance is determined and a matching lower bound among policies satisfying the same expected-regret budget on every such Gaussian instance is established.

Pratham Gupta, Alhad Sethi, Shubhada Agrawal · 1 citation
#machine learning Preprint Oct 2026

Taylor Representations for Model-Free RL in Networked MDPs

A scalable model-free actor--critic algorithm is proposed, establishing finite-sample critic and near-stationarity guarantees for a linear LSTD critic and a more expressive neural TD parameterization is introduced.

Salah Chikhi, A. Chaoui, A. Ozdaglar et al. · 0 citations
Sep 2026

Limiting-Kernel Q(λ): Bridging Short and Long Horizons

It is proved that, under aperiodicity and in the near-on-policy regime, the operator underlying LKQL improves the policy evaluation convergence rate over its truncated counterpart for sufficiently large $n$, and that LKQL itself converges almost surely to the optimal values in finite Markov decision processes (MDPs) un...

Tolga Ok, A. S. Kolarijani, Peyman Mohajerin Esfahani et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.