Yang Peng

Sharp Statistical Rates for Asynchronous TD Learning with Markovian Data

Online Inference for Quantile Temporal Difference Learning in Distributional Reinforcement Learning

This paper establishes functional central limit theorems for both synchronous and asynchronous QTD, which show that the averaged iterates of QTD converge weakly to a rescaled Brownian motion, and provides online inference methods.

A Finite-Sample Analysis of Quantile Temporal-Difference Learning

A global high-probability last-iterate guarantee for synchronous tabular QTD under general positive, nonincreasing step-size sequences and arbitrary initialization in the natural parameter range is established.

A Finite Sample Analysis for Quantile Temporal Difference Learning in Distributional Reinforcement Learning

We establish a global finite-sample guarantee for synchronous quantile temporal-difference learning (QTD) in tabular distributional reinforcement learning. The proof separates two stability mechanisms. A global comparison argument, based on the order monotonicity of reward cumulative distribution functions and the $W_\...