This paper introduces a variant of UCBVI which does not maintain strictly optimistic estimates, and uses MOSS-style exploration bonuses whose logarithmic terms adapt to both visit counts and estimated variances, and develops time-uniform and variance-aware concentration bounds that may be of independent interest.
G. Zamir, Victor Boone, M. Zurek et al.· 0 citations
A prelimit coupling framework for analyzing nonsmooth stochastic approximation procedures is developed, which establishes Wasserstein convergence to a unique stationary distribution, characterize the limiting steady-state distribution as the step size vanishes, and shows that nonsmoothness can create a bias of order of...
It is derived that the asymptotic bias of nonsmooth contractive stochastic approximation (SA) with constant stepsize is proportional to the square root of the stepsize, which stands in sharp contrast to smooth SA.