Nonsmooth Learning Algorithms Behave Differently from Smooth Ones
Many learning algorithms, including Q-learning, update their estimates through noisy recursive rules. When these updates use a constant step size and involve nonsmooth operators, their long-run behavior can be difficult to characterize: the iterates do...
For constant-stepsize stochastic approximation (SA), the iterates converge in distribution to a stationary law that depends on the stepsize $\alpha.$ Steady-state convergence (SSC) concerns the limit of the scaled stationary distribution as $\alpha \downarrow 0.$ Existing SSC theory requires i.i.d. or additive noise an...
We develop Gaussian approximation bounds in higher-order Wasserstein distance $W_p$, $p\geq2$, for sums of multivariate martingale differences generated by a uniformly ergodic Markov chain. Under an $L^{(2+\eta)p}$-moment condition with $\eta>0$, we establish the explicit bound $$ O\left( p^3 \|A\|_4^2 + pd^{1/4}\|A\|_...
Yi-Xuan Zhang, Qiao-Min Xie· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.