A spectral gap for Metropolis-adjusted Langevin algorithm with a uniformly randomized step size

Abstract

Let $\pi(\mathrm{d} x)\propto e^{-U(x)}\, \mathrm{d} x$ on $\mathbb{R}^d$, where $U$ is continuously differentiable and $m$-strongly convex with a globally $L$-Lipschitz gradient, $0<m\leq L<\infty$, and $\kappa=L/m$. Fixed-step Metropolis-adjusted Langevin algorithm (MALA) has known warm-start mixing-time upper bounds of order $\kappa\sqrt d$, up to logarithmic factors. A spectral-gap lower bound at the corresponding scale $(\kappa \sqrt{d})^{-1}$ would give an upper bound of order $\kappa \sqrt{d}$ on the Monte Carlo asymptotic variance relative to independent sampling, uniformly over all square-integrable functions. However, when $\kappa$ is bounded away from~1, the best fixed-step spectral gap that can be guaranteed uniformly over this target class is at most of order $\max\{\log(\kappa d)/(\kappa d), e^{-cd}\}$ for some positive universal constant~$c$. We show that uniform randomization of the step size improves this worst-case guarantee. At each iteration, the algorithm draws $h\sim\operatorname{Unif}(0,H)$ and performs one ordinary MALA transition. Choosing $H$ of order $[L\sqrt{d(1+\log d+\log\kappa)}]^{-1}$ yields a right spectral-gap lower bound of order \[ \frac{1}{\kappa\sqrt{d(1+\log d+\log\kappa)}}, \] uniformly over the target class. Thus, given $\kappa>1$, for all square-integrable functions, the ratio of the Monte Carlo asymptotic variance relative to independence sampling has an upper bound of order $\sqrt{d \log d}$. This contrasts with fixed-MALA, where the ratio can be as bad as $d/\log d$ in thew worst-case scenario. The spectral gap also gives geometric convergence of the lazy kernel from every initial density in $L^2(\pi)$, central limit theorems, and nonstationary mean-square error bounds. This work was developed with substantial assistance from ChatGPT.