Convergence of Denoising Diffusion Probabilistic Models: A R\'{e}nyi Divergence Perspective

Abstract

Denoising diffusion probabilistic models (DDPMs) have emerged as a powerful framework for generative modeling by transforming standard Gaussian inputs into outputs that approximate a target distribution. While DDPMs have been proved to converge in total variation, KL divergence, and Wasserstein distance, these discrepancy measures share a fundamental limitation of being insensitive to distributional mismatch in low-probability regions. To address this, we adopt the stronger R\'enyi divergence as the discrepancy measure. Leveraging a Girsanov-type change-of-measure argument, we establish the first R\'enyi characterization of DDPMs to the best of our knowledge, showing that the total error decomposes into a mixing error, a discretization error, and a score estimation error. We assume the target distribution is sub-Gaussian and characterize the score error via the cumulant generating function, relaxing both the log-Sobolev-inequality and $L^\infty$-score-error conditions used in prior R\'enyi divergence analyses of inexact Langevin Monte Carlo, a score-based sampling algorithm closely related to DDPM. Our results also establish an iteration complexity for achieving $\epsilon$-accuracy in $\alpha$-R\'enyi divergence that is asymptotically proportional to $\alpha^3/\epsilon$. Finally, by extending our analysis, we establish the first R\'enyi divergence convergence guarantee for critically damped DDPMs, a kinetic variant governed by critically damped Langevin dynamics. Similar to existing characterizations of critically damped DDPMs, this result shares the structural scaling of standard DDPMs, differing by at most a logarithmic factor in the mixing error exponent, and supports the conjecture that critically damped DDPMs do not yield a theoretical improvement.