Continuous diffusion language models generate all tokens in parallel, yet high-quality generation can still require hundreds of network evaluations (NFEs). We study how distributional distillation can reduce this cost by exploiting the student's probabilistic token outputs. Our unified formulation connects the student'...
Paul Le Van Kiem, Dario Shariatian, Umut Simsekli et al.· 0 citations
Diffusion models (DMs) are a state-of-the-art generative method to approximately sample from an unknown distribution. Their training and evaluation primarily rely on an Evidence Lower Bound (ELBO), which relates the Kullback-Leibler (KL) divergence of model samples to the score matching loss along the path, which serve...
Benjamin Dupuis, Tyler Farghly, Maxime Haddouche et al.· 0 citations
This work builds on a principled continuous-time approximation of Markov algorithms and introduces a new, exact entropy flow formula for such processes, and establishes novel connections to a well-studied family of modified logarithmic Sobolev inequalities.
Benjamin Dupuis, Maxime Haddouche, George Deligiannidis et al.· 2 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.