Feb 2024· Neural Information Processing Systems· 52 citations· ⚡ 7 influential· 93 references
Computer ScienceMathematics
TL;DR
This work benchmarks several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks), and proposes a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer.
Abstract
We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.
The analysis from independent random search to a broader family of local search operators, and it is proved that when sensitivities vary across instances, no adaptive policy can avoid worst-case regret that grows linearly in the trajectory length.
Yuan Cao, Yi-Fu Tang, Hang-Qi Li et al.· 0 citations
A noisy-space action-value (Q-)function that assigns values to diffusion latents through the distribution of executed actions induced by the denoising process is introduced, which admits a precise semantic interpretation and derives a noisy-space policy gradient (NSPG) that optimizes noisy latents using only clean acti...
Mahmoud Selim, Cristina Cipriani, K. H. Johansson· 1 citation
A conditioning mechanism for diffusion models based on multi-speed joint diffusion of the target and the condition learns an unconditional joint score network and enforces conditioning at inference via a plug-in correction term, and derives explicit conditional reverse-time SDEs and approximate probability-flow ODEs.
Libo Chen, Souvik Ghosh, Teo Deveney et al.· 0 citations
The empirical gap between these method families is identified as a variance-reduction effect rather than a difference in RL principle, and a multi-sample KDE value-gradient estimator that reuses rollout groups, together with scale-bounded weight families that retain stable existing recipes while excluding singular ones...
Yi-Xian Xu, Yuanrui Zhang, Sheng-Jie Luo et al.· 3 citations
POGP is introduced, a framework that learns a prefix value function at every intermediate denoising step through a Bellman-style recursion over the denoising chain, and indicates that supervising intermediate denoising steps is useful not only for adaptive early stopping, but also as an auxiliary objective that improve...
Rohit Kumar Salla, M. Saravanan, Simon Stepputtis· 2 citations
A framework for designing the reference process that achieves tractable time-dependent conditional distributions and then construct the reference process realizing them as its marginals is proposed, revealing that score matching is not fundamental to diffusion-model training but instead emerges naturally through revers...
Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.
