Filling Deterministic Approximate Gap in Denoising Diffusion Probabilistic Models with Energy

Diffusion probabilistic models (DPMs) have shown the best generative quality and admit impressive likelihood meanwhile. However, its variants take very long generative processes during sampling, because of the small noise scale assumption, say, sets the variance of the forward diffusion kernel to be very small. Under which, the generative denoising processes are of tractable Gaussian form. To shorten the generative process, we explore the non-Gaussian reverse kernel under the large noise scale assumption and analysis the deterministic approximate gap of Gaussian denoising process. To fill the gap, we enhance the standard DPM with conditional energy and introduce variantional sampler to construct tractable generative process. The new probabilistic model, termed denoising diffusion energy-based model (DDEBM), are trained with two stage adversarial optimization. Experiments show our models demonstrate competitive generative performance with only four steps.