Autoregressive Diffusion Models with non-Uniform Generation Order

Filip Ekström Kelvinius, Fredrik Lindsten

Keywords: discrete diffusion models, autoregressive diffusion models, molecules

TL;DR: We consider limiting the distribution of generation orders in autoregressive diffusion models to improve generative performance

Abstract: Diffusion models for discrete data have gained increasing interest lately. Recent methods use an autoregressive formulation, but where the generation order is random. In this work, we turn our attention to the distribution of the generation order. Instead of using a uniform distribution over all possible orders, we propose to limit the distribution for facilitating learning the generative model, while still keeping the benefit of not having to rely on a fixed generation order. We empirically show how limiting the generation order can improve the generative performance in generating molecular graphs.

Submission Number: 47