Complex Preferences for Different Convergent Priors in Discrete Graph Diffusion

Alex M Tseng, Nathaniel Lee Diamant, Tommaso Biancalani, Gabriele Scalia

Keywords: diffusion models, graph generation, prior distribution, discrete diffusion

TL;DR: The performance of discrete diffusion models depends on the diffusion kernel, but the best kernel is not the most obvious one

Abstract: Diffusion models have achieved state-of-the-art performance in generating many different kinds of data, including images, text, and videos. Despite their success, there has been limited research on how the underlying diffusion process and the final convergent prior can affect generative performance; this research has also been limited to continuous data types and a score-based diffusion framework. To fill this gap, we explore how different _discrete_ diffusion kernels (which converge to different prior distributions) affect the performance of diffusion models for graphs. To this end, we developed a novel formulation of a _family_ of discrete diffusion kernels which are easily adjustable to converge to different Bernoulli priors, and we study the effect of these different kernels on generative performance. We show that the quality of generated graphs is sensitive to the prior used, and that the optimal choice cannot be explained by obvious statistics or metrics, which challenges the intuitions which previous works have suggested.

Submission Number: 12