SparseDiff: Sparse Discrete Diffusion for Scalable Graph Generation
Abstract: Graph generative models encounter significant scaling challenges due to the need to predict the presence or type of edges for every node pair, resulting in quadratic complexity. While some models attempt to support large graph generation, they often impose restrictive assumptions, such as enforcing cluster or hierarchical structures, which can limit generalizability and result in unstable generation quality across various graph types. To address this, we introduce SparseDiff, a novel diffusion framework that leverages the inherent sparsity in large graphs - a highly relaxed assumption that enables efficient sparse modeling without sacrificing generation quality for different datasets. Based on that, SparseDiff reduces the complexity of the three core components in graph diffusion models. It first introduces an efficient noising process that samples sparse noisy graphs with linear complexity relative to the number of edges. During training, SparseDiff combines query edge-based random attention with edge-based graph attention mechanisms, matching graph transformer performance while reducing space complexity. Finally, for inference, at each denoising step, SparseDiff maintains sparsity by incrementally reconstructing the adjacency matrix via adding edge subsets defined by query edges. SparseDiff achieves state-of-the-art results on both small and large datasets, showing its robustness across varying graph sizes and its scalability. Additionally, it ensures faster convergence for large graphs, achieving a fourfold speedup on the large-scale Ego dataset compared to dense models. SparseDiff's efficiency, combined with its effective control over space complexity, positions it as a powerful solution for scaling applications involving large graphs.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Alessandro_Sperduti1
Submission Number: 3614
Loading