Diffusion Generative Modeling on Lie Group Representations

Marco Bertolini, Tuan Le, Djork-Arné Clevert

Keywords: generative modeling, diffusion models, lie groups, representation theory

TL;DR: We introduce Diffusion-Based Generative Modeling on the (flat) Lie group representation space rather than on the (curved) Lie group

Abstract:

We introduce a novel class of score-based diffusion processes that operate directly in the representation space of Lie groups.
Leveraging the framework of Generalized Score Matching, we derive a class of Langevin dynamics that decomposes as a direct sum of Lie algebra representations,
enabling the modeling of any target distribution on any (non-Abelian) Lie group.
Standard score-matching emerges as a special case of our framework when the Lie group is the translation group.
We prove that our generalized generative processes arise as solutions to a new class of paired stochastic differential equations (SDEs), introduced here for the first time.
We validate our approach through experiments on diverse data types, demonstrating its effectiveness in real-world applications such as $\text{SO}(3)$-guided molecular conformer generation and modeling ligand-specific global $\text{SE}(3)$ transformations for molecular docking, showing improvement in comparison to Riemannian diffusion on the group itself.
We show that an appropriate choice of Lie group enhances learning efficiency by reducing the effective dimensionality of the trajectory space and enables the modeling of transitions between complex data distributions.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 9541