CMAD: Cooperative Multi-Agent Diffusion via Stochastic Optimal Control

Riccardo Barbano, Alexander Denker, Runchang Li, Zeljko Kereta, Francisco Vargas

Keywords: Diffusion models, Compositional Generation, Stochastic Optimal Control

TL;DR: We recast compositional generation with multiple pre-trained diffusion models as a cooperative stochastic optimal control problem, steering their joint trajectories toward a shared objective without assuming explicit target densities.

Abstract: Continuous-time generative models have achieved remarkable success in image restoration and synthesis. However, controlling the composition of multiple pre-trained models remains an open challenge. Current approaches largely treat composition as an algebraic combination of probability densities, such as via products or mixtures of experts. This perspective assumes the target distribution is known explicitly, which is almost never the case. In this work, we propose a different paradigm that formulates compositional generation as a cooperative Stochastic Optimal Control problem. Rather than combining probability densities, we treat pre-trained diffusion models as interacting agents whose diffusion trajectories are jointly steered, via optimal control, toward a shared objective defined on their aggregate output. We validate our framework on conditional MNIST generation and compare it against a naïve inference-time DPS-style baseline that replaces learned cooperative control with per-step gradient guidance.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 27