Product of Experts for Visual Generation

ICLR 2026 Conference Submission14067 Authors

Keywords: generative models, image generation, video generation

TL;DR: An inference-time sampling framework for image and video generation that compose knowledge from heterogeneous perceptual models.

Abstract: Modern neural models capture rich priors and have complementary knowledge over shared data domains, e.g., images and videos. Integrating diverse knowledge from multiple sources—including visual generative models, visual language models, and sources with human-crafted knowledge such as graphics engines and physics simulators remains under-explored. We propose a probabilistic framework that combines information from these heterogeneous models, where expert models jointly shape a product distribution over outputs. To sample from this product distribution for controllable image/video synthesis tasks, we introduce an annealed MCMC sampler in combination with SMC-style resampling to enable efficient inference-time model composition. Our framework empirically yields better controllability than monolithic methods and additionally provides flexible user interfaces for specifying visual generation goals.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 14067