Fuzzy Logic Composition of Diffusion Models

Peter Blohm, Vikas K Garg

Keywords: Diffusion, Score Composition, Mixture of Experts, Product of Experts, DeMorgans Law, Fuzzy Logic

TL;DR: We propose a set of score composition operators for diffusion models and showcase that they improve combinatorial reasoning capabilities of ensembles.

Abstract:

We investigate the composition of diffusion models as combinatorial operations on their underlying probability distributions.
Theory from fuzzy logic allows us to define a class of score-composition operators that follow DeMorgan's law.
We reframe Mixture of Experts in this context to obtain the class of *Dombi* operators, which provide a parametrized tradeoff between smoothness and precise combinatorial reasoning.
We show that score-composition with Products of Experts (PoE) can lead to unstable behaviour and implicit temperature rescaling.
In preliminary experiments on SAT problems in continuous settings, we show that Dombi composition enables combinatorial reasoning and does not suffer from PoE's bias.

Submission Number: 35