BERNOULLI FLOW MODELS

Hao Mo, Ajian Liu, Liying Yang, Shumin Yao, Yanyan Liang

Keywords: Binary Diffusion, Flow Matching, Bernoulli Distribution, Generative Models

Abstract: Diffusion-based generative modeling for data with Bernoulli distributions has broad potential applications, but it relies on carefully designed forward processes. Recently, flow matching-based methods have addressed this issue. However, when these methods are naively applied to the Bernoulli distribution, their dependence on predicting the instantaneous velocity field during sampling can introduce invalid Bernoulli parameters, leading to model collapse. To address this challenge, we introduce **Bernoulli Flow Models (BFM)**, a novel generative framework that fuses flow matching with vanilla binary diffusion. BFM ensures valid Bernoulli parameters throughout the sampling process by deriving a one-step forward transition kernel and a closed-form, normalized posterior based on the pre-defined flow-matching probability path in the Bernoulli parameter space. As a result, BFM simplifies the training process of current binary diffusion models and can be easily integrated into existing architectures with minimal modification. We empirically validate the generative performance of BFM on high-dimensional binary manifolds, including Ising model simulations, both unconditional and conditional image generation. Experiments show that our model achieves comparable performance to both continuous and discrete space generative models.

Primary Area: generative models

Submission Number: 2554