Flow Matching Generalizes Through Discretization Bias

ICLR 2026 Conference Submission20209 Authors

Keywords: Flow matching, generative model, nonparametric regression, kernel density estimation, implicit bias, ODE discretization

Abstract: Flow models exhibit an extraordinary ability to generalize, generating realistic samples far beyond the training data. This phenomenon lacks a simple explanation. We argue that the key mechanism is not the accurate solution of a continuous-time ODE, but rather the error introduced by its discretization. To isolate this effect within the flow matching framework, we introduce the \emph{Empirical Velocity Field (EVF)}, a non-parametric estimator of the \emph{conditional velocity field} derived by replacing the target distribution with its empirical measure. The exact ODE flow driven by the EVF turns out to be uninteresting, yielding a kernel density estimate that collapses onto the training data. However, its discretization is remarkably powerful. We show that even a single Euler step induces a projection-like effect, concentrating samples on the underlying data manifold and creating diverse, high-quality samples. We support this with extensive empirical evidence and provide a theoretical analysis of the one-step estimator that quantifies this projection, offering a rigorous foundation for how discretization generates structured samples. Our findings argue that the generative success of flow matching is fundamentally driven by the implicit bias of numerical ODE solvers.

Primary Area: generative models

Submission Number: 20209