Flow Matching in the Low-Noise Regime: Pathologies and a Contrastive Remedy

Weili Zeng, Yichao Yan

Keywords: Flow Matching, low-noise pathology, representation learning, Local Contrastive Flow

Abstract: Flow matching has recently emerged as a powerful alternative to diffusion models, providing a continuous-time formulation for generative modeling and representation learning. Recent progress in generative visual foundation models suggests that flow-matching models could serve not only as generators but also as unified backbones for downstream discriminative tasks. This raises a natural question: can flow matching reliably learn high-quality representations directly from clean data? Yet, we show that this framework suffers from a fundamental instability in the low-noise regime. As noise levels approach zero, arbitrarily small perturbations in the input can induce large variations in the velocity target, causing the condition number of the learning problem to diverge. This ill-conditioning not only slows optimization but also forces the encoder to reallocate its limited Jacobian capacity toward noise directions, thereby degrading semantic representations. We provide the first theoretical analysis of this phenomenon, which we term the \textbf{low-noise pathology}, establishing its intrinsic link to the structure of the flow-matching objective. Our analysis reveals that this pathology creates a major bottleneck for using flow matching as a viable representation-learning framework, limiting its suitability as a discriminative visual backbone. Building on these insights, we propose \textbf{Local Contrastive Flow} (LCF), a hybrid training protocol that replaces direct velocity regression with contrastive feature alignment at small noise levels, while retaining standard flow matching at moderate and high noise. Empirically, LCF not only improves convergence speed but also stabilizes representation quality. These results suggest that addressing the low-noise pathology is essential for advancing flow-matching models toward unified generative–discriminative visual foundation models.

Submission Number: 6