Fixed-Point Distillation of Flow Matching Models

Paul Jeha, Heiner Kremer, Fabian Falck, Jannes Gladrow

Keywords: Flow Matching, Deep Equilibrium Models, Distillation, Fixed-Point Computation

TL;DR: Distilling Flow Matching models in fixed points.

Abstract: Flow matching models generate high-quality samples but require many sequential network evaluations at inference time. We propose a distillation method that compresses multiple explicit Euler steps of a pre-trained flow matching model into a single implicit (fixed-point) step. We train a student model so that solving one fixed-point equation reproduces the teacher's multi-step trajectory over the same interval. Emerging analog hardware motivates this formulation: such devices solve fixed-point problems orders of magnitude more efficiently than conventional hardware. On a 2D distribution, a single implicit step of the distilled student matches the teacher's sample quality at 10~explicit steps. Even students with fewer layers than the teacher achieve this, suggesting that fixed-point distillation compresses both computation and model capacity.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 86