Alleviating Suboptimality of Flow Maps with Improved Self-Distillation Guidance

YoungJoong Kim, Duhoe Kim, Woosung Kim, Jaesik Park

Keywords: Consistency Model, Flow Map Model, Generative Model

Abstract: Consistency-based approaches have been proposed for fast generative modeling, achieving competitive results compared to diffusion and flow matching models. However, these methods often rely on heuristics to mitigate training instability, which in turn limits reproducibility and scalability. To address this limitation, we propose the generalized flow map framework, unifying recent consistency-based methods under a common perspective. Within this framework, we investigate the suboptimality of existing approaches and identify two key factors for reproducibility: time-condition relaxation and marginal velocity guidance. To incorporate these, we leverage self-distillation to guide consistency models along the marginal velocity. We further propose improved Self-Distillation (iSD) by exploring the design space of flow maps, thereby reducing reliance on heuristics. Our formulation naturally extends to classifier-free guidance, achieving four-step generation with an FID of 11.06 on ImageNet $256\times256$. iSD shows qualitatively comparable results to prior few-step generative models, providing a theoretical and empirical foundation for reproducible consistency training.

Primary Area: generative models

Submission Number: 8749