A Guide to Training Consistency Models

Wenze Liu, Xintao Wang, Pengfei Wan, Xiangyu Yue

Keywords: diffusion models, consistency models

Abstract: While the theoretical foundations of consistency models are well-understood, their practical implementation is often hindered by complex and entangled training pipelines. The interplay between critical components is not always transparent, making systematic improvement difficult. To address this, we propose a practical training playbook for consistency models. Our approach begins with a na\"ive baseline and proceeds to deconstruct the training process, isolating and examining the impact of key modules: time step discretization, time conditioning, loss weighting, time sampling strategies, the auxiliary task with variable upper limit, and distribution-level losses. This modular analysis provides a clear view of how each element contributes to the overall performance. Following this guide, we demonstrate the ability to build models that achieve both state-of-the-art results and substantially faster convergence. Notably, for the first time, consistency models trained from scratch now surpass the leading EDM diffusion model on CIFAR-10 under the same network architecture. They achieve a remarkable $1$-step FID of $2.53$ and a $2$-step FID of $1.92$.

Primary Area: generative models

Submission Number: 951