Convergence of Consistency Model with Multistep Sampling under General Data Assumptions

Yiding Chen, Yiyi Zhang, Owen Oertell, Wen Sun

Abstract: Diffusion models accomplish remarkable success in data generation tasks across various domains. However, the iterative sampling process is computationally expensive. Consistency models are proposed to learn consistency functions to map from noise to data directly, which allows one-step fast data generation and multistep sampling to improve sample quality. In this paper, we study the convergence of consistency models when the self-consistency property holds approximately under the training distribution. Our analysis requires only mild data assumption and applies to a family of forward processes. When the target data distribution has bounded support or has tails that decay sufficiently fast, we show that the samples generated by the consistency model are close to the target distribution in Wasserstein distance; when the target distribution satisfies some smoothness assumption, we show that with an additional perturbation step for smoothing, the generated samples are close to the target distribution in total variation distance. We provide two case studies with commonly chosen forward processes to demonstrate the benefit of multistep sampling.

Lay Summary:

Consistency models can generate realistic images or data very quickly, sometimes in just one step. This speed makes them appealing for real-world use, but it also raises two important questions: Why do these fast methods work? And how can we improve their results even further?

In this paper, we study consistency models from a theoretical perspective. We build a rigorous mathematical framework to explain their behavior. Our analysis shows that when these models are trained well, the data they generate closely matches the true underlying data distribution.

We also study a technique called multistep sampling, which improves the model’s output without retraining it. Instead of generating a sample in one step, the model takes several carefully designed steps. This adds some computation but can lead to much higher-quality results. However, more steps aren’t always better — adding too many can actually hurt performance. We explain when and why this happens and show how to choose the right balance.

Together, our findings help explain the success of consistency models and offer practical guidance for making them even more effective in fast data generation tasks.

Primary Area: Theory

Keywords: Consistency models, diffusion models, learning theory

Submission Number: 13478