Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy

Hancheng Ye, Jiakang Yuan, Renqiu Xia, Xiangchao Yan, Tao Chen, Junchi Yan, Botian Shi, Bo Zhang

Abstract:

Diffusion models have recently achieved great success in the synthesis of highquality images and videos. However, the existing denoising techniques in diffusion
models are commonly based on step-by-step noise predictions, which suffers from
high computation cost, resulting in a prohibitive latency for interactive applications.
In this paper, we propose AdaptiveDiffusion to relieve this bottleneck by adaptively
reducing the noise prediction steps during the denoising process. Our method
considers the potential of skipping as many noise prediction steps as possible
while keeping the final denoised results identical to the original full-step ones.
Specifically, the skipping strategy is guided by the third-order latent difference
that indicates the stability between timesteps during the denoising process, which
benefits the reusing of previous noise prediction results. Extensive experiments on
image and video diffusion models demonstrate that our method can significantly
speed up the denoising process while generating identical results to the original
process, achieving up to an average 2 ∼ 5× speedup without quality degradation.