ParaSolver-Turbo: Accelerating Parallel Diffusion Integrator via Intrinsic Partially Linear Structure

Jianrong Lu, Zhiyu Zhu, Hui LIU, Junhui Hou

Keywords: Diffusion Models

TL;DR: A parallel sampling algorithm for accelerating inference of diffusion models

Abstract:

This paper explores the challenge of accelerating the sequential inference process of Diffusion Probabilistic Models (DPMs). We tackle this critical issue from a dynamic system perspective, in which the inherent sequential nature is transformed into a parallel sampling process.
Specifically, we first reveal that the sequential integral solver of the diffusion model can be approximated by a full linear solver, enabling efficient computation for parallel integral solvers of DPMs. Based on such a linear formulation, we then introduce a unified framework that reformulates the original nonlinear sequential integral process of diffusion model as a system of partial linear equations. Moreover, we further develop an immediate update strategy to solve the system. In addition, we prove that (1) the system admits a unique root corresponding precisely to the trajectory of the sequential integral solver; (2) solving the system guarantees convergence to the trajectory of sequential integral solvers in equal or fewer iterations.
Building on these insights, we present \textit{ParaSolver-Turbo}, a partial linear parallel integral solver to accelerate a broad class of sequential and parallel sampling methods such as DDPM and ParaSolver.
Extensive experiments validate that ParaSolver-Turbo achieves $2\times\sim50\times$ speedup in terms of wall-clock time without measurable quality degradation. The source code will be released publicly.

Primary Area: generative models

Submission Number: 4081