Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction

Soochul Park, Yeon Ju Lee

Keywords: Generative Models, Diffusion Models, Fast Sampling, ODE Solver

TL;DR: This work generalizes multistep samplers via parameterizing the prediction type, integration domain, and second-order residual term, and introduces classification-based parameter learning.

Abstract:

Diffusion models achieve state-of-the-art image quality.
However, sampling is costly at inference time because it requires a large number of function evaluations (NFEs).
To reduce NFEs, classical ODE numerical methods have been adopted.
Yet, the choice of prediction type and integration domain leads to different sampling behaviors.
To address these issues, we introduce Dual-Solver, which generalizes multistep samplers through learnable parameters that continuously (i) interpolate among prediction types, (ii) select the integration domain, and (iii) adjust the residual terms.
It retains the standard predictor-corrector structure while preserving second-order local accuracy.
These parameters are learned via a classification-based objective using a frozen pretrained classifier (e.g., MobileNet or CLIP).
For ImageNet class-conditional generation (DiT, GM-DiT) and text-to-image generation (SANA, PixArt-$\alpha$), Dual-Solver improves FID and CLIP scores in the low-NFE regime ($3 \le$ NFE $\le 9$) across backbones.

Primary Area: generative models

Submission Number: 10165