Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance

ICLR 2026 Conference Submission24486 Authors

Keywords: Diffusion models, Classifier-free guidance

Abstract:

Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality.
Autoguidance (AG) has extended this idea, but it relies on an auxiliary network and leave solver-induced errors unaddresed.
In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes a critical factor to deteriorate sample quality.
Our key observation is that these errors align with the dominant eigenvector, motivating us to target the solver-induced error as a guidance signal.
We propose **E**mbedded **R**unge–**K**utta based **Guid**ance (ERK-Guid), which exploits detected stiffness to reduce LTE and stabilize sampling.
We theoretically and empirically analyze stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid.
Our experiments on both synthetic datasets and popular benchmark dataset, ImageNet, demonstrate that ERK-Guid consistently outperforms state-of-the-art methods.

Primary Area: generative models

Submission Number: 24486