Fixing Model-Fitting: Compressing Guidance for Better Sampling

ICLR 2026 Conference Submission14945 Authors

Keywords: Diffusion, guidance

TL;DR: The paper analyses model-fitting problem when using guidance, and propose a method to solve that

Abstract: Model-fitting, a phenomenon where generated samples are overly adjusted to the model used for guidance rather than the intended conditions, is a key drawback and often leads to suboptimal outcomes. The root cause of this problem is the consecutiveness of guidance timesteps throughout the diffusion sampling process. In this work, We quantify this effect and show that breaking the consecutiveness of standard guidance alleviates the problem. Based on this insight, our method, Compress Guidance, distributes a small number of guidance steps across the full sampling process, yielding substantial improvements in image quality and diversity while cutting guidance cost by over 80\%. Experiments on both label-conditional and text-to-image generation, across multiple datasets and models, confirm that Compress Guidance consistently surpasses baselines in image quality with significantly lower computational overhead.

Primary Area: generative models

Submission Number: 14945