SoftCFG: Uncertainty-guided Stable Guidance for Visual Autoregressive Model

Dongli Xu, Aleksei Tiulpin, Matthew B. Blaschko

Keywords: visual autoregressive model

TL;DR: We propose SoftCFG, an uncertainty-guided and context-aware regularizer for AR image generation that stabilizes classifier-free guidance and achieves SOTA FID of ARs on ImageNet $256\times256$ without extra training or changes.

Abstract: Autoregressive (AR) models have emerged as powerful tools for image generation by modeling images as sequences of discrete tokens. While Classifier-Free Guidance (CFG) has been adopted to improve conditional generation, its application in AR models faces two key issues: guidance diminishing, where the conditional–unconditional gap quickly vanishes as decoding progresses, and over-guidance, where strong conditions distort visual coherence. To address these challenges, we propose SoftCFG, an uncertainty-guided inference method that distributes adaptive perturbations across all tokens in the sequence. The key idea behind SoftCFG is to let each generated token contribute certainty-weighted guidance, ensuring that the signal persists across steps while resolving conflicts between text guidance and visual context. To further stabilize long-sequence generation, we introduce Step Normalization, which bounds cumulative perturbations of SoftCFG. Our method is training-free, model-agnostic, and seamlessly integrates with existing AR pipelines. Experiments show that SoftCFG significantly improves image quality over standard CFG and achieves state-of-the-art FID on ImageNet 256 × 256 among autoregressive models.

Primary Area: generative models

Submission Number: 4083