Decoupled Classifier-Free Guidance for Counterfactual Diffusion Models

ICLR 2026 Conference Submission12357 Authors

Keywords: Counterfactual Generation; Diffusion Models; Classifier-Free Guidance

Abstract: Counterfactual generation aims to simulate realistic hypothetical outcomes under causal interventions. Diffusion models have emerged as a powerful tool for this task, combining DDIM inversion with conditional generation and classifier-free guidance (CFG). In this work, we identify a key limitation of CFG for counterfactual generation: it prescribes a global guidance scale for all attributes, leading to significant spurious changes in inferred counterfactuals. To mitigate this, we propose \textit{Decoupled Classifier-Free Guidance} (DCFG), a flexible and model-agnostic guidance technique that enables attribute-wise control following a causal graph. DCFG is implemented via a simple attribute-split embedding strategy that disentangles semantic inputs, enabling selective guidance on user-defined attribute groups. Our experiments demonstrate that DCFG significantly improves the axiomatic soundness of inferred counterfactuals on challenging medical imaging data, mitigating spurious amplification effects, and enhancing counterfactual reversibility.

Primary Area: generative models

Submission Number: 12357