DLM-SWAI: Steering Diffusion Language Models Before They Unmask

ACL ARR 2026 May Submission15820 Authors

Keywords: Model steering, Diffusion language models, Logit intervention, Controllable Generation

Abstract: Steering language model generation toward desired textual properties is essential for practical deployment, and inference-time methods are particularly appealing because they enable controllable generation without retraining. Recent work has also highlighted diffusion language models as an emerging generation paradigm with distinct decoding properties. However, most existing steering approaches either rely on auxiliary models or are designed for autoregressive next-token decoding, making them difficult to apply to diffusion language models (DLMs), which generate text through iterative denoising of partially masked sequences. Therefore, we propose DLM-SWAI, a simple training-free steering method that biases the token distribution at each denoising step using pre-computed token-level style scores. Experiments on style and safety control tasks show that DLM-SWAI effectively steers diffusion language models while preserving generation quality and requiring minimal computational overhead. Ablations further reveal a controllable trade-off between steering strength and fluency, and our analysis links class-wise steerability to the strength of token-level attribute cues. Our code is available at https://anonymous.4open.science/r/dlm-swai-2358.

Paper Type: Long

Research Area: Generation

Research Area Keywords: inference methods, analysis, prompting, safety

Contribution Types: NLP engineering experiment, Publicly available software and/or pre-trained models

Languages Studied: English

EMNLP 2026 AI Reviewing Experiment: no

Submission Number: 15820