Beyond Fully Random Masking: Attention-Guided Denoising and Optimization for Diffusion Language Models

ACL ARR 2026 January Submission1044 Authors

Keywords: Diffusion Language Models, Post-training, Reinforcement Learning, Attention Mechanism, Mathematical Reasoning, Code Generation

Abstract:

Diffusion large language models (dLLMs) offer an efficient alternative to autoregressive models through parallel decoding, yet existing post-training methods largely rely on random masking strategies that overlook intrinsic token dependencies. In this work, we present an empirical analysis of attention in dLLMs and show that tokens attending more strongly to revealed context exhibit greater generation stability and play a critical role in reasoning.
Motivated by these findings, we propose AGDO, an attention-guided denoising and optimization framework that aligns both training and optimization with attention-derived dependencies. AGDO determines the denoising order based on attention structure and emphasizes attention-critical tokens during supervised fine-tuning and reinforcement learning. Experiments on mathematical and coding benchmarks demonstrate that AGDO consistently improves reasoning performance, outperforming state-of-the-art post-training methods for dLLMs.

Paper Type: Long

Research Area: Language Models

Research Area Keywords: applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 1044