Reasoning on the Manifold: Bidirectional Consistency for Self-Verification in Diffusion Language Models

Jiaoyang Ruan, Xin Gao, Yinda Chen, Hengyu Zeng, Liang Du, Guanghao Li, Jie Fu, Jian Pu

TL;DR: We propose Bidirectional Manifold Consistency (BMC), a training-free geometric signal for diffusion LLMs that enables verification without labels, smarter inference-time compute, and dense rewards for RL alignment.

Abstract: While Diffusion Large Language Models (dLLMs) offer structural advantages for global planning, efficiently verifying that they arrive at correct answers via valid reasoning traces remains a critical challenge. In this work, we propose a geometric perspective: Reasoning on the Manifold. We hypothesize that valid generation trajectories reside as stable attractors on the high-density manifold of the learned distribution, whereas invalid paths exhibit off-manifold drift. To operationalize this, we introduce Bidirectional Manifold Consistency (BMC), a training-free, unsupervised metric that quantifies the stability of the generated sequence through a forward-masking and backward-reconstruction cycle. Empirically, we demonstrate BMC's versatility across the full reasoning lifecycle: (1) in Diagnosis, it serves as a robust discriminator of solution validity without ground truth answer; (2) in Inference, it enables rejection resampling to effectively concentrate computational resources on complex reasoning tasks; and (3) in Alignment, it functions as a dense geometric reward that transforms sparse outcome supervision into fine-grained guidance, empowering models to self-evolve beyond standard baselines. Our results establish intrinsic geometric stability as a robust indicator of correctness for dLLMs.

Lay Summary: Modern AI language models can solve math problems and answer complex questions, but they sometimes produce confident answers that contain hidden mistakes, such as a wrong step buried in otherwise fluent reasoning. Detecting these mistakes without already knowing the correct answer is a major challenge. This work studies a new kind of language model called a diffusion language model, which generates text by repeatedly refining a noisy draft into a clear final version. We discovered that this iterative refinement process leaves behind a useful signal: when the model arrives at a correct answer, it can reliably reconstruct the reasoning even after we hide most of it; when the answer is wrong, the reconstruction drifts toward something different. We can therefore tell a correct reasoning chain from a flawed one simply by testing its stability during this reconstruction process, requiring no human labels or external checker. We show that this stability signal helps in three practical ways: spotting errors in generated answers, deciding when the model should try again on hard questions, and training the model to favor more reliable reasoning over time.

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: Diffusion Language Models, Generative Models, Reasoning

Submission Number: 2927