Closing the Data-Efficiency Gap Between Autoregressive and Masked Diffusion LLMs

ICLR 2026 Conference Submission13765 Authors

Keywords: LLM, diffusion language model, knowledge injection, fine-tuning

Abstract: Despite autoregressive large language models (arLLMs) having been the dominant paradigm in language modeling, they resist knowledge injection via fine-tuning due to inherent shortcomings such as the "reversal curse" - the challenge of answering questions that reverse the original information order in the training sample. Masked diffusion large language models (dLLMs) are rapidly emerging as a powerful alternative to the arLLM paradigm, with evidence of better data efficiency and free of the ``reversal curse'' in pre-training. However, it is unknown whether these advantages still extend to the post-training phase, i.e. whether pre-trained dLLMs can easily acquire new knowledge through fine-tuning. To assess post-training knowledge acquisition and generalization, we perform fine-tuning using 3 different datasets on arLLMs and dLLMs and evaluate them with two types of QA formats: forward style QA (questions follow the original information order of the training sample) and backward style QA (questions reverse the original information order of the training sample), which probes the reversal curse. We first show that arLLMs heavily rely on paraphrases to generalize knowledge text into question-answering (QA) performance; paraphrases are only effective when the information order in paraphrased text matches the QA style. In contrast, dLLMs achieve strong performance on both forward and backward style QAs without paraphrases, with paraphrases yielding only marginal additional gains. Lastly, inspired by the performance of dLLM fine-tuning, we propose a new masked fine-tuning paradigm for knowledge injection in pre-trained arLLMs. The proposed paradigm drastically improves the data efficiency in arLLMs fine-tuning, closing the gap with dLLMs.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 13765