DIP: Dynamic In-Context Planner For Diffusion Language Models

ACL ARR 2026 January Submission3945 Authors

Keywords: Diffusion Language Models, Efficient Inference, In-Context Learning

Abstract:

Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples.
However, due to the bidirectional attention mechanism, DLMs incur substantial computational cost as context length increases. This work addresses this issue with a key discovery: unlike the sequential generation in autoregressive language models (ARLMs), the diffusion generation paradigm in DLMs allows $\textit{efficient dynamic adjustment of the context}$ during generation.
Building on this insight, we propose $\textbf{D}$ynamic $\textbf{I}$n-Context $\textbf{P}$lanner (DIP), a context-optimization method that dynamically selects and inserts in-context examples during generation, rather than providing all examples in the prompt upfront.
Results show DIP maintains generation quality while achieving up to 12.9$\times$ inference speedup over standard inference and 1.17$\times$ over KV cache-enhanced inference.

Paper Type: Short

Research Area: Language Models

Research Area Keywords: LLM Efficiency, prompting

Contribution Types: Model analysis & interpretability, Reproduction study, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models

Languages Studied: English

Submission Number: 3945