GIFT: Guided Importance-Aware Fine-Tuning for Diffusion Language Models

Guowei Xu, Wenxin Xu, Zhao Jiawang, Kaisheng Ma

Track: long paper (up to 10 pages)

Keywords: diffusion language models, entropy-based methods

TL;DR: We introduce GIFT, a theoretically grounded importance-aware fine-tuning algorithm for diffusion language models.

Abstract:

Diffusion models have recently shown strong potential in language modeling, offering faster generation compared to traditional autoregressive approaches.
While the diffusion mechanism enables the model to reason over entire sequences, it also makes the generation process less predictable and often inconsistent.
This highlights the importance of controlling key tokens that guide the direction of generation.
To address this issue, we propose GIFT, an importance-aware finetuning method for diffusion language models, where tokens are assigned different importance weights based on their entropy.
Derived from diffusion theory, GIFT delivers substantial gains: across diverse settings including different mainstream training datasets ranging from 1k to 10k in size, utilizing LoRA or full parameter fine-tuning, and training on base or instruct models with different architectures, GIFT consistently achieves superior overall performance compared to standard SFT on four widely used reasoning benchmarks (Sudoku, Countdown, GSM8K, and MATH-500).

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Funding: No, the presenting author of this submission does *not* fall under ICLR’s funding aims, or has sufficient alternate funding.

Submission Number: 78