LoRA-Adapted Diffusion

Ruurd Jan Anthonius Kuiper, Lars de Groot, Bram van Es, Maarten van Smeden, Ayoub Bagheri

Track: Type B (Encore Abstracts)

Keywords: Large Language Models, Diffusion, Natural Language Processing, Text Generation

Abstract: Autoregressive models currently dominate text generation but suffer from left-to-right decoding constraints that limit efficiency and bidirectional reasoning. Diffusion-based models offer a flexible alternative but face challenges in adapting to discrete text efficiently. We propose LAD (LoRA-Adapted Diffusion), a framework for non-autoregressive generation that adapts LLaMA models for iterative, bidirectional sequence refinement using LoRA adapters. LAD employs a structural denoising objective combining masking with text perturbations (swaps, duplications and span shifts), enabling full sequence editing during generation. We aim to demonstrate that LAD could be a viable and efficient alternative to training diffusion models from scratch, by providing both validation results in our manuscript, as well as two interactive demos directly available online: https://ruurdkuiper.github.io/tini-lad/

Serve As Reviewer: ~Ruurd_Jan_Anthonius_Kuiper1

Submission Number: 30