Diverse Text Generation through Soft Prompt Tuning

ICLR 2026 Conference Submission17810 Authors

Keywords: Diverse Text Generation, Soft Prompt Tuning

Abstract:

Diverse text generation is crucial for effective exploration in language models. Current sampling-based decoding methods struggle to balance quality and diversity and lack control over generating mutually distinct outputs. Reinforcement learning approaches maintain quality, but require extensive training and are difficult to transfer across domains due to task-specific reward functions.
We propose a lightweight framework that learns diversely initialized continuous soft prompt vectors, which, when prepended to input prompts, guide the model's final-token hidden states into distinct representation regions. This enables diverse generations from identical inputs, as initial hidden state differences amplify through the autoregressive mechanism, creating increasingly divergent generations. By preserving earlier hidden state similarities, our method maintains contextual consistency to task-specific constraints.
Experiments across combinatorial tasks, question generation, and molecular design reveal that our soft prompt tuning method improves diversity while consistently adhering to task-specific constraints. Our approach shows particular strength in complex settings with large exploration spaces, as demonstrated through our novel contribution of a challenging combinatorial dataset specifically designed to evaluate diverse generation capabilities of language models. This lightweight framework provides a unified, broadly applicable solution for diverse text generation across various application domains.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 17810