Making Grid Beam Search Less Greedy

ACL ARR 2025 February Submission1161 Authors

Abstract:

A common formalism for constraining the output of autoregressive text generation models involves *lexical constraints*, words or phrases which are required to occur in the generated text.
DFA-constrained beam search and grid beam search are two widely used paradigms for decoding from autoregressive models while enforcing lexical constraints.
As the former approach requires a number of forward passes exponential
in the number of constraint tokens, it is often dispreferred to the latter, which requires only linearly many forward calls.
However, while grid beam search achieves an exponential speedup, it does so in a manner
which does not treat all of the constraints equally.
In this paper, we demonstrate that grid beam search is biased to
incorporate easier-to-satisfy constraints first, leaving harder constraints to the end of the sequence.
This contrasts with DFA-constrained beam search, which exhibits no such bias.
To address this shortcoming, we propose fair grid beam search, a modification to grid beam search which avoids this bias while still requiring only linearly many forward passes.
Experimentally, we confirm grid beam search's bias on two constrained generation tasks, finding significant differences in how it orders constraint tokens as compared to DFA-constrained beam search and fair grid beam search.
Furthermore, we find that fair grid beam search not only fixes grid beam search's bias, but finds higher-probability strings in the process.

Paper Type: Long

Research Area: Generation

Research Area Keywords: analysis, inference methods, model architectures

Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models, Theory

Languages Studied: English

Submission Number: 1161