Constrained Adaptive Rejection Sampling

Paweł Parys, Sairam Vaidya, Taylor Berg-Kirkpatrick, Loris D'Antoni

Keywords: language models, constrained decoding, rejection sampling

TL;DR: We present Constrained Adaptive Rejection Sampling, a simple yet principled alternative to constrained decoding from language models that enforces constraints without distributional distortion

Abstract:

Language Models (LMs) are increasingly used in applications where generated outputs must satisfy strict semantic or syntactic constraints. Although constrained decoding methods can enforce constrain satisfaction, they often alter the underlying LM distribution, limiting their usefulness in settings such as program fuzzing, where both validity and diversity of samples are essential.
We present Constrained Adaptive Rejection Sampling (CARS), a simple yet principled alternative to constrained decoding that enforces constraints without distributional distortion. CARS begins with unconstrained LM sampling and adaptively rules out constraint-violating continuations by recording them in a trie and subtracting their probability mass from future draws. This adaptive pruning ensures that prefixes that have been proven invalid are never revisited, acceptance rates improve monotonically, and the resulting samples exactly follow the constrained distribution. In experiments on synthetic domains and program fuzzing benchmarks from prior work on constrained decoding, CARS consistently achieves higher efficiency and stronger sample diversity compared to prior constrained decoding techniques.

Primary Area: generative models

Submission Number: 6397