Conditional Deontics over Terminals: A Mildly Context-Sensitive Formal Grammar for Constrained Decoding

ICLR 2026 Conference Submission20084 Authors

Keywords: constrained decoding; formal grammar; parsing theory; formal language theory

Abstract:

Constrained decoding researchers have recently thought to extend context-free constrainers to context-sensitive constraints such as matching selected columns to tables in text-to-sql.

This is challenging, because context-sensitivity is achieved by cross referencing different subtrees in the syntactic structure, which is only partially available during LLM generation.

Recent frameworks such as IterGen gain some context sensitivity but with drawbacks including the need for expensive backtracking or speculative lookahead of LLM, overly permissive semantics risking over-constrain, and complication in interfacing with works on improving running speed such as XGrammar.

To address these concerns, we propose a new mildly-context-sensitive formal grammar called Conditional Deontics over Terminals (CDoT).

Its incremental parser has O(n^3) time complexity each step and allows for GPU acceleration, compared to the O(n^8) of mildly context-sensitive tree-adjoining grammars and the O(n^2) of context-free grammars.

This new formal grammar is strong enough to implement the unit propagation algorithm, which we employ to assist an LLM during solving of a logical puzzle called "Knight and Knave", achieving substantially improved performance at reduced output token budget.

We also evaluate on the traditional constrained decoding evaluation task of text-to-sql.

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Submission Number: 20084