RecRoll: Adaptive Depth First Search in Autoregressive Predictive Space

Mykyta Ielanskyi, Sepp Hochreiter

Track: tiny / short paper (up to 4 pages)

Keywords: search, reasoining, llm, algorithmic, branch and bound

TL;DR: We propose a branch and bound style decoding algorithm for the Reasoning LLMs that helps with optimization problems.

Abstract:

One of the compelling difficulties of autoregressive generation is its one way nature. This often leads to drift and overconfidence effects, particularly in language modeling. We introduce RecRoll (Recapitulate and Roll Back) which augments the autoregressive decoding with backtracking that enables models to dynamically overcome the problems of overconfidence by deconditioning on the selected branch of output. This decoding algorithm enables extended test time inference scaling bypassing some of the limitations of models context length. Our approach compartmentalizes model decoding in long-form reasoning, bridging it with depth first search. We show that RecRoll improves outcomes on several challenging reasoning tasks even without fine tuning the models. The inductive bias that our decoding scheme imposes unto the language models resembles branch and bound algorithm and improves performance on tasks which could be solved symbolically
in such manner. We further discuss several approaches that could be used to fine tune reasoning models for RecRoll.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Submission Number: 122