Information Directed Tree Search: Reasoning and Planning with Language Agents
Keywords: Information gain, tree search, Language agents, reasoning and planning
TL;DR: For language agents that need to solve a task by interaction, we explore the utility of taking a bayesian approach and doing exploration during tree search based on information gain.
Abstract: Solving challenging tasks often require agentic formulation of language models that can do multi-step reasoning and progressively solve the task by collecting various feedback. For computational efficiency, it may be advantageous to quantify the information associated with different feedback and guide the search such that the solution can be obtained quickly. To explore this possibility, we take a Bayesian approach and propose an \textit{information directed tree search} (IDTS) algorithm that makes use of in-context learning to approximate the information associated with different feedback. We explore the effectivity of IDTS on challenging tasks involving programming, formal math, and natural language. Interestingly, while we find advantages over simple uniform search methods, the proposed approach is about comparable to MCTS even though it explores different paths. We discuss some possibilities for our findings and highlight open questions for future work.
Submission Number: 118
Loading