ARIES: Autonomous Reasoning with Large Language Models on Interactive Thought Graph Environments

ACL ARR 2025 May Submission4854 Authors

Abstract: Recent research has shown that the performance of Large Language Models (LLMs) on reasoning tasks can be enhanced by scaling test-time compute. One promising approach, particularly with decomposable problems, involves arranging intermediate solutions as a graph on which transformations are performed to explore the solution space. However, prior works rely on pre-determined, task-specific transformation schedules defined by a set of searched hyperparameters. In this work, we view thought graph transformations as actions in a Markov decision process, and investigate whether LLMs can act as a policy agents on this graph-based environment. We introduce ARIES, a multi-agent architecture for reasoning with LLMs in which LLM policy agents maintain visibility of the thought graph states and dynamically adapt the problem-solving strategy. We observe that using off-the-shelf LLMs as policy agents with ARIES can yield up to 29\% higher accuracy on HumanEval relative to static transformation schedules despite bearing no search cost.

Paper Type: Short

Research Area: Machine Learning for NLP

Research Area Keywords: graph-based methods, generative models, generalization

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models

Languages Studied: English

Submission Number: 4854