Are Large Language Models Meta Reasoners?
Abstract: In this paper, we introduce Meta-Reasoning Prompting (MRP), a novel approach inspired by human meta-reasoning to enhance the flexibility and generality of large language models (LLMs). Traditional in-context learning techniques, such as Tree-of-Thoughts, show promise but lack consistent state-of-the-art performance across diverse tasks due to their specialized nature. MRP addresses this limitation by dynamically selecting and applying different reasoning methods based on the specific requirements of each task, optimizing both performance and computational efficiency. The MRP framework operates in two phases: initially, the LLM selects the most appropriate reasoning method using task input cues and objective descriptions of available methods; subsequently, it applies the chosen method to complete the task. This dynamic strategy mirrors human meta-reasoning, allowing the model to excel in a wide range of problem domains. We evaluate the effectiveness of MRP through comprehensive benchmarks. The results demonstrate that MRP achieves or approaches state-of-the-art performance across these diverse tasks. MRP represents a significant advancement in enabling LLMs to autonomously select suitable reasoning methods, enhancing their ability to handle diverse and complex problem domains efficiently.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: Prompt Engineering, Meta-Reasoning, Reasoning with LLMs, LLM Agents
Contribution Types: NLP engineering experiment
Languages Studied: English
Submission Number: 4869
Loading