Solution Path Routing for Accurate and Efficient Language Model Reasoning

ACL ARR 2025 February Submission4384 Authors

Abstract:

Language models can solve problems in multiple ways. For instance, they can reason step-by-step in natural language, or generate a program that can produce the final answer. In this work, we first empirically demonstrate that there is no one-size-fits-all solution; in some cases code is a better option with respect to accuracy and token-efficiency, but in other cases only natural language allows a correct answer to be found.
We then examine language models' ability to appropriately perform $\textit{solution path routing}$, choosing the most appropriate solution path based on the problem. We find that models struggle to pick the most appropriate solution path simultaneously with solving the problem, but by using a 2-stage pipeline with explicit routing and then problem solving we are able to achieve efficiency gains and sometimes performance improvements.

Paper Type: Short

Research Area: Language Modeling

Research Area Keywords: Code, Reasoning, Solution Routing

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 4384