HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning

Simeng Han, Tianyu Liu, Chuhan Li, Xuyuan Xiong, Arman Cohan

Keywords: meta-selection, math reasoning, logical reasoning

TL;DR: We introduce HYBRIDMIND, an adaptive strategy that shows natural language reasoning and symbolic reasoning are complementary.

Abstract:

LLMs approach logical and mathematical reasoning through natural or symbolic languages. While natural language offers human-accessible flexibility but suffers from ambiguity, symbolic reasoning provides precise, machine-executable inferences at the cost of strict domain constraints. We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach for each reasoning
problem. Through extensive experiments, we evaluate both prompting-based approaches with state-of-the-art LLMs and fine-tuned open-source models. We find that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o’s natural language reasoning by 4.4% on FOLIO and 1.3% on MATH. More notably, using GPT-3.5-turbo as a prompted meta-selector yields a 10% improvement on
FOLIO’s challenging subset compared to GPT-4o. We will release our code and data to support future research.

Submission Number: 169