HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning

Simeng Han, Tianyu Liu, Chuhan Li, Xuyuan Xiong, Arman Cohan

Keywords: chain-of-thought, symbolic reasoning, prompt design, finetuning, bootstrapping, self-training

TL;DR: We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach between natural language reasoning and symbolic reasoning for each reasoning problem.

Abstract: LLMs approach logical and mathematical reasoning through natural or symbolic languages. While natural language offers human-accessible flexibility but suffers from ambiguity, symbolic reasoning provides precise, machine-executable inferences at the cost of strict domain constraints. We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach for each reasoning problem. Through extensive experiments, we evaluate both prompting-based approaches with state-of-the-art LLMs and fine-tuned open-source models. We find that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o's natural language reasoning by 4.4% on FOLIO and 1.3% on MATH. More notably, using GPT-3.5-turbo as a prompted meta-selector yields a 10% improvement on FOLIO's challenging subset compared to GPT-4o. We will release our code and data to support future research.

Paper Published: No

Paper Category: Long Paper

Demography: Women in ML/NLP

Academic: Year > 2 PhD Student

Submission Number: 17