Multidimensional Consistency Improves Reasoning in Language Models

ACL ARR 2025 May Submission4610 Authors

Abstract: While Large language models (LLMs) have proved able to address some complex reasoning tasks, we also know that they are highly sensitive to input variation, which can lead to different solution paths and final answers. Answer consistency across input variations can thus be taken as a sign of stronger confidence. Leveraging this insight, we introduce a framework, {\em Multidimensional Reasoning Consistency} where, focusing on math problems, models are systematically pushed to diversify solution paths towards a final answer, thereby testing them for answer consistency across multiple input variations. We induce variations in order of shots in prompt, problem phrasing, and languages used. Experiments on a wide range of open-source state-of-the-art LLMs of various sizes show that reasoning consistency differs by variation dimension, and that by aggregating consistency across dimensions, our framework enhances mathematical reasoning performance on monolingual datasets GSM8K and MATH500, and the multilingual dataset MGSM.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: mathematical NLP

Contribution Types: NLP engineering experiment

Languages Studied: Bengali (BN), Chinese (ZH), French (FR), English (EN), German (DE), Japanese (JA), Russian (RU), Spanish (ES), Swahili (SW), Telugu (TE) and Thai (TH)

Submission Number: 4610