Multilingual Mathematical Reasoning: Advancing Open-Source LLMs in Hindi and English

Avinash Anand, Kritarth Prasad, Chhavi Kirtani, Ashwin R. Nair, Manvendra Kumar Nema, Raj Jaiswal, Rajiv Ratn Shah

Abstract: Large Language Models (LLMs) excel in linguistic tasks but struggle with mathematical reasoning, particularly in non- English languages like Hindi. This research aims to en- hance the mathematical reasoning skills of smaller, resource- efficient open-source LLMs in both Hindi and English. We evaluate models like OpenHathi 7B, LLaMA-2 7B, Wizard- Math 7B, Mistral 7B, LLeMMa 7B, MAmmoTH 7B, Gemini Pro, and GPT-4 using zero-shot, few-shot chain-of-thought (CoT) methods, and supervised fine-tuning. Our approach in- corporates curriculum learning, progressively training mod- els on increasingly difficult problems, a novel Decompo- sition Strategy to simplify complex arithmetic operations, and a Structured Solution Design that divides solutions into phases. Our experiments result in notable performance en- hancements. WizardMath 7B exceeds Gemini’s accuracy on English datasets by +6% and matches Gemini’s performance on Hindi datasets. Adopting a bilingual approach that com- bines English and Hindi samples achieves results comparable to individual language models, demonstrating the capability to learn mathematical reasoning in both languages. This re- search highlights the potential for improving mathematical reasoning in open-source LLMs.