Enhancing Mathematical Problem Solving in LLMs through Execution-Driven Reasoning Augmentation

ACL ARR 2026 January Submission6815 Authors

Keywords: Large Language Models, Multi-Agent Systems, Mathematical Reasoning, Program-Guided, Reasoning Agents, Iterative Self-Improvement

Abstract: Mathematical problem solving is a fundamental benchmark for assessing the reasoning capabilities of artificial intelligence and a gateway to applications in education, science, and engineering where reliable symbolic reasoning is essential. Although recent advances in multi-agent LLM-based systems have enhanced their mathematical reasoning capabilities, they still lack a reliably revisable representation of the reasoning process. Existing agents either operate in rigid sequential pipelines that cannot correct earlier steps or rely on heuristic self-evaluation that can fail to identify and fix errors. In addition, programmatic context can distract language models and degrade accuracy. To address these gaps, we introduce Iteratively Improved Program Construction (IIPC), a reasoning method that iteratively refines programmatic reasoning chains and combines execution feedback with the native Chain-of-thought abilities of the base LLM to maintain high-level contextual focus. IIPC surpasses competing approaches in the majority of reasoning benchmarks on multiple base LLMs. All code and implementations will be released as open source upon publication.

Paper Type: Long

Research Area: Mathematical, Symbolic, Neurosymbolic, and Logical Reasoning

Research Area Keywords: Reasoning and Planning, Language Model Applications, Computational Semantics

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 6815