When Less is More: One Strategic Step in LLM Refinement
Track: long paper (up to 4 pages)
Keywords: self-refinement, self-reasoning, maths, first-step
TL;DR: We propose a tradeoff between accuracy and time in Chain-of-Thought reasoning for math problems by generating only the first step, verified by a second LLM.
Abstract: Addressing hallucinations in LLMs for Math World Problems (MWPs) is key to reliability and efficiency. We optimize the trade-off between accuracy and computation in CoT reasoning by verifying only the first step before proceeding. A verifier assesses correctness, halting generation if incorrect. This approach reduces token generation time by 30\% with under 5\% accuracy loss, while corrections improve accuracy by up to 10\%. By skipping flawed reasoning early, our method balances accuracy and efficiency, cutting unnecessary computation.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 11
Loading