Verbosity-Aware Rationale Reduction: Sentence-Level Rationale Reduction for Efficient and Effective Reasoning

ACL ARR 2025 February Submission2278 Authors

Abstract:

Large Language Models (LLMs) rely on generating extensive intermediate reasoning units (e.g., tokens, sentences) to enhance final answer quality across a wide range of complex tasks.
While this approach has proven effective, it inevitably increases substantial inference costs.
Previous methods adopting token-level reduction without clear criteria result in poor performance compared to models trained with complete rationale.
To address this challenge, we propose a novel sentence-level rationale reduction framework leveraging likelihood-based criteria, *verbosity*, to identify and remove redundant reasoning sentences.
Unlike previous approaches, our method leverages *verbosity* to selectively remove redundant reasoning sentences while preserving reasoning capabilities.
Our experimental results across various reasoning tasks demonstrate that our method improves performance by an average of 7.71% while reducing token generation by 19.87% compared to model trained with complete reasoning paths.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: reasoning, reasoning path reduction, efficient and effective reasoning

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 2278