Less is Not Worse: Effective Reasoning Without Complete Reasoning Chains

Jaehui Hwang, Sangdoo Yun, Byeongho Heo, Dongyoon Han

Keywords: thinking trajectory, large reasoning model, redundancy

Abstract:

Large language models (LLMs) often produce lengthy reasoning traces with substantial token redundancy.
While reasoning processes are generally considered necessary, it has been underexplored whether LLMs truly require the complete trajectory.
To investigate, we conduct (1) attention map analysis and (2) targeted lesion studies that remove token groups, both of which show that intermediate tokens contribute minimally to reasoning quality. Our analyses suggest that the most redundant segments typically appear in the middle of reasoning chains, whereas the earlier and later segments are crucial for accurate final outcomes. We argue that this approach avoids redundant intermediate information and exploits the LLM’s capability to infer concise and coherent intermediate steps by using the known start and end points. Based on these observations, we propose MidCut, a method that removes redundant middle steps during both training and inference. We evaluate MidCut in two scenarios for LLM reasoning: (1) supervised fine-tuning (SFT) for reasoning and (2) decoding strategy for a test-time application.

Submission Number: 105