Language Models Speak in Sentences: Sentence Structure Improves Language Model's Reasoning Capability

ACL ARR 2025 May Submission2361 Authors

Abstract: We propose a new method to improve large language models' (LLMs) performance by incorporating the sentence structure knowledge into the model. Based on the intuitive assumption that a complete sentence is the basic unit of thinking and reasoning for human beings, we test it for LLMs by explicitly inserting special segment tokens to the positions within the input sequences where sentence boundaries are detected, which achieves better performance in complex reasoning tasks by significant margins. Two approaches for incorporating sentence structure knowledge are experimented: In-context learning (ICL) on instruction-tuned models (Llama3-8B-Instruct and Qwen2-7B-Instruct) and supervised finetuning (SFT) on a base model (Llama3-8B finetuned with TULU3), and evaluated in highly reasoning-intensive tasks (e.g., math), both show positive results. Our findings indicate that similar to human reasoning, structured sentences can effectively facilitate LLM reasoning performance; integrating linguistically motivated priors, such as sentence boundaries, is a promising future direction for developing simple-yet-effective prompting techniques.

Paper Type: Long

Research Area: Language Modeling

Research Area Keywords: sentence segmentation; fine-tuning; prompting; inference methods;

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 2361