Unlocking Coherent Reasoning in LLMs with Hierarchical Soft Prompts

Xudong Wang, Chaoning Zhang, Qigan Sun, Yitian Zhou, Jiehui Xie, Jiaquan Zhang, Chenghao Li, Pengcheng Zheng, Caiyan Qin, Guoqing Wang, Yang Yang

Keywords: Large Language Models, Complex Reasoning, Soft Prompt Tuning

Abstract:

Large language models (LLMs) exhibit strong reasoning capabilities in complex tasks. Soft prompt tuning, as a lightweight approach, injects trainable vectors into the input to guide the reasoning process and enhance model performance. Prior studies show that soft prompts effectively activate prior knowledge and improve problem understanding in the early stages of reasoning. However, when they continue to exert strong influence in the middle and later stages, they often disrupt the information flow and degrade reasoning performance. Based on this observation, we argue that the role of soft prompts should not be confined to a single stage of activation and guidance. Instead, they should be inserted at appropriate stages to ensure smooth information transmission across layers. Existing methods, however, typically rely on one-shot static injection and cannot dynamically regulate prompts across stages, leading to functional mismatches during reasoning. To address this limitation, we propose a dynamic hierarchy-aware mechanism(DHAM). This mechanism first employs hierarchical clustering to derive stage-specific representations, and then leverages the semantic guidance capability of soft prompts to adaptively align and activate them, ensuring effective coordination across reasoning stages.
DHAM yields consistent gains across models and benchmarks (e.g., 29.5\%→43.8\% on Llama-2-13B/GSM8K), with ablations showing CKA clustering and moderate stage numbers (e.g., $G=3/4$) perform best, consistent with the stable information flow hypothesis.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 11167