Activation Differential Analysis for Enhancing Chain-of-thought Reasoning

ACL ARR 2025 February Submission7681 Authors

Abstract: Despite the impressive chain-of-thought (CoT) reasoning ability of large language models(LLMs), its underlying mechanisms remains unclear. In this paper, we explore the inner workings of LLM’s CoT ability via the lens of neurons in the feed-forward layers. We propose an efficient method to identify reasoning-critical neurons by analyzing their activation patterns under reasoning chains of varying quality. Based on it, we devise a rather simple intervention method that directly stimulates these reasoning-critical neurons, to guide the generation of high-quality reasoning chains. Extended experiments validate the effectiveness of our method and demonstrate the critical role these identified neurons play in CoT reasoning. Our code and data will be publicly available.

Paper Type: Short

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: knowledge tracing/discovering/inducing

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 7681