RAFT: Reasoning-Aware Fine-Tuning

Jinhao Li, Guangrui Li, Riheng Zhu, Zhenbang Sun, Feng Liu

Keywords: foundation models, reasoning, supervised fine-tuning

Abstract:

Supervised fine-tuning (SFT) adapts large pretrained models to downstream tasks but often fails to learn reasoning-consistent mappings, especially under limited data. Chain-of-Thought (CoT) fine-tuning addresses this by training models to produce explicit reasoning traces, but at the cost of significantly increased inference latency and variable effectiveness across domains. We introduce \emph{Reasoning-Aware Fine-Tuning (RAFT)}, a single-stage framework that distils reasoning signals during training without requiring reasoning generation at inference. RAFT leverages a reasoning-discriminative loss applied to positive and negative reasoning traces sampled from a teacher model, guiding the student to align its internal scoring with valid reasoning while preserving the efficiency of SFT.
Our extensive experiments across visual reasoning, medical VQA, fine-grained recognition, and CommonsenseQA demonstrate that RAFT consistently outperforms SFT and CoT-FT baselines, while maintaining SFT-level inference efficiency. Beyond accuracy, we provide the systematic analysis of RAFT’s \emph{scalability and robustness}: (i) performance improves monotonically with stronger teachers (3B–GPT-4.1), and (ii) RAFT remains effective even with noisy teacher supervision. Compared against preference-optimisation baselines, RAFT delivers complementary advantages by distilling reasoning rather than preferences.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 23947