CPQS-Tuning: Model Perception-Based Fast and Efficient Instruction Fine-Tuning

ACL ARR 2025 February Submission3503 Authors

Abstract: Instruction fne-tuning is a key technique for improving the performance of large language models (LLMs), but it is often limited by low-quality and redundant data. Recent studies suggest that a small amount of high-quality data can yield signifcant performance gains.However, existing data selection methods may not be suitable for rapidly evolving LLMs, and each model may require distinct types of high-quality data. To address these issues, we pro-pose a novel perspective: the hidden states of LLMs implicitly assess the quality of the train-ing data. Building on this insight, we introduce a new instruction-tuning data evaluation met-ric, the Contrastive Perception Quality Score (CPQS), and a dataset fltering approach based on this metric. Experimental results demon-strate that, when trained on less than 2% of the dataset (1,000 samples) from the general-purpose Alpaca and Alpaca_GPT4 datasets,our method outperforms models trained on the full dataset. Furthermore, our approach surpasses current state-of-the-art data selec-tion methods in terms of performance. In downstream tasks, our method achieves an av-erage performance improvement of over 3%compared to models trained with leading data selection algorithms, across multiple bench-mark tests, including GSM8K, HumanEval,and HumanEval-Plus.

Paper Type: Long

Research Area: Efficient/Low-Resource Methods for NLP

Research Area Keywords: fficient/Low-Resource Methods for NLP

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches low compute settings-efficiency, Publicly available software and/or pre-trained models, Data analysis

Languages Studied: English

Submission Number: 3503