Active Instruction Tuning for Large Language Models with Reference-Free Instruction Selection
Abstract: Recent works on efficient instruction tuning have shown that large language models (LLMs) can achieve comparable performance through the calibrated selection of a small subset of high-quality (instruction, response) pairs from labeled instruction pools. Despite reduced computational costs, these approaches often overlook the labor-intensive nature of instruction acquisition for labeling. We introduce a novel paradigm, Active Instruction Tuning with Reference-Free Instruction Selection, which supports instruction selection from both labeled and unlabeled instruction pools. Our experimental results demonstrate that this method not only achieves comparable or superior performance while reducing labeling costs but also matches the performance of prior studies in labeled instruction settings. Furthermore, we pioneer the investigation into the relationship between text evaluation correlated with human subjective evaluations and instruction tuning, confirming the effectiveness of ranking aggregation in enhancing the tuning.
Paper Type: Long
Research Area: Efficient/Low-Resource Methods for NLP
Research Area Keywords: data-efficient training;NLP in resource-constrained settings
Contribution Types: Approaches to low-resource settings
Languages Studied: English
Submission Number: 2234
Loading