Automated Data Curation for Robust Language Model Fine-Tuning

ACL ARR 2024 April Submission487 Authors

Abstract:

Large Language Models have become the de facto approach to sequence-to-sequence text generation tasks, but for specialized tasks/domains, a pretrained LLM lacks specific capabilities to produce accurate or well-formatted responses. Supervised fine-tuning specializes a LLM by training it on dataset of example prompts with target responses, but real-world data tends to be noisy. While many fine-tuning algorithms exist, here we consider a \emph{data-centric AI} perspective on LLM fine-tuning, studying how to \emph{systematically} curate the training dataset to improve the LLM produced via \emph{any} fine-tuning algorithm.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Data curation; LLM

Languages Studied: English

Submission Number: 487