GREATS: Online Selection of High-Quality Data for LLM Training in Every Iteration

Jiachen T. Wang, Tong Wu, Dawn Song, Prateek Mittal, Ruoxi Jia

Keywords: Online Batch Selection, Large Language Model

TL;DR: We develop a principled and efficient online batch selection algorithm that is scalable to large language models.

Abstract: Online batch selection methods offer an adaptive alternative to static training data selection by dynamically selecting data batches during training. However, existing methods either rely on impractical reference models or simple heuristics that may not capture true data informativeness. To address these limitations, we propose \emph{GREedy Approximation Taylor Selection} (GREATS), a principled and efficient online batch selection method that applies greedy algorithm to optimize the data batch quality approximated by Taylor expansion. We develop a series of techniques to scale GREATS to large-scale model training. Extensive experiments with large language models (LLMs) demonstrate that GREATS significantly improves training convergence speed and generalization performance.

Primary Area: Interpretability and explainability

Submission Number: 20576