Prioritized training on points that are learnable, worth learning, and not yet learned

Sören Mindermann, Muhammed Razzak, Mrinank Sharma, Jan M. Brauner, Winnie Xu, Andreas Kirsch, Aidan Gomez, Benedikt Höltgen, Sebastian Farquhar, Yarin Gal

Keywords: Data selection, subset selection, deep learning, active learning

Abstract: We introduce reducible held-out loss selection (RHOLS), a technique for faster model training which selects a sequence of training points that are “just right”. We propose a tractable information-theoretic acquisition function—the reducible heldout loss—to efficiently choose training points that maximize information about a holdout set. We show that the “hard” (e.g. high loss) points usually selected in the optimization literature are typically noisy, leading to deterioration on real-world datasets. At the same time, “easy” (e.g. low noise) samples, often prioritized for curriculum learning, confer less information. In contrast, RHOLS chooses points that are “just right” and trains in fewer steps than the above approaches.

One-sentence Summary: We show why prioritizing difficult samples for faster training can fail, and that prioritizing difficult but learnable samples accelerates training.