Curriculum Discovery through an Encompassing Curriculum Learning Framework

Mohamed Elgaar, Hadi Amiri

Keywords: curriculum learning, natural language processing

Abstract: We describe a curriculum learning framework capable of discovering optimal curricula in addition to performing standard curriculum learning. We show that this framework encompasses existing curriculum learning approaches such as difficulty-based data sub-sampling, data pruning, and loss re-weighting. We employ the proposed framework to address the following key questions in curriculum learning research: (a) What is the best curriculum to train a given model on a given dataset? (b) What are the characteristics of optimal curricula for different datasets and different difficulty scoring functions? We show that our framework outperforms competing state-of-the-art curriculum learning approaches in natural language inference and other text classification tasks. In addition, exhaustive experiments illustrate the generalizability of the discovered curricula across the three datasets and two difficulty scoring functions.

One-sentence Summary: CL framework that works by partitioning training data in three groups based on sample difficulty, developing logistic functions to effectively weight samples, and discovering new curricula that perform best for given models and datasets.