Training with Growing Sets: A Simple Alternative to Curriculum Learning and Self Paced Learning

Melike Nur Mermer, Mehmet Fatih Amasyali

Abstract: Curriculum learning and Self paced learning are popular topics in the machine learning that suggest to put the training samples in order by considering their difficulty levels. Studies in these topics show that starting with a small training set and adding new samples according to difficulty levels improves the learning performance. In this paper we experimented that we can also obtain good results by adding the samples randomly without a meaningful order. We compared our method with classical training, Curriculum learning, Self paced learning and their reverse ordered versions. Results of the statistical tests show that the proposed method is better than classical method and similar with the others. These results point a new training regime that removes the process of difficulty level determination in Curriculum and Self paced learning and as successful as these methods.

TL;DR: We propose that training with growing sets stage-by-stage provides an optimization for neural networks.

Keywords: Neural networks, Curriculum learning, Self paced learning