CurL-AutoML: Curriculum Learning-based AutoML

Lucas Nildaimon dos Santos Silva, Lucas Cardoso Silva, Fernando Rezende Zagatti, Bruno Silva Sette, Helena de Medeiros Caseli, Daniel Lucrédio, Diego Furtado Silva

Abstract: AutoML aims to find the best Machine Learning (ML) pipeline in a complex and high-dimensional search space by evaluating multiple algorithm configurations. However, training multiple ML algorithms is time-consuming, and as AutoML tools are frequently time-constrained, the exploration of the search space may find sub-optimal results. In this work, we explore the application of curriculum learning techniques to overcome this limitation. Curriculum and anti-curriculum learning have improved model performance and accelerated the training process on previous empirical investigations using optimization-based models by ordering examples during model training based on their difficulty. We apply and compare curriculum strategies on an AutoML system to accelerate the search space exploration and find good-performing machine learning pipelines efficiently. The results indicate that AutoML can benefit from a curriculum strategy. Furthermore, in most of the evaluated scenarios, the curriculum strategies led to better classification results.