DOES THE DEFINITION OF DIFFICULTY MATTER ? SCORING FUNCTIONS AND THEIR ROLE FOR CURRICULUM LEARNING

ICLR 2026 Conference Submission12623 Authors

Keywords: Curriculum Learning, Sample Difficulty, Scoring Function Similarity, Computer Vision, Computer Audition, Deep Learning

Abstract: Curriculum learning (CL) relies on the simple and intuitive assumption that the non-uniform sampling of training instances based on some measure of sample difficulty is beneficial for learning, with the postulated benefits being faster convergence and improved test-set performance. The motivation for CL is oftentimes grounded on anthropomorphisation – humans, it is argued, often rely on curricula for their learning. However, this simple premise hinges on the notion of sample difficulty for which there is no established definition. Previous research on the benefits of CL begins by settling on a specific definition of difficulty, without questioning the potential bias that this a priori definition introduces. In the present contribution, we conduct an extensive experimental study on the robustness and similarity of the most common scoring functions for sample difficulty estimation on two benchmark datasets from the vision and audio domains. We report a strong dependence of scoring functions on the training hyperparameters, including randomness, which can partly be mitigated through ensemble scoring. While we do not find a general advantage of CL over uniform sampling, we observe that the ordering in which data is presented for CL-based training plays an important role in model performance. Furthermore, we find that the robustness of scoring functions across random seeds positively correlates with CL performance. Finally, we uncover that models trained with different CL strategies complement each other by boosting predictive power through late fusion, likely due to differences in the learnt concepts. Alongside our findings, we release a toolkit implementing sample difficulty and CL-based training in a modular fashion.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 12623