Noise Tolerance and Robustness Ranking in Machine Learning Models

Cristina Padró-Ferragut, María José Ramírez-Quintana, Fernando Martínez-Plumed

Abstract: Machine Learning models often excel in controlled environments but may struggle with noisy, incomplete, or shifted real-world data. Ensuring that these models maintain high performance despite these imperfections is crucial for practical applications, such as medical diagnosis or autonomous driving. This paper introduces a novel framework to systematically analyse the robustness of Machine Learning models against noisy data. We propose two empirical methods: (1) Noise Tolerance Estimation, which calculates the noise level a model can withstand without significant degradation in performance, and (2) Robustness Ranking, which ranks Machine Learning models by their robustness at specific noise levels. Utilizing Cohen’s kappa statistic, we measure the consistency between a model’s predictions on original and perturbed datasets. Our methods are demonstrated using various datasets and Machine Learning techniques, identifying models that maintain reliability under noisy conditions.