KNN-enhanced Deep Learning Against Noisy Labels

Shuyu Kong, You Li, Jia Wang, Amin Rezaei, Hai Zhou

Abstract:

Supervised learning on Deep Neural Networks (DNNs) is
data hungry. Optimizing performance of DNN in the presence of noisy labels has become of paramount importance
since collecting a large dataset will usually bring in noisy
labels. Inspired by the robustness of K-Nearest Neighbors
(KNN) against data noise, in this work, we propose to apply deep KNN for label cleanup. Our approach leverages
DNNs for feature extraction and KNN for ground-truth label inference. We iteratively train the neural network and
update labels to simultaneously proceed towards higher
label recovery rate and better classification performance.
Experiment results show that under the same setting, our
approach outperforms existing label correction methods
and achieves better accuracy on multiple datasets, e.g.,
76.78% on Clothing1M dataset.