Incorporating Implicit Regularization to Enhance the Transition Matrix Method for Effective Handling of Diverse Label Noise

Haixin Yang, Ruirui Li, Xiangzhong Fang, Yukun Yang, Naihao Wang

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Learning with Noisy Labels, Transition Matrix, Implicit Regularization, Theoretical Analysis

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a concise transition matrix method in conjunction with implicit regularization to handle learning from data with diverse label noise and provide theoretical analysis on the effectiveness of the method.

Abstract: Among various methods for learning with noisy labels, the transition matrix method has attracted sustained attention due to its simplicity and statistical consistency. However, estimating the transition matrix for each sample may be unidentifiable and computationally expensive in the case of instance-dependent label noise and real-world situations. In this paper, we propose a concise method that only requires estimating a global matrix, combining with implicit regularization, to replace the estimation of the individual transition matrix for each sample. Specifically, by estimating the transition matrix, we can determine the overall probability transfer from correct labels to noisy labels and use implicit regularization to adjust the sparse form representation of the difference between the estimated posterior probability distribution and the noisy label distribution. This approach can be applied to diverse types of noise as well as alleviating the problem of inaccurate posterior probability estimation. We theoretically analyze the consistency and generalization results of the proposed method and conduct experiments on synthetic and real-world datasets with different types of label noise. The experimental results show that our method significantly outperforms previous transition matrix methods and has a wider range of applicability. Additionally, our method achieves impressive results without the need for additional auxiliary techniques. Our code will be open source and put on Github.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2477