Estimating Transition Matrix with Diffusion Models for Instance-Dependent Label Noise

Haixin Yang, Ruirui Li, Xiangzhong Fang, Yukun Yang, Naihao Wang

Keywords: Transition Matrix, Label Noise, Diffusion Models

TL;DR: The paper proposes a method that models the transition matrix using diffusion models to handle instance-dependent label noise problem.

Abstract: Learning with noisy labels is a common problem in weakly supervised learning, where the transition matrix approach is a prevalent method for dealing with label noise. It estimates the transition probabilities from a clean label distribution to a noisy label distribution and has garnered continuous attention. However, existing transition matrix methods predominantly focus on class-dependent noise, making it challenging to incorporate feature information for learning instance-dependent label noise. This paper proposes the idea of using diffusion models for estimating transition matrix in the context of instance-dependent label noise. Specifically, we first estimate grouped transition matrices through clustering. Then, we introduce a process of adding noise and denoising with the transition matrix, incorporating features extracted by unsupervised pre-trained models. The proposed method enables the estimation of instance-dependent transition matrix and extends the application of transition matrix method to a broader range of noisy label data. Experimental results demonstrate the significant effectiveness of our approach on both synthetic and real-world datasets with instance-dependent noise. The code will be open sourced upon acceptance of the paper.

Primary Area: Diffusion based models

Submission Number: 18102