Weaken Noisy Feature: Boosting Semi-supervised Learning by Noise Estimation

Junjiang Liu, Dandan Sun, Hailun Xia, Jiangtao Bai, Xinyue Fan

Abstract: Due to poor bias calibration, current semi-supervised learning (SSL) is over-confident in false predictions. Modeling data noise helps alleviate confirmation bias. Specifically, we consider a posteriori knowledge that the data embedding follows a Gaussian distribution. We let the model predict and learn the mean (classification features) and variance (noise features) for each sample. In this way, the network can estimate and isolate noisy information in the latent space. Additionally, we propose Noise Estimation Curriculum Learning (NECL) to fit differences in noise level between classes. NECL can encourage the model to learn from hard classes actively. Extensive experiments demonstrate the effectiveness of our method, which is also orthogonal to FixMatch-based frameworks. InfoMatch with our components (NE-Info) showed state-of-the-art (SOAT) performance on several benchmarks such as CIFAR-10/100, SVHN, STL-10 and ImageNet.