On The Power of Deep But Naive Partial Label Learning

Joon Suk Huh, Junghoon Seo

Abstract: Partial label learning (PLL) is a class of weakly supervised learning where each training instance consists of a data and a set of candidate labels containing a unique ground truth la- bel. To tackle this problem, a majority of current state-of- the-art methods employs either label disambiguation or av- eraging strategies. So far, PLL methods without such tech- niques have been considered impractical. In this paper, we challenge this view by revealing the hidden power of the old- est and naivest PLL method when it is instantiated with deep neural networks. Specifically, we show that, with deep neu- ral networks, the naive model can achieve competitive perfor- mances against the other state-of-the-art methods, suggesting it as a strong baseline for PLL. We also address the question of how and why such a naive model works well with deep neural networks. Our empirical results indicate that deep neu- ral networks trained on partially labeled examples generalize very well even in the over-parametrized regime and without label disambiguations or regularizations. We point out that existing learning theories on PLL are vacuous in the over- parametrized regime. Hence they cannot explain why the deep naive method works. We propose an alternative theory on how deep learning generalize in PLL problems.