Feature Attribution for Label-Specific Feature Selection in Multi-Label Learning

Jiaxuan Li, Xiaoyan Zhu, Jiayin Wang

Keywords: Multi-Label Learning, Feature Selection

TL;DR: An extremely efficient multi-label feature selection algorithm

Abstract: Multi-label dimensionality reduction is an important but difficult problem. Feature selection approach searches a feature subset for multiple labels, which preserves label correlations well; however, not all features are necessary for specific labels. Label-specific approach constructs diverse feature spaces for different labels, paying more concentration on label characteristics but facing the complexity pressure arising from the number of labels. In this paper, a feature attribution based label-specific feature selection method is proposed, striking a balance between efficiency and accuracy. Feature attribution quantifies the contribution of every feature to the prediction, which commonly can be achieved with the gradient of deep learning output with respect to input. For multi-label learning, a simple shallow neural network is sufficient to construct a multi-input multi-output mapping, where the gradient of each label with respect to each input feature can be used for label-specific feature ranking and selection. Compared to state-of-the-art multi-label feature selection methods, the proposed method achieves comparable or superior performance while requiring less than 10\% of their runtime in most cases.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 7600