Group feature selection using non-class data

Chunna Li, Yuangang Pan, Weijie Chen, Ivor W. Tsang, Yuanhai Shao

Abstract: Existing embedded feature selection methods barely let non-class data contribute to feature selection. However, in some learning tasks, when non-class data have contribution to classification, they should also have an influence to the selection of useful features. For instance, \(F_\infty\)-norm support vector machine is an effective embedded group feature selection method that performs classification simultaneously. In this paper, we find out that it implicitly uses a kind of non-class data formulated as coordinate Universum when implementing group feature selection, and the information contained in this non-class data could be a meaningful group-wise \(F_{\infty }\)-norm penalization. As far as we know, this is the first time that \(F_{\infty }\)-norm penalization is understood from this angle. We prove that useful features can be identified through this non-class data that contribute to classifier construction. In addition, to fully explore the classification information provided by this non-class data, we improve \(F_\infty\)-norm support vector machine by deeming the non-class data as a middle class to better classify positive and negative classes. Experiments show that the non-class data in the proposed method help reduce the labelled data in some sense. Furthermore, it improves \(F_\infty\)-norm support vector machine in terms of both classification and group feature selection.

External IDs:dblp:journals/ml/LiPCTS25