Feature Selection in Learning Using Privileged Information

Rauf Izmailov, Blerta Lindqvist, Peter Lin

Abstract: The paper considers the problem of feature selection in learning using privileged information (LUPI), where some of the features (referred to as privileged ones) are only available for training, while being absent for test data. In the latest implementation of LUPI, these privileged features are approximated using regressions constructed on standard data features, but this approach could lead to polluting the data with poorly constructed and/or noisy features. This paper proposes a privileged feature selection method that addresses some of these issues. Since not many LUPI datasets are currently available in open access, while calibration of parameters of the proposed method requires testing it on a wide variety of datasets, a modified version of the method for traditional machine learning paradigm (i.e., without privileged features) was also studied. This lead to a novel mechanism of error rate reduction by constructing and selecting additional regression-based features capturing mutual relationships among standard features. The results on calibration datasets demonstrate the efficacy of the proposed feature selection method both for standard classification problems (tested on multiple calibration datasets) and for LUPI (for several datasets described in the literature).