Sparse Kernel-Based Feature Weighting

Shuang-Hong Yang, Yujiu Yang, Bao-Gang Hu

Abstract: The success of many learning algorithms hinges on the reliable selection or construction of a set of highly predictive features. Kernel-based feature weighting bridges the gap between feature extraction and subset selection. This paper presents a rigorous derivation of the Kernel-Relief algorithm and assesses its effectiveness in comparison with other state-of-art techniques. For practical considerations, an online sparsification procedure is incorporated into the basis construction process by assuming that the kernel bases form a causal series. The proposed sparse Kernel-Relief algorithm not only produces nonlinear features with extremely sparse kernel expressions but also reduces the computational complexity significantly.