Capped ℓp-Norm LDA for Outliers Robust Dimension Reduction

Zheng Wang, Feiping Nie, Canyu Zhang, Rong Wang, Xuelong Li

Abstract: Linear discriminant analysis technique is an effective strategy to solve the long-standing issue, i.e., the “curse of dimensionality” that brings many obstacles on high-dimensional data storage and analysis. However, the projections are prone to be affected, especially when the training set contains outlier samples whose distribution deviates from the globality. In many real-world applications, the outlier samples contaminated by noisy signal or spottiness have negative effects on the classification and clustering performance. To address this issue, we propose to develop a novel capped ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -norm LDA model for robust dimension reduction against to outliers specifically. Proposed method integrates the capped ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -norm based loss into the objective, which not only suppresses the light outliers but also works well even though the training set is contaminated seriously. Furthermore, we derive an alternative iterative re-weighted optimization algorithm to minimize the proposed objective based on capped ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</sub> -norm with rigorous convergence proofs. Extensive experiments conducted on synthetic and real-world datasets demonstrate the robustness against to outliers of proposed method.