Maximum margin binary classifiers using intrinsic and penalty graphs

Berkay Kicanaoglu, Alexandros Iosifidis, Moncef Gabbouj

Abstract: In this paper a variant of the binary Support Vector Machine classifier that exploits intrinsic and penalty graphs in its optimization problem is proposed. We show that the proposed approach is equivalent to a two-step process where the data is firstly mapped to an optimal discriminant space of the input space and, subsequently, the original SVM classifier is applied. Our approach exploits the underlying data distribution in a discriminant space in order to enhance SVMs generalization ability. We also extend this idea to the Least Squares SVM classifier, where the adoption of the intrinsic and penalty graphs acts as a regularizer incorporating discriminant information in the overall solution. Experiments on standard and recently introduced datasets verify our analysis since, in the cases where the classes forming the problem are not well discriminated in the original feature space, the exploitation of both intrinsic and penalty graphs enhances performance.