DISVMs: Fast SVMs Training on Large-Scale Data Sets

Lijuan Cui, Changjian Wang, Ziyang Li, Yuxing Peng

Abstract: Support Vector Machines (SVMs) are powerful classification tools. However, the model training is very time-consuming when meeting large scale data sets. Some efforts have been devoted to screening out non-support vectors (non-SVs) to accelerate the training. But their processes rely on prior knowledge of other classifiers with different parameters to screen out non-SVs. In this paper, we propose Directional Indicator Support Vector Machines (DISVMs) to efficiently identify non-SVs. DISVMs employs a directional indicator, which points to the approximately orthogonal direction of the separating hyperplane, to qualitatively define the location of different samples and thus identify non-SVs. Furthermore, DISVMs leverages a two-stage algorithm: the first stage is to compute the directional indicator. The second stage is to identify non-SVs using the indicator. To avoid misjudgement, we propose CnSV method for non-SVs based on the majority rule. DISVMs screens out non-SVs with light computation and little accuracy loss. Experiments show that our approach significantly reduces the total computation cost.