Distributed Optimal Margin Distribution Machine

Yilin Wang, nan cao, Teng Zhang, Hai Jin

Keywords: Distributed machine learning, margin distribution, classification, kernel learning

Abstract: Optimal margin Distribution Machine (ODM), a newly proposed statistical learning framework rooting in the novel margin theory, demonstrates better generalization performance than the traditional large margin based counterparts. Nonetheless, the same with other kernel methods, it suffers from the ubiquitous scalability problem in terms of both computation time and memory. In this paper, we propose a Distributed solver for ODM (DiODM), which leads to nearly ten times speedup for training kernel ODM. It exploits a novel data partition method to make the local ODM trained on each partition has a solution close to the global one. When linear kernel used, we extend a communication efficient distributed SVRG method to further accelerate the training. Extensive empirical studies validate the superiority of our proposed method compared to other off-the-shelf distributed quadratic programming solvers for kernel methods.

One-sentence Summary: We propose a specially designed Distributed solver for ODM, a novel margin based model, which maintains great generalization performance and achieves nearly ten times speedup simultaneously