Byzantine-Tolerant Distributed Multiclass Sparse Linear Discriminant Analysis

Yajie Bao, Weidong Liu, Xiaojun Mao, Weijia Xiong

Keywords: Linear discriminant analysis, multiclass classification, distributed sparse estimation, Byzantine-robust algorithm

TL;DR: In this paper, we proposed a communication efficient distributed sparse linear discriminant analysis (Mean-DSLDA) algorithm under a normal distributed system and its Byzantine-tolerant version (Median-DSLDA) for the multi-classification problem.

Abstract: Communication cost and security issues are both important in large-scale distributed machine learning. In this paper, we investigate a multiclass sparse classification problem under two distributed systems. We propose two distributed multiclass sparse discriminant analysis algorithms based on mean-aggregation and median-aggregation under normal distributed system or Byzantine failure system. Both of them are computation and communication efficient. Several theoretical results, including estimation error bounds, and support recovery, are established. With moderate initial estimators, our iterative estimators achieve a (near-)optimal rate and exact support recovery after a constant number of rounds. Experiments on both synthetic and real datasets are provided to demonstrate the effectiveness of our proposed methods.