Distributed empirical risk minimization over directed graphs

Ran Xin, Anit Kumar Sahu, Soummya Kar, Usman A. Khan

Abstract: In this paper, we present stochastic optimization for empirical risk minimization over directed graphs. Using a novel information fusion approach that utilizes both row- and column-stochastic weights simultaneously, we propose SAB, a decentralized stochastic gradient method with gradient tracking, and show that the proposed algorithm converges linearly to an error ball around the optimal solution with a constant step-size. We provide a sketch of the convergence analysis as well as the generalization of the proposed algorithm. Finally, we illustrate the theoretical results with the help of experiments with real data.