Decentralized Non-Smooth Optimization Over the Stiefel Manifold

Jinxin Wang, Jiang Hu, Shixiang Chen, Zengde Deng, Anthony Man-Cho So

Abstract: We focus on a class of non-smooth optimization problems over the Stiefel manifold in the decentralized setting, where a connected network of many agents cooperatively mini-mize a finite-sum objective function with each component being weakly convex in the ambient Euclidean space. Such optimization problems, albeit frequently encountered in applications, are quite challenging due to their non-smoothness and non-convexity. To tackle them, we propose an iterative method called the decentralized Riemannian sub gradient method (DRSM). When the problem at hand possesses a sharpness property, we show the local linear convergence of DRSM using geometrically di-minishing stepsizes. Numerical experiments are conducted to demonstrate the superior performance of DRSM in different applications.