A consensus-based decentralized algorithm for non-convex optimization with application to dictionary learning

Hoi-To Wai, Tsung-Hui Chang, Anna Scaglione

Abstract: In handling massive-scale signal processing problems arising from `big-data' applications, key technologies could come from the development of decentralized algorithms. In this context, consensus-based methods have been advocated because of their simplicity, fault tolerance and versatility. This paper presents a new consensus-based decentralized algorithm for a class of non-convex optimization problems that arises often in inference and learning problems, including `sparse dictionary learning' as a special case. For the proposed algorithm, we provide sufficient conditions for convergence to a stationary point. Numerical results demonstrate the efficacy of the proposed algorithm and provide evidence that validates our convergence claim.