A Newton Tracking Algorithm with Exact Linear Convergence Rate for Decentralized Consensus Optimization

Jiaojiao Zhang, Qing Ling, Anthony Man-Cho So

Abstract: This paper considers the decentralized consensus optimization problem defined over a network where each node holds a twice continuously differentiable local objective function. Our goal is to minimize the summation of local objective functions and find the exact optimal solution using only local computation and neighboring communications. We propose a novel Newton tracking algorithm, in which each node updates its local variable along a local Newton direction modified with neighboring and historical information. We investigate the connections between the proposed Newton tracking algorithm and several existing methods, including gradient tracking and second-order algorithms. Under the strong convexity assumption, we prove that our proposed algorithm converges to the exact optimal solution at a linear rate. We also present numerical results to demonstrate the efficacy of Newton tracking and validate the theoretical findings.