Decentralized Parallel SGD Based on Weight-Balancing for Intelligent IoV

Yuan Yuan, Jiguo Yu, Xiaolu Cheng, Zongrui Zou, Dongxiao Yu, Zhipeng Cai

Abstract: Training machine learning models in a decentralized way has attracted tremendous attention on intelligent Internet of Vehicles (IIoV). However, it is highly dynamic and asymmetric for the connections between vehicles in IIoV due to the mobility of vehicles and the complex communication environment, which poses great challenges on designing efficient distributed learning algorithms. To address this problem, we focus on the basic stochastic gradient descent (SGD) algorithm and propose a decentralized parallel SGD algorithm (DPSGD-WB) for the complex IIoV. The algorithm is based on weight-balancing to overcome the difficulty caused by the dynamic and asymmetric connectivity in IIoV. With rigorous analysis, we show that DPSGD-WB converges on the optimal rate of $O(1/\sqrt {Kn})$ , where $n$ is the number of vehicle terminals and $K$ is the number of iterations. To the best of our knowledge, our proposed algorithm is the first known decentralized parallel SGD algorithm that can be implemented in asymmetric and dynamic intelligent IoV systems. Finally, extensive experiments demonstrate the efficacy of our algorithm.