Deep Partial Updating

Zhongnan Qu, Cong Liu, Junfeng Guo, Lothar Thiele

Keywords: Partial updating, communication constraints, server-to-edge, deep neural networks

Abstract: Emerging edge intelligence applications require the server to continuously retrain and update deep neural networks deployed on remote edge nodes to leverage newly collected data samples. Unfortunately, it may be impossible in practice to continuously send fully updated weights to these edge nodes due to the highly constrained communication resource. In this paper, we propose the weight-wise deep partial updating paradigm, which smartly selects only a subset of weights to update at each server-to-edge communication round, while achieving a similar performance compared to full updating. Our method is established through analytically upper-bounding the loss difference between partial updating and full updating, and only updates the weights which make the largest contributions to the upper bound. Extensive experimental results demonstrate the efficacy of our partial updating methodology which achieves a high inference accuracy while updating a rather small number of weights.

One-sentence Summary: To iteratively improve the deployed deep neural network with newly collected data samples, we propose the server only sends a weight-wise partial update to edge devices to save the communication and computation resources.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2007.03071/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=4oqqEOQNA9