CiNet: Redesigning Deep Neural Networks for Efficient Mobile-Cloud Collaborative Inference

Xin Dai, Xiangnan Kong, Tian Guo, Yixian Huang

Abstract: Deep neural networks are increasingly used in end devices such as mobile phones to support novel features, e.g., image classification. Traditional paradigms to support mobile deep inference fall into either cloud-based or on-device—both require access to an entire pre-trained model. As such, the efficacy of mobile deep inference is limited by mobile network conditions and computational capacity. Collaborative inference, a means to splitting inference computation between mobile devices and cloud servers, was proposed to address the limitations of traditional inference through techniques such as image compression or model partition. In this paper, we improve the performance of collaborative inference from a complementary direction, i.e., through redesigning deep neural networks to satisfy the collaboration requirement from the outset. Specifically, we describe the design of a collaboration-aware convolutional neural network, referred to as CiNet, for image classification. CiNet consists of a mobile-side extractor submodel that outputs a small yet relevant patch of the image and a cloud-based submodel that classifies on the image patch. We evaluated the efficiency of CiNet in terms of inference accuracy, computational cost and mobile data transmission on three datasets. Our results demonstrate that CiNet achieved comparable inference accuracy while incurring orders of magnitude less computational cost and 99% less transmitted data, when comparing to both traditional and collaborative inference approaches.