AIoT Bench: Towards Comprehensive Benchmarking Mobile and Embedded Device Intelligence

Chunjie Luo, Fan Zhang, Cheng Huang, Xingwang Xiong, Jianan Chen, Lei Wang, Wanling Gao, Hainan Ye, Tong Wu, Runsong Zhou, Jianfeng Zhan

Abstract: Due to increasing amounts of data and compute resources, the deep learning achieves many successes in various domains. Recently, researchers and engineers make effort to apply the intelligent algorithms to the mobile or embedded devices. In this paper, we propose a benchmark suite, AIoT Bench, to evaluate the AI ability of mobile and embedded devices. Our benchmark (1) covers different application domains, e.g. image recognition, speech recognition and natural language processing; (2) covers different platforms, including Android and Raspberry Pi; (3) covers different development frameworks, including TensorFlow and Caffe2; (4) offers both end-to-end application workloads and micro workloads.