A review of AI edge devices and lightweight CNN and LLM deployment

Kailai Sun, Xinwei wang, Qianchuan Zhao

Abstract: Artificial Intelligence of Things (AIoT) which integrates artificial intelligence (AI) and the Internet of Things (IoT), has attracted increasing attention recently. With the remarkable development of AI, convolutional neural networks (CNN) have achieved great success from research to deployment in many applications. However, deploying complex and state-of-the-art (SOTA) AI models on edge applications is increasingly a big challenge. This paper investigates literature that deploys lightweight CNNs on AI edge devices in practice. We provide a comprehensive analysis of them and many practical suggestions for researchers: how to obtain/design lightweight CNNs, select suitable AI edge devices, and compress and deploy them in practice. Finally, future trends and opportunities are presented, including the deployment of large language models, trustworthy AI and robust deployment.