TinyML-Enabled Intelligent Question-Answer Services in IoT Edge Consumer Devices

Xuan Wu, Xuanye Lin, Zhen Zhang, Chien-Ming Chen, Thippa Reddy Gadekallu, Saru Kumari, Sachin Kumar

Abstract: Presently, the prevalence of large language models has driven the rapid popularization of question-and-answer applications. However, the training and deployment of large language models involve high-resource computing, posing a challenge for many small or edge devices in the Internet of Things (IoT). Therefore, in the context of IoT edge consumer electronic devices, deploying question-and-answer models based on TinyML becomes more meaningful. In this paper, we propose a tiny deep learning-based Question-Answer scheme to realize end-to-end dialog service by Machine Reading Comprehension. In order to obtain semantic representation, we design and apply a pre-training model to generate the semantic embedding representation of questions and answers and propose a pre-training fine-tuning method based on the twin model. In addition, we also introduce a compression method based on embedded representation and design a forward compression network and a cyclic compression network based on an encoder. The experimental results show that our method is more accurate than state-of-the-art schemes.

External IDs:dblp:journals/tce/WuLZCGKK24