Keyword Spotting in the Homomorphic Encrypted Domain Using Convolution Decomposition

Chenyu Dong, Peijia Zheng, Weiqi Luo

Abstract: In this paper, we propose an end-to-end keyword spotting scheme that applies deep learning techniques in a homomorphic encryption domain. Leveraging the complex number encryption capability of homomorphic encryption algorithms, we introduce a complex neural network and design every modules of it, to accommodate the properties and characteristics of homomorphic encryption. Considering the relatively slow computation speed of homomorphic encryption, we employ convolution decomposition to reduce the cost of computational of the network, effectively minimizing computational time while preserving network performance. We have tested our proposed method and found that it significantly accelerates computation speed with minimal performance sacrifice when compared to the current state-of-the-art methods, thereby better balancing network performance and efficiency.