CryptoRNN - Privacy-Preserving Recurrent Neural Networks Using Homomorphic Encryption

Maya Bakshi, Mark Last

Abstract: Recurrent Neural Networks (RNNs) are used extensively for mining sequential datasets. However, performing inference over an RNN model requires the data owner to expose his or her raw data to the machine learning service provider. Homomorphic encryption allows calculations to be performed on ciphertexts, where the decrypted result is the same as if the calculation has been made directly on the plaintext. In this research, we suggest a Privacy-Preserving RNN–based inference system using homomorphic encryption. We preserve the functionality of RNN and its ability to make the same predictions on sequential data, within the limitations of homomorphic encryption, as those obtained for plaintext on the same RNN model. In order to achieve this goal, we need to address two main issues. First, the noise increase between successive calculations and second, the inability of homomorphic encryption to work with the most popular activation functions for neural networks (sigmoid, ReLU and tanh). In this paper, we suggest several methods to handle both issues and discuss the trade-offs between the proposed methods. We use several benchmark datasets to compare the encrypted and unencrypted versions of the same RNN in terms of accuracy, performance, and data traffic.