Parallel FHE-Based Neural Network Inference with Knowledge Distillation for Efficient Privacy-Preserving Image Classification

Dian Jiao, Junyu Lin, Jiageng Chen, Jichao Xiong, Weizhi Meng, Chunhua Su

Abstract: We present a novel parallel FHE-based inference framework specifically designed for encrypted image classification, substantially improving computational efficiency while preserving high accuracy. Although Fully Homomorphic Encryption (FHE) provides strong privacy guarantees, its computational overhead typically hinders real-world applicability. To address this issue, we propose three key innovations: (1) a patch-level parallel inference architecture that partitions images for simultaneous encrypted computation, (2) an overlapping patch mechanism integrated with a dual-stage knowledge distillation pipeline to mitigate global feature loss while maintaining efficient encrypted inference, and (3) use of the CKKS scheme, which natively supports floating-point arithmetic and SIMD operations. Experiments on benchmark datasets highlight the effectiveness of our approach: on MNIST, it achieves 99.05% accuracy with a latency of only 1.09 s, while on CIFAR-10, it attains 82.74% accuracy in 47.3 s of encrypted inference. Compared with classical baselines such as CryptoNets and CryptoDL, our framework reduces latency by 35.9\(\times \) \(\sim \) 293.6\(\times \), and it outperforms cutting-edge solutions (LoLa, Falcon, bi-CryptoNets) by a 1.1\(\times \) \(\sim \) 2.3\(\times \) speedup, while maintaining comparable or acceptable accuracy. Overall, this work advances practical Privacy-Preserving Machine Learning (PPML) by striking a favorable balance among accuracy, latency, and data privacy, paving the way for secure, real-world machine learning deployments.

External IDs:dblp:conf/ksem/JiaoLCXMS25