Image Privacy Protection by Particle Swarm Optimization Based Pivot Pixel Modification

Jishen Yang, Yan Huang, Junjie Pang, Zhenzhen Xie, Wei Li

Abstract: The image classification models based on neural networks recently have outperformed most of the traditional models, and rapidly been developed and implemented by industry because of the capability of qualifying various computer vision tasks. Hence, the exposure of users’ image data to unauthorized powerful models causes more information leak in a shorter time. Through experiments, we find that for each input image, the change of the image’s prediction scores by each pixels’ RGB value change is different. Also, the pattern of the sensitivity on each pixel is highly related to the category and composition of the input image. By utilizing this feature, we present Pivot Pixel Noise Generator by Particle Swarm Optimization to generate noise points on original images to lower the target model’s accuracy of correctly predicting the target image’s label, so to protect the information contained in the target image from the image classification models. The model performs in a semi-black-box manner and balances the number of queries to the target and total number of modified points. We also propose an initialization strategy for the model, PSO Knowledge Transfer, which initializes the model’s parameters with experience learned from previous runs to further reduce the number of query times and noise points. The model is evaluated using the image classification benchmark model ResNet50 and shows an advantage compared to the baseline algorithm.