Robust Secure Swap: Responsible Face Swap With Persons of Interest Redaction and Provenance Traceability

Yunshu Dai, Jianwei Fei, Fangjun Huang, Chip Hong Chang

Abstract:

As AI generative models evolve, face swap technology has become increasingly accessible, raising concerns over potential misuse. Celebrities may be manipulated without consent, and ordinary individuals may fall victim to identity fraud. To address these threats, we propose Secure Swap, a method that protects persons of interest (POI) from face-swapping abuse and embeds a unique, invisible watermark into nonPOI swapped images for traceability. By introducing an ID Passport layer, Secure Swap redacts POI faces and generates watermarked outputs for nonPOI. A detachable watermark encoder and decoder are trained with the model to ensure provenance tracing.
Experimental results demonstrate that Secure Swap not only preserves face swap functionality but also effectively prevents unauthorized swaps of POI and detects different embedded model's watermarks with high accuracy.
Specifically, our method achieves a 100% success rate in protecting POI and over 99% watermark extraction accuracy for nonPOI. Besides fidelity and effectiveness, the robustness of protected models against image-level and model-level attacks in both online and offline application scenarios is also experimentally demonstrated.

Lay Summary: This paper introduces a new method to make face-swapping technology safer and more responsible. We designed a system that prevents important individuals from having their faces swapped without permission. For other cases, our method adds invisible marks into the generated images, so it’s possible to trace where and how the image was created. This makes it easier to track fake or edited content online. Our system is strong against various attacks and can keep working even when someone tries to trick or break it. In the future, we plan to make this method work with the latest image generation tools to ensure that face-swapping is used in fair and ethical ways.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Primary Area: Applications->Computer Vision

Keywords: Face swap, generative adversarial networks, responsible and ethical AI, provenance tracing

Submission Number: 3224