Self-Optimization Training for Weakly Supervised Image Manipulation Localization

Zhangchen Zhu, Jiafeng Li, Ying Wen

Abstract: With the continuous evolution of image manipulation techniques, there is an urgent need for an effective method to detect and localize manipulated images. However, existing fully supervised methods require large amounts of costly pixel-level annotations, whereas weakly supervised methods often fall short in localization performance due to their inability to accurately localize tampered regions with precise boundaries. To tackle this issue, we propose a Self-Optimization Weakly Supervised Localization (SO-WSL) framework, which consists of two main components: a Pseudo-Label Generator (PLG) and a Self Iterative Optimization (SIO) module. The PLG employs Class Activation Maps (CAM) to guide the Segment Anything Model (SAM) in generating pseudo-labels with distinct edges, while the SIO module enhances the model’s focus on forgery-specific features by applying masks to suspected tampered regions and iteratively refining the pseudo-labels, thereby improving localization accuracy. Extensive experiments have shown that our SO-WSL framework significantly outperforms existing weakly supervised methods and can even compete with some fully supervised approaches.

External IDs:dblp:conf/icassp/ZhuL025