An Edge-Aware Transformer Framework for Image Inpainting Detection

Liangpei Hu, Yuanman Li, Jiaxiang You, Rongqin Liang, Xia Li

Abstract: Image inpainting methods based on Generative Adversarial Networks are very powerful in producing visually realistic images. It is widely used in image processing and computer vision, such as recovering damaged photos. However, image inpainting may also be maliciously used to change or delete contents, e.g. removing key objects to report fake news. Such inpainting forged images can bring serious adverse effects to society. Most existing inpainting forgery detection approaches using convolutional neural networks (CNN) have limited receptive fields and do not fully exploit the edge information of the forged regions, making them fail to effectively model the global information of forged regions and well preserve their edges. To fight against inpainting forgeries (not only deep learning (DL) based but also traditional ones), in this work, we propose an edge-aware transformer framework for image inpainting detection. To better perform feature extraction and learn discriminative features, we propose a two-stream Transformer to learn the global body features and fake edge features respectively. Further, a multi-modality cross attention module is employed to propagate their information interactively, thus greatly improving the detection results. Extensive experiments demonstrate the superiority of our scheme over existing ones, and our method exhibits desirable detection generalizability for both DL-based inpainting and traditional inpainting.