A Forensic Representation to Detect Non-Trivial Image Duplicates, and How it Applies to Semantic Segmentation

M. Cicconet, H. Elliott, D.L. Richmond, D. Wainstock, M. Walsh

Abstract: Manipulation and re-use of images in scientific publications is a recurring problem, at present lacking a scalable solution. Existing tools for detecting image duplication are mostly manual or semi-automated, despite the fact that generating data for a learning-based approach is straightforward, as we here illustrate. This paper addresses the problem of determining if, given two images, one is a manipulated version of the other by means of certain geometric and statistical manipulations, e.g. copy, rotation, translation, scale, perspective transform, histogram adjustment, partial erasing, and compression artifacts. We propose a solution based on a 3-branch Siamese Convolutional Neural Network. The ConvNet model is trained to map images into a 128-dimensional space, where the Euclidean distance between duplicate (respectively, unique) images is no greater (respectively, greater) than 1. Our results suggest that such an approach can serve as tool to improve surveillance of the published and in-peer-review literature for image manipulation. We also show that as a byproduct the network learns useful representations for semantic segmentation, with performance comparable to that of domain-specific models.

Keywords: metric learning, image similarity, image forensics, siamese network, semantic segmentation

TL;DR: A forensic metric to determine if a given image is a copy (with possible manipulation) of another image from a given dataset.