Deep Self-Dissimilarities as Powerful Visual Fingerprints

Idan Kligvasser, Tamar Rott Shaham, Yuval Bahat, Tomer Michaeli

Keywords: descriptors, deep-features, IQA, image-restoration, super-resolution, motion-deblurring

Abstract: Features extracted from deep layers of classification networks are widely used as image descriptors. Here, we exploit an unexplored property of these features: their internal dissimilarity. While small image patches are known to have similar statistics across image scales, it turns out that the internal distribution of deep features varies distinctively between scales. We show how this deep self dissimilarity (DSD) property can be used as a powerful visual fingerprint. Particularly, we illustrate that full-reference and no-reference image quality measures derived from DSD are highly correlated with human preference. In addition, incorporating DSD as a loss function in training of image restoration networks, leads to results that are at least as photo-realistic as those obtained by GAN based methods, while not requiring adversarial training.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: We exploit deep features dissimilarity, and suggest to use it as both image quality measure and as a loss function for image restoration tasks

Code: https://github.com/kligvasser/DSD