Deep Self-Dissimilarities as Powerful Visual FingerprintsDownload PDF

21 May 2021, 20:43 (edited 25 Oct 2021)NeurIPS 2021 SpotlightReaders: Everyone
  • Keywords: descriptors, deep-features, IQA, image-restoration, super-resolution, motion-deblurring
  • TL;DR: We exploit deep features dissimilarity, and suggest to use it as both image quality measure and as a loss function for image restoration tasks
  • Abstract: Features extracted from deep layers of classification networks are widely used as image descriptors. Here, we exploit an unexplored property of these features: their internal dissimilarity. While small image patches are known to have similar statistics across image scales, it turns out that the internal distribution of deep features varies distinctively between scales. We show how this deep self dissimilarity (DSD) property can be used as a powerful visual fingerprint. Particularly, we illustrate that full-reference and no-reference image quality measures derived from DSD are highly correlated with human preference. In addition, incorporating DSD as a loss function in training of image restoration networks, leads to results that are at least as photo-realistic as those obtained by GAN based methods, while not requiring adversarial training.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
  • Code:
11 Replies