Webly-supervised zero-shot learning for artwork instance recognition.

Riccardo Del Chiaro, Andrew D. Bagdanov, Alberto Del Bimbo

Abstract: Highlights • We extend our previous work on webly-supervised learning for artwork instance recognition on the NoisyArt dataset. • Experiments show that normalization of visual representations boosts robustness to noise in webly-supervised learning. • Our results on webly-supervised instance recognition significantly outperform earlier techniques. • Zero-shot learning experiments show that unseen instances can be recognized even when using webly-supervised training data. Abstract This paper describes experiments on supervised approaches to webly-labeled artwork instance recognition and zero-shot learning for unseen artwork instance recognition. We build on our earlier work on webly-supervised learning using the NoisyArt dataset. The dataset consists of more than 90,000 images and in more than 3,000 webly-supervised classes, and a subset of 200 classes with verified test images. Document embeddings are provided for short descriptions of all artworks. NoisyArt is designed to support research on webly-supervised artwork instance recognition, zero-shot learning, and other approaches to visual recognition of cultural heritage objects. We report results of experiments on artwork instance recognition using the NoisyArt dataset of webly-labeled images as well as on the CMU-Oxford Sculptures dataset. In addition, we perform extensive experiments on zero-shot learning using webly-labeled training images for unseen artwork recognition. Our results demonstrate the benefits and limitations of zero-shot learning for instance recognition over webly-supervised data. Previous article in issue Next article in issue MSC 41A05 41A10 65D05 65D17 Recommended articles Citing articles (0) View full text © 2019 Elsevier B.V. All rights reserved.