Similarity-based calibration method for zero-shot recognition in multi-object scenes

Doo Soo Chang, Gun Hee Cho, Yong Suk Choi

Abstract: The objective of Zero-Shot Learning (ZSL) is to classify the class labels of unseen objects using external knowledge representing semantic information. Traditional zero-shot recognition models have the limitation that they rely only on the visual appearance of an unseen object. To alleviate this limitation, we propose a novel method that calibrates the visual prediction of an unseen object by using contextual information based on similarities between the unseen object and its surrounding seen objects in a multi-object scene. We incorporate the proposed method into each of the traditional models and conduct a comparative evaluation between the models with and without our calibration algorithm. The evaluation results show consistent performance improvements by a significant margin.