A Constrained Generative Approach to Generalized Zero-shot Object Recognition

Debasmit Das, C. S. George Lee

Abstract: Generative adversarial network (GAN)-based methods for zero-shot learning (ZSL) use the base category data as output and the semantic descriptors as conditional input for training so that the network can synthesize data for the novel classes. Using these generated data, the ZSL problem is converted into a supervised learning problem. Since the GAN is trained on only the base categories, the model prediction is biased towards the base categories. Also, the generated data for the novel categories might not accurately represent the ground truth. To address these problems, we propose a three-way solution. Firstly, we constrain the generation process such that the generated data from the novel classes can be highly discriminated from that of the base classes. This constraint tries to get rid of the biasness problem. Secondly, we enforce semantic consistency by reconstructing the semantic attributes from the generated data. Finally, we selectively adapt and transform the generated data from the novel classes to be close to the ground-truth unlabeled test data. We evaluated our framework on five standard datasets for ZSL and found our method to be highly competitive when compared with previous work. We also carried out additional studies to better understand our framework.