An Integrated Approach to Open-World Compositional Zero-Shot Learning

Hirunima Jayasekara, Khoi Pham, Nirat Saini, Abhinav Shrivastava

Keywords: Open World Compositional Zero Shot Learning, Vision Language Models

TL;DR: Paper introduces a OW-CZSL framework using a single transformer model, with a Top-K selection module to streamline pair exploration during inference and a sparse linear layer to disentangle attributes and objects, reducing computational complexity.

Abstract: Open-World Compositional Zero-Shot Learning (OW-CZSL) addresses the challenge of recognizing novel compositions of known primitives and entities. Even though prior works utilize language knowledge for recognition, such approaches exhibit limited interactions between language-image modalities. Our approach primarily focuses on enhancing the inter-modality interactions through fostering richer interactions between image and textual data. Additionally, we introduce a novel module aimed at alleviating the computational burden associated with exhaustive exploration of all possible compositions during the inference stage. While previous methods exclusively learn compositions jointly or independently, we introduce an advanced hybrid procedure that leverages both learning mechanisms to generate final predictions. Our proposed model, achieves state-of-the-art in OW-CZSL in three datasets.

Submission Number: 4