Leveraging Attribute Knowledge for Open-set Action Recognition

Kaixiang Yang, Junyu Gao, Yangbo Feng, Changsheng Xu

Abstract: Open-set action recognition(OSAR) aims to recognize known classes and reject unknown classes. Most OSAR methods focus on learning a favorable threshold to distinguish known and unknown samples in a pure data-driven manner. However, these methods do not utilize the prior knowledge of action classes. In this paper, we propose to Leverage Attribute Knowledge (LAK) for OSAR. Specifically, the class-attribute knowledge learning is designed to integrate attribute knowledge into the model based on spatial-temporal features. Here, attributes are used as a bridge, linking known and unknown classes implicitly to make up the knowledge gap. Furthermore, a learnable relation matrix is adaptively adjusted during training to obtain the class-attribute relations that are expected to be generalized in open-set settings. Extensive experiments on three popular datasets show that the proposed method achieves state-of-the-art performance.