Triadic Elastic Structure Representation for Open-Set Incremental 3D Object Retrieval

Yang Xu, Yifan Feng, Lin Bie

Abstract: In open-set environments, the introduction of unseen classes adds a third stage of data for incremental learning, alongside seen classes of old and new. Existing methods by binary optimization between old and new classes struggle to model the complex triadic relationships among old, new, and unseen classes. In this paper, we introduce the Triadic Elastic Structure Learning (TESR) framework for open-set incremental 3D object retrieval. Specifically, to overcome the Bi-Directional Catastrophic Forgetting (BDCF) problem in open-set incremental learning, we employ the Triadic Regularized Embedding (TRE) module. This module helps to globally preserve the triadic relationships among the three stages of classes, while simultaneously facilitating semantic-specific learning for new classes from a local perspective. To address the challenge of Bi-Directional Knowledge Transfer (BDKT), our method leverages high-order correlations among objects of different classes through the Elastic Structure Distillation (ESD) module, by constructing an elastic hypergraph based on incremental and global correlations. We construct four multi-modal datasets for this open-set incremental 3D object retrieval: OIES, OINT, OIMN, and OIAB. Extensive experiments and ablation studies on these four benchmarks demonstrate the superiority of our method over current state-of-the-art approaches.