Learning State-Based Node Representations from a Class Hierarchy for Fine-Grained Open-Set Detection

Spandan Pyakurel, Qi Yu

Abstract: Fine-Grained Openset Detection (FGOD) poses a fundamental challenge due to the similarity between the openset classes and those closed-set ones. Since real-world objects/entities tend to form a hierarchical structure, the fine-grained relationship among the closed-set classes as captured by the hierarchy could potentially improve the FGOD performance. Intuitively, the hierarchical dependency among different classes allows the model to recognize their subtle differences, which in turn makes it better at differentiating similar open-set classes even they may share the same parent. However, simply performing openset detection in a top-down fashion by building a local detector for each node may result in a poor detection performance. Our theoretical analysis also reveals that maximizing the probability of the path leading to the ground-truth leaf node also results in a sub-optimal training process. To systematically address this issue, we propose to formulate a novel state-based node representation, which constructs a state space based upon the entire hierarchical structure. We prove that the state-based representation guarantees to maximize the probability on the path leading to the ground-truth leaf node. Extensive experiments on multiple real-world hierarchical datasets clearly demonstrate the superior performance of the proposed method.

Lay Summary: We develop AI models capable of recognizing previously unseen image categories that differ from those present in the training data. Our approach leverages the hierarchical structure of known categories to identify instances of unknown but related categories. For example, given a training hierarchy such as Bird → Sparrow → Fox Sparrow, our goal is to correctly flag an unseen class like Le Conte’s Sparrow as an unknown subcategory of Sparrow. Prior methods typically address this challenge by classifying images at each node of the training hierarchy independently. However, we find that this approach yields limited performance. To address this, we propose a new methodology that fully exploits the hierarchical relationships among all nodes in the training taxonomy, including parent-child, ancestor-descendant, and sibling-sibling relationships. We encode these relationships through a tailored state formulation and train our model to learn from the global structure of the hierarchy, leading to significantly improved performance in recognizing novel categories.

Link To Code: https://github.com/ritmininglab/STATE-FGOD

Primary Area: Deep Learning->Robustness

Keywords: Openset detection, hierarchical classification

Submission Number: 8142