From Comparison to Composition: Towards Understanding Machine Cognition of Unseen Categories

Minghao Fu, Sheng Zhang, Guangyi Chen, Zijian Li, Fan Feng, Yifan Shen, Shaoan Xie, Kun Zhang

Keywords: Concept Learning; Cognition Science; Novel Category Discovery;

TL;DR: A provably representation learning framework for unseen category cognition with providing theoretical guarantees and estimation framework through comparison and composition.

Abstract: Humans are known to acquire and generalize visual concepts through a natural compare–then–compose process. We ask whether this mechanism can provide principled conditions under which machines generalize existing knowledge to unseen categories. In this work, we formalize cognition of the unseen as two complementary mechanisms for deep learning models: comparison, which uncovers latent concepts by capturing cross-category variations among seen classes, and composition, which extrapolates these concepts continuously to unseen classes. Even without parametric assumptions, we establish identifiability guarantees for learning latent concepts and unseen categories via sufficient contrast and independent support separation, denoted as Comparison–C}omposition Cognition (C^3). Guided by these results, we instantiate a structurally constrained generative model mirroring our theoretical assumptions. Our results on simulated data corroborate our theoretical claims and the effectiveness of our proposed methodology. In the setting of visual cognition with unseen labels, aka On-the-fly Category Discovery, our instantiated approach improves state-of-the-art baselines by +3.8\% average accuracy across fine-grained benchmarks. Taken together, our framework offers principled conditions and practical guidance for representational compositionality, offering a theory-to-practice path for generalization to unseen categories.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 9939