Real-Time Evaluation for Novel Class Discovery at Test Time

Fan Lyu, Tianle Liu, Zhang Zhang, Fuyuan Hu, Liang Wang

Keywords: Test-Time Discovery, Novel Category Discovery, Online Learning, Test-Time Training

TL;DR: We introduce Test-Time Discovery (TTD), a real-time evaluation protocol for novel class discovery under sequential test-time conditions.

Abstract: We introduce Test-Time Discovery (TTD), a real-time evaluation protocol for novel class discovery under sequential test-time conditions. Unlike post-hoc NCD evaluation, which assesses clustering only after the full test set is processed, TTD requires models to classify known categories and discover novel ones in real time as samples arrive. To address this setting, we propose a training-free Hash Memory (HM) method. HM encodes feature norm and direction into semantic-aware hash codes, enabling Locality-Sensitive Hashing (LSH) for efficient retrieval and consistent reuse of discovered classes. A global-to-local strategy combines prototypes for stable known-class predictions with memory-based reasoning for flexible novel discovery. A lightweight self-correction mechanism further improves reliability by removing mislabeled entries from early discoveries. Experiments on diverse benchmarks show that HM achieves more accurate and stable real-time discovery than NCD and TTT methods, while maintaining performance on known classes. Our code will be released.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 8315