LLM-assisted Semantic Reasoning for Open-Set Active Learning

ICLR 2026 Conference Submission14245 Authors

Keywords: Active Learning, Open-Set Active Learning, LLM-based Semantic Reasoning, VLM-Based Semantic Reasoning

TL;DR: We integrate LLM-based semantic reasoning with traditional deep networks to address open-set active learning, achieving SOTA results on benchmark datasets.

Abstract: Active learning (AL) methods primarily concentrate on closed-set annotations where irrelevant data is absent. However, real-world applications inevitably contain various forms of irrelevant data. This open-set annotation challenge has been explored in some studies, yet two key issues remain. The first is balancing between selecting maximally relevant data and querying uncertain samples, which often increases the proportion of irrelevant data. The second is the inability to distinguish between relevant and irrelevant samples before any labeling, commonly referred to as the cold-start problem. We tackle these challenges with our method named LaSeR (LLM-assisted Semantic Reasoning), which leverages LLM-generated image descriptions and VLM-based similarity scores to, introduce a metric capable of separating relevant from irrelevant data before labeling, and incorporates diversity in the selected samples to enhance model performance. Subsequently in later AL rounds, as more labeled data becomes available, we transfer this knowledge into a detector model to further improve the efficiency of our selection process. Extensive experimental results demonstrate that our method outperforms state-of-the-art AL approaches, as well as recent methods specifically designed for open-set active annotation on standard benchmark datasets.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 14245