Informed Mixing -- Improving Open Set Recognition with Deep Dynamic Data Augmentation

Jiawen Xu, Margret Keuper

Keywords: open set recognition, model generalization, data augmentation

Abstract:

Conventionally trained image classifiers recently excel in accuracy across diverse tasks. One practical limitation is however that they assume all potential classes to be seen during training, i.e., they cannot tell "I don't know" when encountering an unknown class.
Open set recognition (OSR), which solves this problem of detecting novel classes during inference, therefore remains an open problem and is receiving increasing attention. Thereby, a crucial challenge is to learn features that are relevant for unseen categories from given data, for which these features might not be discriminative. Previous work has shown that the introduction of self-supervised contrastive learning to supervised paradigms can support diverse feature learning and thereby benefit OSR. However, the diversity in contrastive learning is commonly introduced through crafted augmentation schemes. To improve upon this aspect and "optimize to learn" more diverse features, we propose GradMix, a data augmentation method that dynamically leverages gradient-based attribution maps of the model during training. The idea is to mask out the activated areas in previous epochs so that the models can pay attention to broader areas and learn to extract features beyond of what is most discriminative for every class. The resulting models are expected to learn more diverse features from the same data source and thus to improve in OSR and model generalization. Extensive experiments on open set recognition, close set classification, and out-of-distribution detection reveal that our method performs well on these tasks that can often outperform the state-of-the-art. GradMix is also beneficial for increasing robustness to common corruptions.
In self-supervised learning, GradMix can increase the accuracy of downstream linear classifiers compared with baselines, indicating its benefit for model generalization.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1091