Noise-Augmented Deep Neural Networks for Image Classification: Insights from Information Theory
Keywords: Image Classification, ViT, Noise
TL;DR: We explore the influence of proactively adding noise to embeddings in classification tasks within the field of computer vision.
Abstract: In this study, we explore the impact of proactively injecting noise into deep learning models, focusing particularly on classification problems, such as image classification and domain adaptation. While noise is typically seen as harmful, our findings reveal that, under certain conditions, noise can beneficially influence the entropy of the system, enhancing the learning outcomes. We employ information entropy to characterize the complexity of the learning tasks and categorize noise into two types, positive noise (PN) and harmful noise (HN), based on whether it helps reduce task complexity. We theoretically prove that positive noise reduces task complexity and demonstrate the presence of positive noise through extensive experiments on Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). We further propose NoisyNN, an innovative approach to leverage positive noise. NoisyNN achieves state-of-the-art performance on various image classification and domain adaptation tasks. Extensive experiments conducted on {15 datasets}, including popular image datasets and out-of-distribution datasets, demonstrate the efficacy of our method. Our study provides the community with a new paradigm for improving model performance.
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4973
Loading