DIVINE: Diverse-Inconspicuous Feature Learning to Mitigate Abridge Learning

Saheb Chhabra, Kartik Thakral, Surbhi Mittal, Mayank Vatsa, Richa Singh

Abstract: Deep learning algorithms aim to minimize overall error and exhibit impressive performance on test datasets across various domains. However, they often struggle with out-of-distribution (OOD) data samples. We posit that deep models primarily capture prominent features beneficial for the task while neglecting subtle yet discriminative features, a phenomenon we refer to as Abridge Learning. To address this issue and encourage more comprehensive feature utilization, we introduce DIVINE (DIVerse and INconspicuous FEature Learning), a novel approach that leverages iterative feature suppression guided by dominance maps to ensure that models engage with a diverse and complementary set of discriminative features. Through extensive experiments on multiple datasets, including MNIST, CIFAR-10, CIFAR-100, TinyImageNet, and their corrupted and perturbed variants (CIFAR-10-C/P, CIFAR-100-C/P, TinyImageNet-C/P), we demonstrate that DIVINE significantly improves model robustness and generalization. On perturbation benchmarks, DIVINE achieves mean Flip Rates (mFR) of 5.36%, 3.10%, and 21.85% on CIFAR-10-P, CIFAR-100-P, and TinyImageNet-P respectively, compared to 6.53%, 11.75%, and 31.90% for standard training methods exhibiting Abridge Learning. Moreover, DIVINE attains state-of-the-art results on CIFAR-100-P, demonstrating that addressing Abridge Learning leads to more robust models against real-world distribution variations.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=K7gICLoCEo&noteId=LuiAyKDNr6

Changes Since Last Submission:

We thank the reviewers and the area editor for their valuable feedback, which has helped us improve the quality of our manuscript. Below is a summary of the revisions made in response to their comments:

- Added Grad-CAM visualizations of CIFAR-10 samples as Figure 8 in the main paper, along with corresponding analysis to highlight differences in feature attribution between the baseline and DIVINE models.
- Prepared a deanonymized camera-ready version of the paper and included an acknowledgment section.
- Added a link to the code in the paper for the proposed DIVINE algorithm to support reproducibility.
- Corrected minor grammatical errors to enhance the readability and clarity of the manuscript.

Assigned Action Editor: ~Charles_Xu1

Submission Number: 4642