Monitoring Shortcut Learning using Mutual Information

Mohammed Adnan, Yani Ioannou, Kenyon Tsai, Angus Galloway, Hamid Tizhoosh, Graham W. Taylor

Keywords: shortcuts, shortcut learning, spurious correlation, mutual information, information theory

TL;DR: We propose that the mutual information between input and learned representation can be used a metric to measure shortcut learning.

Abstract: The failure of deep neural networks to generalize to out-of-distribution data is a well-known problem and raises concerns about the deployment of trained networks in safety-critical domains such as healthcare, finance, and autonomous vehicles. We study a particular kind of distribution shift — shortcuts or spurious correlations in the training data. Shortcut learning is often only exposed when models are evaluated on real-world data that does not contain the same spurious correlations, posing a serious dilemma for AI practitioners to properly assess the effectiveness of a trained model for real-world applications. In this work, we propose to use the mutual information (MI) between the learned representation and the input as a metric to find where in training the network latches onto shortcuts. Experiments demonstrate that MI can be used as a domain-agnostic metric for detecting shortcut learning.

Confirmation: Yes