Visualizing the Diversity of Representations Learned by Bayesian Neural Networks
Abstract: Explainable Artificial Intelligence (XAI) aims to make learning machines less opaque, and offers researchers and practitioners various tools to reveal the decision-making strategies of neural networks. In this work, we investigate how XAI methods can be used for exploring and visualizing the diversity of feature representations learned by Bayesian Neural Networks (BNNs). Our goal is to provide a global understanding of BNNs by making their decision-making strategies a) visible and tangible through feature visualizations and b) quantitatively measurable with a distance measure learned by contrastive learning. Our work provides new insights into the posterior distribution in terms of human-understandable feature information with regard to the underlying decision-making strategies. The main findings of our work are the following: 1) global XAI methods can be applied to explain the diversity of decision-making strategies of BNN instances, 2) Monte Carlo dropout with commonly used Dropout rates exhibit increased diversity in feature representations compared to the multimodal posterior approximation of MultiSWAG, 3) the diversity of learned feature representations highly correlates with the uncertainty estimate for the output and 4) the inter-mode diversity of the multimodal posterior decreases as the network width increases, while the intra-mode diversity increases. These findings are consistent with the recent Deep Neural Networks theory, providing additional intuitions about what the theory implies in terms of humanly understandable concepts.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: As requested by the reviewers "WYUH", "gAeF", and the action editors we have included the experimental results for the impact of pre-training in Appendix F of our camera-ready version. We refer to these results in the second paragraph of Section 4.5 in the manuscript.
Assigned Action Editor: ~Cedric_Archambeau1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 868
Loading