Challenges in Explaining Representational Similarity through Identifiability

Beatrix Miranda Ginn Nielsen, Luigi Gresele, Andrea Dittadi

Track: Extended Abstract Track

Keywords: representational similarity, identifiability

TL;DR: For a certain model class, we show the relationship between representations of different models and how even close to optimal loss is not sufficient for representations to become close to equivalent.

Abstract: The phenomenon of different deep learning models producing similar data representations has garnered significant attention, raising the question of why such _representational similarity_ occurs. Identifiability theory offers a partial explanation: for a broad class of discriminative models, including many popular in representation learning, those assigning equal likelihood to the observations yield representations that are linear transformations of each other, if a suitable diversity condition holds. In this work, we identify two key challenges in applying identifiability theory to explain representational similarity. First, the assumption of exact likelihood equality is rarely satisfied by practical models trained with different initializations. To address this, we describe how the representations of two models deviate from being linear transformations of each other, based on their difference in log-likelihoods. Second, we demonstrate that even models with similar and near-optimal loss values can produce highly dissimilar representations due to an underappreciated difference between loss and likelihood. Our findings highlight key open questions and point to future research directions for advancing the theoretical understanding of representational similarity.

Submission Number: 15