Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks
Track: long paper (up to 9 pages)
Keywords: CKA, Centered Kernel Alignment, fMRI, MEG, Neural, Alignment, debiased, CNNs, ANNs
TL;DR: Using Centered Kernel Alignment as a similarity quantification metric between brains and ANNs has various problems that we highlight, which we show are fixed with the application of a debiasing step.
Abstract: Centred Kernel Alignment (CKA) has recently emerged as a popular metric to compare activations from biological and artificial neural networks (ANNs) in order to quantify the alignment between internal representations derived from stimuli sets (e.g. images, text, video) that are presented to both systems (Sucholutsky et al., 2023; Han et al., 2023). In this paper we highlight issues that the community
should take into account if using CKA as an alignment metric with neural data. Neural data are in the low-data high-dimensionality domain, which is one of the cases where (biased) CKA results in high similarity scores even for pairs of random matrices. Using fMRI and MEG data from the THINGS project (Hebart et al., 2023), we show that if biased CKA is applied to representations of different sizes in the low-data high-dimensionality domain, they are not directly comparable due to biased CKA’s sensitivity to differing feature-sample ratios and not stimuli-driven responses. This situation can arise both when comparing a preselected area of interest (e.g. ROI) to multiple ANN layers, as well as when determining to which ANN layer multiple regions of interest (ROIs) / sensor groups of different dimensionality are most similar. We show that biased CKA can be artificially driven to its maximum value when using independent random data of different sample-feature ratios. We further show that shuffling sample-feature pairs of real neural data does not drastically alter biased CKA similarity in comparison to unshuffled data, indicating an undesirable lack of sensitivity to stimuli-driven neural responses. Positive alignment of true stimuli-driven responses is only achieved by using debiased CKA. Lastly, we report findings that suggest biased CKA is sensitive to the inherent structure of neural data, only differing from shuffled data when debiased CKA detects stimuli-driven alignment.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 71
Loading