Kernel Alignment using Manifold Approximation

Mohammad Tariqul Islam, Du Liu, Deblina Sarkar

Track: tiny / short paper (up to 5 pages)

Domain: machine learning

Abstract: Centered kernel alignment (CKA) is a popular metric for comparing representation, determining equivalence of networks, and conducting neuroscience research. However, CKA does not account for the underlying manifold and relies on many heuristics that make it behave differently at different scales of data. In this work, we propose Manifold-approximated Kernel Alignment (MKA) that incorporates manifold geometry into the alignment task. We derive a theoretical framework for MKA. We perform empirical evaluations on synthetic datasets and real-world examples to characterize and compare MKA to CKA. Our findings suggest that manifold-aware kernel alignment provides a more robust foundation for measuring representations, with potential applications in representation learning.

Submission Number: 52