Getting aligned on representational alignment

Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Christopher J Cueva, Erin Grant, Iris Groen, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine Hermann, Kerem Oktar, Klaus Greff, Martin N Hebart, Nathan Cloos, Nikolaus Kriegeskorte, Nori Jacoby et al. (13 additional authors not shown)

Authors that are also TMLR Expert Reviewers: ~Simon_Kornblith1

Abstract: Biological and artificial information processing systems form representations of the world that they can use to categorize, reason, plan, navigate, and make decisions. How can we measure the similarity between the representations formed by these diverse systems? Do similarities in representations then translate into similar behavior? If so, then how can a system's representations be modified to better match those of another system? These questions pertaining to the study of \emph{representational alignment} are at the heart of some of the most promising research areas in contemporary cognitive science, neuroscience, and machine learning. In this Perspective, we survey the exciting recent developments in representational alignment research in the fields of cognitive science, neuroscience, and machine learning. Despite their overlapping interests, there is limited knowledge transfer between these fields, so work in one field ends up duplicated in another, and useful innovations are not shared effectively. To improve communication, we propose a unifying framework that can serve as a common language for research on representational alignment, and map several streams of existing work across fields within our framework. We also lay out open problems in representational alignment where progress can benefit all three of these fields. We hope that this paper will catalyze cross-disciplinary collaboration and accelerate progress for all communities studying and developing information processing systems.

Certifications: Survey Certification, Expert Certification

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

We have implemented the suggestions from Reviewer d8Kx including a) adding in the suggested additional citations, literature, discussion, and explanations (relating to the points from the Reviewer's weaknesses section, and points 2, 4, 6, 7, 8, 11, 12 from the Reviewer's other issues list), b) fixing figure 1 and updating its caption to be more informative (relating to points 3 and 5 from other issues), c) using the Semantic Scholar API to pull the Semantic Scholar TLDRs (one-sentence summaries) of the majority of the papers we cite (those that could be found programmatically by searching the API) and adding these to the references so that readers have some more information about what each paper actually covers/proposes without having to look it up (relating to points 1, 2, 10, and 13 from other issues). We think the paper is in much stronger shape after this revision and hope that it will indeed be an almost “one-stop shop for learning about representational alignment” as the reviewer so kindly suggested!

Based on suggestions from Reviewer L8p3, we have added a sentence about the two suggested references to Section 2.3.1., incorporated some actionable guidelines (which can primarily be found in Sections 3 and 5), and better emphasized how researchers should interpret and use our framework as summarized by the schematic in Fig 2.

Finally, we resolved the minor stylistic issues brought up by Reviewer CMuC.

Assigned Action Editor: ~Sergey_Plis1

Submission Number: 4844