SSPictR: Spatial Semantic Pointer Picture Representation

Anna Penzkofer, Karim Habashy, Abhishek Tyagi, Yuhao Zhu, Chris Eliasmith, Andreas Bulling

Keywords: scene representation, scene recognition, cognitive maps, spatial semantic pointers, vector symbolic algebras

TL;DR: We propose SSPictR, a cognitively-inspired scene representation that is inherently efficient and interpretable and generalises across different tasks

Abstract: The development of image representations that capture semantic and spatial information efficiently, which are also interpretable and generalisable, remains unsolved. Drawing from a cognitive modelling framework, we propose SSPictR – a biologically plausible image representation based on spatial semantic pointers (SSPs). SSPictR encodes semantic labels and their spatial locations extracted from segmentation maps and only requires a single vector to capture a fully decodable neuro-symbolic representation of a natural scene. It is inherently interpretable, offers a high compression factor and significantly faster inference speed on downstream tasks, such as scene recognition. We evaluate the efficiency and generalisability of SSPictR on the popular Places365, and ADE20K datasets for scene recognition, on COCOStuff for segmentation reconstruction, and on VISC and Savoias for prediction of visual complexity. We show that the scene representations provided by SSPictR are more generalisable within and across these tasks while only requiring a fraction of model parameters and, therefore, offer 25 times higher inference speed, with comparable accuracy. As such, SSPictR opens up a new direction for future research on cognitively-inspired image representations that are not only significantly smaller but also more interpretable and generalisable.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11892