Predicting episodic structure from overlapping input in binary networks with homeostasis
Keywords: Semantics, Episodes, Homeostasis, Regularization, Hopfield Network, Synaptic Plasticity, Predictive Learning
TL;DR: We present a protocol for generating semantically-charged overlapping input patterns, and a binary network that can extract these semantics into its recurrent weights, then using these for prediction of the structure of corrupted input.
Abstract: How neural networks process overlapping input patterns is a fundamental question in both neuroscience and artificial intelligence. Traditionally, overlaps in neural activity are viewed as interference, requiring separation for better performance. However, an alternative perspective suggests that these overlaps may encode meaningful semantic relationships between concepts. In this paper, we propose a framework where persistent overlap between episodic patterns represent semantic components across episodic experiences, and the statistics of these overlaps how each semantic concept relates to others.
To explore this idea, we introduce an Episode Generation Protocol (EGP) that defines a mapping between the semantic structure of episodes and input pattern generation. Paired with our EGP, we use Homeostatic Binary Networks (HBNs), a simplified yet biologically-inspired model incorporating key features such as adjustable inhibition, Hebbian learning, and homeostatic plasticity.
Our contributions are threefold: (1) We formalize a link between episodic semantics and neural patterns through our EGP. This EGP can be used for systematic study of semantic learning in artificial neural networks. (2) We introduce HBNs as an analytically tractable network that extracts semantic structure in its internal model (3) We show that HBNs align their performance with Maximum A Posteriori and Maximum Likelihood Estimation strategies depending on the homeostatic regime. Similarly, we provide an example of how our EGP can be used as an experimental protocol in neuroscience to make different models of learning compete.
Primary Area: applications to neuroscience & cognitive science
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13667
Loading