Dendritic Localized Learning: Toward Biologically Plausible Algorithm

Changze Lv, Jingwen Xu, Yiyang Lu, Xiaohua Wang, Zhenghua Wang, Zhibo Xu, Di Yu, Xin Du, Xiaoqing Zheng, Xuanjing Huang

TL;DR: We propose Dendritic Localized Learning (DLL), a biologically plausible learning algorithm.

Abstract:

Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success.
However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, as highlighted by the existing literature.
Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultaneously or yield suboptimal results.
Inspired by the dynamics and plasticity of pyramidal neurons, we propose Dendritic Localized Learning (DLL), a novel learning algorithm designed to overcome these challenges.
Extensive empirical experiments demonstrate that DLL satisfies all three criteria of biological plausibility while achieving state-of-the-art performance among algorithms that meet these requirements.
Furthermore, DLL exhibits strong generalization across a range of architectures, including MLPs, CNNs, and RNNs.
These results, benchmarked against existing biologically plausible learning algorithms, offer valuable empirical insights for future research.
We hope this study can inspire the development of new biologically plausible algorithms for training multilayer networks and advancing progress in both neuroscience and machine learning.
Our code is available at https://github.com/Lvchangze/Dendritic-Localized-Learning.

Lay Summary:

Training AI often uses backpropagation, which works well but doesn't mimic how brains learn. Scientists note three key flaws: it requires unnatural symmetry in connections, relies on global error signals, and separates learning into distinct phases, unlike the brain's integrated process. While other methods tried to fix these, most fall short by either ignoring some flaws or performing poorly.

This study introduces Dendritic Localized Learning (DLL), a brain-inspired approach modeled after how neurons’ branching structures (dendrites) adapt. Tests show DLL overcomes all three flaws while matching top AI performance. It works across diverse models, including those for images or language, and outperforms other biologically inspired methods.

By bridging neuroscience and AI, DLL offers clues to how brains learn efficiently, potentially improving both AI systems and our understanding of biology. The researchers hope this sparks innovation in brain science and AI development. The code is freely available for others to use and build on.

Link To Code: https://github.com/Lvchangze/Dendritic-Localized-Learning

Primary Area: Applications->Neuroscience, Cognitive Science

Keywords: Biological Plausibility, Learning Algorithm

Submission Number: 209