Dendritic Integration Inspired Artificial Neural Networks Capture Data Correlation

Chongming Liu, Jingyang Ma, Songting Li, Douglas Zhou

Keywords: brain-inspired models, dendritic bilinear integration rule, quadratic neuron, deep convolutional neural networks

Abstract: Incorporating biological neuronal properties into Artificial Neural Networks (ANNs) to enhance computational capabilities is under active investigation in the field of deep learning. Inspired by recent findings indicating that dendrites adhere to quadratic integration rule for synaptic inputs, this study explores the computational benefits of quadratic neurons. We theoretically demonstrate that quadratic neurons inherently capture correlation within structured data, a feature that grants them superior generalization abilities over traditional neurons. This is substantiated by few-shot learning experiments. Furthermore, we integrate the quadratic rule into Convolutional Neural Networks (CNNs) using a biologically plausible approach, resulting in innovative architectures—Dendritic integration inspired CNNs (Dit-CNNs). Our Dit-CNNs compete favorably with state-of-the-art models across multiple classification benchmarks, e.g., ImageNet-1K, while retaining the simplicity and efficiency of traditional CNNs. All source code are available at https://github.com/liuchongming1999/Dendritic-integration-inspired-CNN-NeurIPS-2024.

Primary Area: Deep learning architectures

Submission Number: 15423