BioNAS: Incorporating Bio-inspired Learning Rules to Neural Architecture Search

Imane Hamzaoui, Riyadh Baghdadi

Keywords: Feedback Alignment, Neural Architecture Search, Learning Rules, Biologically Plausible

TL;DR: neural architecture search with different feedback alignment techniques improves accuracy and robustness

Abstract:

Bio-inspired neural networks have gained traction due to their adversarial robustness, energy efficiency, and for being biologically plausible. While these bio-inspired networks have shown significant progress, they still fall short in terms of accuracy and are hard to scale to complex tasks. In this paper, we propose to use neural architecture search to further improve state-of-the-art bio-inspired neural networks. We
achieve this thanks to BioNAS, a framework for neural architecture search that explores different bio-inspired neural network architectures and learning rules. The novelty of BioNAS lies in exploring the use of different bio-inspired learning rules for the different layers of the model. The motivation for this choice comes from recent work in the field suggesting that different learning mechanisms might be used in different
regions of the human brain. Using BioNAS, we get state-of-the-art bio-inspired neural network performance achieving an accuracy of 94.86 on CIFAR10, 76.48 on CIFAR-100 and 43.42 on ImageNet16-120, surpassing state-of-the-art bio-inspired neural networks. We show that a part of this improvement comes from the use of different learning rules instead of using a single algorithm for all the layers. We release BioNAS to the community and make the code available via this link (https://anonymous.4open.science/r/LR-NAS-DFE1)

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3917