CoMNet: Where Biology Meets ConvNets
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: ConvNet, cortical modules, pyramidal neurons, long range connections
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: A new improved ConvNet template architecture
Abstract: Designing ConvNet and exploring its design space is a highly challenging research
area. In this paper, inspired by the structural organization of cortical modules in the
biological visual cortex, we present a pragmatically designed ConvNet architecture,
called CoMNet which is simplified yet powerful. The bio-inspired design of CoM-
Net offers efficiency in multiple dimensions such as network depth, parameters,
FLOPs, latency, branching, and memory budget at once while having a simple
design space, in contrast to the existing designs which are limited only to fewer
dimensions. We also develop a Multi-Dimensional Efficiency (MDE) evaluation
protocol to compare models across dimensions. Our comprehensive evaluations
show that in the MDE setting, CoMNet outperforms many representative ConvNet
designs such as ResNet, ResNeXt, RegNet, RepVGG, and ParNet (Figure 1).
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4351
Loading