A Generalized Convolutional Neural Network for Small Dataset Classification

Zhenhua Chen, David J. Crandall

Keywords: Generalized, ConvNets, Classification

TL;DR: A generalized version of ConvNets.

Abstract: We propose a novel variant of neural networks, Generalized Convolutional Neural Networks, GConvNets, characterized by structured neurons. In contrast to conventional neural networks such as ConvNets, which predominantly employ 'scalar' neurons, GConvNets utilize structured 'tensor' neurons. In other words, we generalize ConvNets by substituting each scalar neuron in ConvNets with a tensor neuron in GConvNets, while preserving the weight-sharing mechanism. These structured neurons manifest as tensors with adaptable shapes and dimensions across different layers. To ensure their practical applicability, we have developed a mechanism that enables seamless handling of hybrid structured tensor neurons as they transition from one layer to the next. We conducted a comparative analysis between GConvNets and the currently popular ConvNets, which include ResNets, MobileNets, EfficientNets, RegNets, among others, using datasets such as CIFAR10, CIFAR100, and Tiny ImageNet. The experimental results demonstrate that GConvNets exhibit superior efficiency in terms of parameter usage.

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 695