MGIC: Multigrid-in-Channels Neural Network Architectures

Moshe Eliasof, Jonathan Ephrath, Lars Ruthotto, Eran Treister

Keywords: Multigrid methods, Efficient neural networks, PDEs and CNNs

Abstract:

Multigrid (MG) methods are effective at solving numerical PDEs in linear complexity. In this work we present a multigrid-in-channels (MGIC) approach that tackles the quadratic growth of the number of parameters with respect to the number of channels in standard convolutional neural networks (CNNs). Indeed, lightweight CNNs can achieve comparable accuracy to standard CNNs with fewer parameters; however, the number of weights still scales quadratically with the CNN's width. Our MGIC architectures replace each CNN block with an MGIC counterpart that utilizes a hierarchy of nested grouped convolutions of small group size to address this.
Hence, our proposed architectures scale linearly with respect to the network's width while retaining full coupling of the channels as in standard CNNs.
Our extensive experiments on image classification, segmentation, and point cloud classification show that applying this strategy to different architectures reduces the number of parameters while obtaining similar or better accuracy.

Publication Status: This work is unpublished.