Momentum Capsule Networks
Abstract: Capsule networks are a class of neural networks that aim at solving some limiting factors of Convolutional Neural Networks. However, baseline capsule networks have failed to reach state-of-the-art results on more complex datasets due to the high computation and memory requirements. We tackle this problem by proposing a new network architecture, called Momentum Capsule Network (MoCapsNet). MoCapsNets are inspired by Momentum ResNets, a type of network that applies reversible residual building blocks. Reversible networks allow for recalculating activations of the forward pass in the backpropagation algorithm, so those memory requirements can be drastically reduced. In this paper, we provide a framework on how invertible residual building blocks can be applied to capsule networks. We will show that MoCapsNet beats the accuracy of baseline capsule networks on MNIST, SVHN, CIFAR-10 and CIFAR-100 while using considerably less memory. The source code is available on https://github.com/moejoe95/MoCapsNet.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: New version of the paper, including experiments on the more difficult dataset CIFAR-100.
Assigned Action Editor: ~Joan_Bruna1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 12
Loading