Riemannian Residual Networks

Anonymous

Keywords: neural network, riemannian, manifold, resnet

TL;DR: We construct generalized residual neural networks that can learn over Riemannian data.

Abstract: Recent methods in geometric deep learning have introduced various neural networks to operate on Riemannian manifolds. These methods are often inspired by and directly generalize standard Euclidean neural networks. In practice, extending these is difficult and has only been done for a select few manifolds. In this work, we examine the residual neural network (ResNet) and show how to extend this construction to general Riemannian manifolds. Originally introduced to help solve the vanishing gradient problem, ResNets have become ubiquitous in machine learning due to their beneficial learning properties, excellent empirical results, and easy-to-incorporate nature when building varied neural networks. We find that our Riemannian ResNets mirror these desirable properties and generalize well to non-Euclidean manifolds (regardless of topology).