Connecting Sphere Manifolds Hierarchically for Regularization

Damien Scieur, Youngsung Kim

Keywords: Hierarchy, Manifold, Classification

Abstract: This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: Assuming we know a hierarchical structure over classes, we regularize the neural network by forcing the classifier of each class to belong to a sphere manifold, whose center is the classifier of its super-class.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=_BeQkMO-ou