Geometry of Deep Convolutional Networks

Stefan Carlsson

Abstract:

We give a formal procedure for computing preimages of convolutional
network outputs using the dual basis defined from the set of
hyperplanes associated with the layers of the network. We point out
the special symmetry associated with arrangements of hyperplanes of
convolutional networks that take the form of regular
multidimensional polyhedral cones. We discuss the efficiency of of
large number of layers of nested cones that result from incremental
small size convolutions in order to give a good compromise between
efficient contraction of data to low dimensions and shaping of
preimage manifolds. We demonstrate how a specific network flattens a
non linear input manifold to an affine output manifold and discuss
it's relevance to understanding classification properties of deep
networks.

Keywords: convolutional networks, geometry

TL;DR: Analysis of deep convolutional networks in terms of associated arrangement of hyperplanes