THE PREIMAGE OF RECTIFIER NETWORK ACTIVITIES

Stefan Carlsson, Hossein Azizpour, Ali Razavian, Josephine Sullivan, Kevin Smith

Abstract:

We give a procedure for explicitely computing the complete preimage of activities of a layer in a rectifier network with fully connected layers, from knowledge of the weights in the network. The most general characterization of preimages is as piecewise linear manifolds in the input space with possibly multiple branches. This work therefore complements previous demonstrations of preimages obtained by heuristic optimization and regularization algorithms Mahendran & Vedaldi (2015; 2016) We are presently empirically evaluating the procedure and it’s ability to extract complete preimages as well as the general structure of preimage manifolds.

ICLR 2017 CONFRENCE TRACK SUBMISSION: https://openreview.net/forum?id=HJcLcw9xg&noteId=HJcLcw9xg

TL;DR: Proposes a procedure to construct the preimage of a ReLU network inference and discusses it's implications.

Conflicts: kth.se