Deep Kernel Machines via the Kernel Reparametrization Trick

Jovana Mitrovic, Dino Sejdinovic, Yee Whye Teh

Abstract: While deep neural networks have achieved state-of-the-art performance on many tasks across varied domains, they still remain black boxes whose inner workings are hard to interpret and understand. In this paper, we develop a novel method for efficiently capturing the behaviour of deep neural networks using kernels. In particular, we construct a hierarchy of increasingly complex kernels that encode individual hidden layers of the network. Furthermore, we discuss how our framework motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.

TL;DR: Encoding deep neural networks using a hierarchy of increasingly complex kernels. This motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.

Conflicts: ox.ac.uk

Keywords: Theory, Deep learning