Deep Kernel Machines via the Kernel Reparametrization Trick

Jovana Mitrovic, Dino Sejdinovic, Yee Whye Teh

Feb 17, 2017 (modified: Feb 17, 2017) ICLR 2017 workshop submission readers: everyone
  • Abstract: While deep neural networks have achieved state-of-the-art performance on many tasks across varied domains, they still remain black boxes whose inner workings are hard to interpret and understand. In this paper, we develop a novel method for efficiently capturing the behaviour of deep neural networks using kernels. In particular, we construct a hierarchy of increasingly complex kernels that encode individual hidden layers of the network. Furthermore, we discuss how our framework motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.
  • TL;DR: Encoding deep neural networks using a hierarchy of increasingly complex kernels. This motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.
  • Conflicts: ox.ac.uk
  • Keywords: Theory, Deep learning

Loading