Deep Kernel Machines via the Kernel Reparametrization TrickDownload PDF

08 Feb 2023, 12:56 (modified: 17 Feb 2017, 17:57)ICLR 2017 workshop submissionReaders: Everyone
TL;DR: Encoding deep neural networks using a hierarchy of increasingly complex kernels. This motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.
Abstract: While deep neural networks have achieved state-of-the-art performance on many tasks across varied domains, they still remain black boxes whose inner workings are hard to interpret and understand. In this paper, we develop a novel method for efficiently capturing the behaviour of deep neural networks using kernels. In particular, we construct a hierarchy of increasingly complex kernels that encode individual hidden layers of the network. Furthermore, we discuss how our framework motivates a novel supervised weight initialization method that discovers highly discriminative features already at initialization.
Keywords: Theory, Deep learning
Conflicts: ox.ac.uk
6 Replies

Loading