Enhancing deep neural networks via multiple kernel learning

Ivano Lauriola, Claudio Gallicchio, Fabio Aiolli

Abstract: Highlights • We introduce KerNET, which combines Deep Neural Networks and Multiple Kernel learning; • The framework optimally aggregates the hidden representations of a Deep Neural Network. • KerNET enables to boost the predictive accuracy of individual Deep Neural Networks. • KerNET allows to relieve the overfitting problem in deeper architectures. Abstract Deep neural networks and Multiple Kernel Learning are representation learning methodologies of widespread use and increasing success. While the former aims at learning representations through a hierarchy of features of increasing complexity, the latter provides a principled approach for the combination of base representations. In this paper, we introduce a general framework in which the internal representations computed by a deep neural network are optimally combined by means of Multiple Kernel Learning. The resulting ensemble methodology is instantiated for Multi-layer Perceptrons architectures (both fully trained and with random-weights), and for Convolutional Neural Networks. Experimental results on several benchmark datasets concretely show the advantages and potentialities of the proposed approach.