Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Deep Semiparametric Learning
Mika Sarkin Jain, Jack Lindsey
Feb 12, 2018 (modified: Feb 12, 2018)ICLR 2018 Workshop Submissionreaders: everyone
Abstract:We introduce a semiparametric approach to deep learning. Inspired by complementary learning systems theory in cognitive neuroscience, our approach combines elements of parametric and nonparametric learning by giving a neural network access to a differentiable nearest-neighbor algorithm. Analysis of our model suggests that it is able to leverage the respective advantages of nonparametric and parametric methods. Our model displays robustness to domain adaptation, rapid learning on limited training sets, and well-clustered embeddings while retaining the expressive power and generalization capabilities characteristic of parametric methods. We demonstrate that our model relies more heavily on nearest-neighbors information in early training but better approximates a purely parametric model as training progresses.
TL;DR:Combining parametric and nonparametric methods aids domain transfer and fast learning.
Keywords:deep learning, nonparametric, episodic learning, nearest neighbors, complementary learning systems
Enter your feedback below and we'll get back to you as soon as possible.