On Episodes, Prototypical Networks, and Few-Shot Learning

Steinar Laenen, Luca Bertinetto

Keywords: few-shot learning, meta-learning, metric learning, deep learning

Abstract:

Episodic learning is a popular practice among researchers and practitioners interested in few-shot learning. It consists of organising training in a series of learning problems, each relying on small “support” and “query” sets to mimic the few-shot circumstances encountered during evaluation.
In this paper, we investigate the usefulness of episodic learning in Prototypical Networks, one of the most popular algorithms making use of this practice.
Surprisingly, in our experiments we found that, for Prototypical Networks, it is detrimental to use the episodic learning strategy of separating training samples between support and query set, as it is
a data-inefficient way to exploit training batches. This “non-episodic” version of Prototypical Networks, which corresponds to the classic Neighbourhood Component Analysis, reliably improves over its episodic counterpart in multiple datasets, achieving an accuracy that is competitive with the state-of-the-art, despite being extremely simple.

One-sentence Summary: We analysed the effectiveness of episodic learning in Prototypical Networks and found out that, despite adding complexity and hyper-parameters, it severely affects its performance.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2012.09831/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=IacZhp4vpb