Attentional meta-learners for few-shot polythetic classification

Ben Day, Ramon Viñas Torné, Nikola Simidjievski, Pietro Lio

Keywords: Meta-learning, self-attention, feature-selection

Abstract: Polythetic classifications, based on shared patterns of features that need neither be universal nor constant among members of a class, are common in the natural world and greatly outnumber monothetic classifications over a set of features. We show that threshold meta-learners, such as Prototypical Networks, require an embedding dimension that is exponential in the number of features to emulate these functions. In contrast, attentional classifiers, such as Matching Networks, are polythetic by default and able to solve these problems with a linear embedding dimension. However, we find that in the presence of task-irrelevant features, inherent to meta-learning problems, attentional models are susceptible to misclassification. To address this challenge, we propose a self-attention feature-selection mechanism that adaptively dilutes non-discriminative features. We demonstrate the effectiveness of our approach in meta-learning Boolean functions, and synthetic and real-world few-shot learning tasks.

One-sentence Summary: Attentional meta-learners are well-suited to recognising classes based on patterns of traits and benefit from feature-selection.