Few-shot Learning with Big Prototypes

Anonymous

Abstract: Metric-based meta-learning is one of the de facto standards in few-shot learning. It composes of representation learning and metrics calculation designs. Previous works construct class representations in different ways, varying from mean output embedding to covariance and distributions. However, using embeddings in space lacks expressivity and cannot capture class information robustly, while statistical complex modeling poses difficulty to metric designs. We propose to use tensor fields (``areas'') to model classes from the geometrical perspective. Specifically, we present big prototypes, where class prototypes are represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere's center and the radius. Extending from points to areas, hyperspheres are much more expressive than embeddings. Moreover, it is more convenient to perform metric-based classification with big prototypes than statistical modeling, as we only need to calculate the distance from a data point to the surface of the hypersphere. Following this idea, we also develop two variants of big prototypes under other measurements. Extensive experiments on few-shot learning tasks across NLP and CV and comparison with 20+ competitive baselines demonstrate the effectiveness of big prototypes.

Paper Type: long