Keywords: few-shot learning, gaussian processes, bayesian deep learning, uncertainty estimation
Abstract: Few-shot classification (FSC), the task of adapting a classifier to unseen classes given a small labeled dataset, is an important step on the path toward human-like machine learning. Bayesian methods are well-suited to tackling the fundamental issue of overfitting in the few-shot scenario because they allow practitioners to specify prior beliefs and update those beliefs in light of observed data. Contemporary approaches to Bayesian few-shot classification maintain a posterior distribution over model parameters, which is slow and requires storage that scales with model size. Instead, we propose a Gaussian process classifier based on a novel combination of Pólya-Gamma augmentation and the one-vs-each softmax approximation that allows us to efficiently marginalize over functions rather than model parameters. We demonstrate improved accuracy and uncertainty quantification on both standard few-shot classification benchmarks and few-shot domain transfer tasks.
One-sentence Summary: We propose a Gaussian process approach to few-shot classification based on the one-vs-each softmax approximation and Pólya-Gamma augmentation, and demonstrate competitive few-shot accuracy and strong uncertainty quantification.
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
Code: [![github](/images/github_icon.svg) jakesnell/ove-polya-gamma-gp](https://github.com/jakesnell/ove-polya-gamma-gp) + [![Papers with Code](/images/pwc_icon.svg) 1 community implementation](https://paperswithcode.com/paper/?openreview=lgNx56yZh8a)
Data: [CUB-200-2011](https://paperswithcode.com/dataset/cub-200-2011), [mini-Imagenet](https://paperswithcode.com/dataset/mini-imagenet)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](https://www.catalyzex.com/paper/arxiv:2007.10417/code)