FiT: Parameter Efficient Few-shot Transfer LearningDownload PDF

Published: 21 Oct 2022, Last Modified: 05 May 2023NeurIPS 2022 Workshop MetaLearn PosterReaders: Everyone
Keywords: few-shot learning, meta-learning, transfer learning
TL;DR: We propose FiT, a parameter efficient few-shot image classification system that uses a Naive Bayes head, FiLM layers that modulate a pretrained backbone, and an episodic fine-tuning protocol that achieves SOTA on the VTAB-1k benchmark.
Abstract: Model parameter efficiency is key for enabling few-shot learning, inexpensive model updates for personalization, and communication efficient federated learning. In this work, we develop FiLM Transfer (FiT) which combines ideas from transfer learning (fixed pretrained backbones and fine-tuned FiLM adapter layers) and meta-learning (automatically configured Naive Bayes classifiers and episodic training) to yield parameter efficient models with superior classification accuracy at low-shot. We experiment with FiT on a range of downstream datasets and show that it achieves better classification accuracy than the leading Big Transfer (BiT) algorithm at low-shot and achieves state-of-the art accuracy on the challenging VTAB-1k benchmark, with fewer than 1% of the updateable parameters.
0 Replies