Representation Change in Model-Agnostic Meta-Learning

Anonymous

Keywords: meta-learning, maml, representation change, representation reuse

Abstract:

Last year, an exciting adaptation of one of the most popular optimization-based meta-learning approaches, model-agnostic meta-learning (MAML) [Finn et al., 2017], was proposed in

- Jaehoon Oh, Hyungjun Yoo, ChangHwan Kim, Se-Young Yun (ICLR, 2021) BOIL: Towards Representation Change for Few-shot Learning

The authors adapt MAML by freezing the last layer to force body only inner learning (BOIL). Interestingly, this is complementary to ANIL (almost no inner loop) proposed in

- Aniruddh Raghu, Maithra Raghu, Samy Bengio, Oriol Vinyals (ICLR, 2020) Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML

Both papers attempt to understand the success of MAML and improve it.
Oh et al. [2021] compare BOIL, ANIL, and MAML and show that both improve the performance of MAML, but BOIL outperforms ANIL, especially when the task distribution varies between training and testing.

ICLR Paper: https://openreview.net/forum?id=umIdUL8rMH, https://openreview.net/forum?id=9SF-oupe14b