Reproducibility and Stability Analysis in Metric-Based Few-Shot LearningDownload PDF

Anonymous

06 Mar 2019 (modified: 03 May 2019)ICLR 2019 Workshop RML Blind SubmissionReaders: Everyone
  • Keywords: reproducibility, few-shot, machine learning, statistics
  • TL;DR: We propose a study of the stability of several few-shot learning algorithms subject to variations in the hyper-parameters and optimization schemes while controlling the random seed.
  • Abstract: We propose a study of the stability of several few-shot learning algorithms subject to variations in the hyper-parameters and optimization schemes while controlling the random seed. We propose a methodology for testing for statistical differences in model performances under several replications. To study this specific design, we attempt to reproduce results from three prominent papers: Matching Nets, Prototypical Networks, and TADAM. We analyze on the miniImagenet dataset on the standard classification task in the 5-ways, 5-shots learning setting at test time. We find that the selected implementations exhibit stability across random seed, and repeats.
2 Replies

Loading