Data for free: Fewer-shot algorithm learning with parametricity data augmentation

Owen Lewis, Katherine Hermann

Mar 24, 2019 ICLR 2019 Workshop LLD Blind Submission readers: everyone
  • Keywords: data augmentation, algorithm learning, RNN
  • TL;DR: Learned data augmentation instills algorithm-favoring inductive biases that let RNNs learn list-processing algorithms from fewer examples.
  • Abstract: We address the problem of teaching an RNN to approximate list-processing algorithms given a small number of input-output training examples. Our approach is to generalize the idea of parametricity from programming language theory to formulate a semantic property that distinguishes common algorithms from arbitrary non-algorithmic functions. This characterization leads naturally to a learned data augmentation scheme that encourages RNNs to learn algorithmic behavior and enables small-sample learning in a variety of list-processing tasks.
0 Replies