Lifelong Perceptual Programming By Example

Alexander L. Gaunt, Marc Brockschmidt, Nate Kushman, Daniel Tarlow

TL;DR: Combination of differentiable interpreters and neural networks for lifelong learning of a model composed of neural and source code functions

Abstract: We introduce and develop solutions for the problem of Lifelong Perceptual Programming By Example (LPPBE). The problem is to induce a series of programs that require understanding perceptual data like images or text. LPPBE systems learn from weak supervision (input-output examples) and incrementally construct a shared library of components that grows and improves as more tasks are solved. Methodologically, we extend differentiable interpreters to operate on perceptual data and to share components across tasks. Empirically we show that this leads to a lifelong learning system that transfers knowledge to new tasks more effectively than baselines, and the performance on earlier tasks continues to improve even as the system learns on new, different tasks.

Keywords: Supervised Learning, Deep learning

Conflicts: @microsoft.com