SketchEmbedNet: Learning Novel Concepts by Imitating Drawings

Alexander Wang, Mengye Ren, Richard Zemel

Keywords: generative, probabilistic, sketch, drawing, few-shot learning, classification, embedding learning

Abstract: Sketch drawings are an intuitive visual domain that appeals to human instinct. Previous work has shown that recurrent neural networks are capable of producing sketch drawings of a single or few classes at a time. In this work we investigate representations developed by training a generative model to produce sketches from pixel images across many classes in a sketch domain. We find that the embeddings learned by this sketching model are extremely informative for visual tasks and infer a unique visual understanding. We then use them to exceed state-of-the-art performance in unsupervised few-shot classification on the Omniglot and mini-ImageNet benchmarks. We also leverage the generative capacity of our model to produce high quality sketches of novel classes based on just a single example.

One-sentence Summary: Learning a generative image to sketch model for few-shot classification and novel visual understanding in both embedding and image space.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2009.04806/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=Tlmm-_DGd3