Generation of 3D Human Models and Animations Using Simple Sketches

Alican Akman, Yusuf Sahillioglu, T. Metin Sezgin

Abstract: Generating 3D models from 2D images or sketches is a widely studied important problem in computer graphics. We describe the first method to generate a 3D human model from a single sketched stick figure. In contrast to the existing human modeling techniques, our method requires neither a statistical body shape model nor a rigged 3D character model. We exploit Variational Autoencoders to develop a novel framework capable of transitioning from a simple 2D stick figure sketch, to a corresponding 3D human model. Our network learns the mapping between the input sketch and the output 3D model. Furthermore, our model learns the embedding space around these models. We demonstrate that our network can generate not only 3D models, but also 3D animations through interpolation and extrapolation in the learned embedding space. Extensive experiments show that our model learns to generate reasonable 3D models and animations.

TL;DR: In this paper, we propose a model that learns to generate reasonable 3D human models and animations using simple sketches. We exploit novel deep learning architecture which is called Variational Auto Encoders to implement our neural network..

Keywords: Sketch-based shape modeling, deep learning, 2D sketches, 3D shapes, static and dynamic 3D human models, computer graphics