- Keywords: spatiotemporal data, deep learning, generative models, multiresolution learning, behavioral cloning, imitation learning
- TL;DR: We show how to effectively learn spatiotemporal behavioral policies over long time horizons using multi resolution generative models with adversarial training.
- Abstract: We present a multi-resolution approach for generative modeling of spatiotemporal sequences. Our approach focuses on a key challenge of modeling long-term temporal dependencies that is common in complex spatiotemporal phenomena. For instance, realistic modeling of basketball players behavior requires capturing long-term goals and how they influence short-term decisions. Our multi-resolution approach has several attractive properties. First, it is completely unsupervised, and requires no additional labeling of high-level semantics such as long-term goals. Second, the multi-resolution aspect allows us to model conditional distributions beyond forward sampling, such as conditioning on future behavior. Finally, our approach integrates generative adversarial training, which enables us to learn generative models that significantly outperform conventional generative sequence modeling. We validate the effectiveness of our model on synthetic sequences and spatiotemporal basketball player trajectory generation.