Learning Human Vehicle Interactions using Simulated Data

Chuyu Liu, Tae Soo Kim

Abstract: When a deep neural network based action classifier can be trained in an end-to-end manner using adequately large training datasets, it has shown to establish impressive benchmarks even without much alterations to the model architecture. However, it is yet unclear how to utilize such heavily parameterized models when it is difficult to obtain sufficient labeled training examples. In this work, we show that we can simulate training videos to improve performance of deep neural network based action classification models. We show that pretraining the model first using a large number of simulated examples and then later finetuning using available real instances leads to much better results. We evaluate our approach using a subset of the DIVA dataset which is a challenging benchmark containing fine-grained human-vehicle interactions captured from surveillance cameras. We show that pretraining with simulated data improves performance of state-of-the-art action classifiers by a considerable margin.