Interactive learning of human activities using active video composition

Michael S. Ryoo

Abstract: In order to recognize human activities, the system is required to learn properties (i.e. structures) of the activities from training samples. However, existing supervised learning approaches lack interactiveness, solely relying on biased training examples provided by the user. Such approaches are limited especially for real-world scenarios where the amount of training videos is not sufficient. In this paper, we present the novel concept of active video composition learning, a new interactive activity learning paradigm designed to overcome the limitations of the previous paradigm. In contrast to previous passive learning systems relying on user-provided training videos, our approach learns human activities by actively interacting with the human teacher. The idea is to make the system generate composed videos with the activity structure it is most uncertain of and ask the human to provide activity labels for such video queries. In addition, a methodology to semantically represent the system's activity learning status is designed, making the system to interactively inform the human teacher its status and benefit from such interaction. The experimental results confirm that our approach interactively estimates the decision boundaries in the activity structure space, and does it more reliably even when very few real training videos are provided.