Mining Video Data: Learning about Activities

Anthony G. Cohn

Abstract: In this talk I will present ongoing work at Leeds on building models of video activity. I will present techniques, both supervised and unsupervised, for learning the spatio-temporal structure of tasks and events from video or other sensor data. In both cases, the representation will exploit qualititive spatio-temporal relations. A novel method for robustly transforming video data to qualitative relations will be presented. For supervised learning I will show how the supervisory burden can be reduced using what we term “deictic supervision”, whilst in the unsupervised case I will present a method for learning the most likely interpretation of the training data. I will also show how objects can be “functionally categorised” according to their spatio-temporal behaviour and how the use of type information can help in the learning process, especially in the presence of noise. I will present results from several domains including a kitchen scenario and an aircraft apron.