Crowd analytics via one shot learning and agent based inference

Peter H. Tu, Ming-Ching Chang, Tao Gao

Abstract: For the purposes of inferring social behavior in crowded conditions, three concepts have been explored: 1) the GE Sherlock system which makes use of computer vision algorithms for the purposes of the opportunistic capture of various of social cues, 2) a one shot learning paradigm where behaviors can be identified based on as few as a single example and 3) an agent based approach to inference where generative models become the basis for social behavior recognition. The Sherlock system makes use of tracking, facial analysis, gaze estimation and upper body motion analysis. The one-shot learning paradigm makes use of semantically meaningful affects as descriptors. The agent based inference methods allows for the incorporation of cognitive models as a basis for inference.