Multimodal complex event detection framework for wide area surveillance

Himaanshu Gupta, Li Yu, Asaad Hakeem, Tae Eun Choe, Niels Haering, M. Locasto

Abstract: There is an increasing demand for detecting complex events from heterogeneous sensor networks to support the Global War On Terror (GWOT). This paper presents a framework addressing several vital aspects of a multimodal complex event detection system for wide area surveillance. The paper discusses the key modules for an effective system including complex event specification, cross-camera tracking for wide area surveillance, simple and complex event detection from video and network data, and high-level complex event inference from multimodal data using Markov Logic Networks. A novel testing framework is also discussed for data collection, system debugging and performance evaluation.