Event Detection over Live and Archived Streams

Shanglian Peng, Zhanhuai Li, Qiang Li, Qun Chen, Wei Pan, Hailong Liu, Yanming Nie

Abstract: It is becoming increasingly crucial to integrate pattern matching functionality over live and archived event streams with hybrid event queries for various complex event processing(CEP) applications. As existing Stream Processing Engine(SPE) and DBMS alone can not accommodate hybrid event queries, we investigate system integration issues and scheduling algorithm optimizations of hybrid event processing based on discrete pattern in a single framework. First, hierarchical event stream storage is introduced to synchronize data access over live/archived event streams. Second, live and historical partial pattern matching caching mechanisms are proposed to provide effective partial pattern match search and update. Third, to reduce database access overhead, sub-window based event detect scheduling algorithms are proposed. Empirical performance study in a prototype hybrid event processing engine demonstrates effectiveness of our approaches.