Spatio-Temporal Feature Extraction/Recognition in Videos Based on Energy Optimization

Hidetomo Sakaino

Abstract: Videos are spatio-temporally rich in static to dynamic objects/scenes, sparse to dense, and periodic to non-periodic motions. Particularly, the dynamic texture (DT) exhibits complex appearance and motion changes that remain a challenge to deal with. This paper presents an energy optimization method for feature extraction and recognition in videos. For noise and background jitter, the Tikhonov regularization with eigen-vector and Frenet-Serret formula-based energy constraints is also proposed. The different periodicity of DT can be adapted by the time-varying number of learning temporal frames. The optimal duration of an image sequence is determined from the temporal property of its eigen-values. Unlike the state-of-the-art recognition methods, i.e., sparse coding and slow feature analysis, the proposed method can capture the physical property of objects and scenes: velocity, acceleration, and orientation. Also, the static and dynamic image regions can be locally classified. Owing to these spatio-temporal features, stability, robustness, and accuracy of feature extraction and recognition are enhanced. Using DT videos, the superiority of the proposed method compared to the state-of-the-art recognition methods is experimentally shown.