Generic Action Start Detection

Yuexi Zhang, Ming Chen, Yikang Li, Jenhao Hsiao, Octavia I. Camps, Chiuman Ho

Abstract: The online detection of action start in video data has witnessed an increase in attention from both academia and industry, for abundant use-cases (e.g., an alert mechanism in videos used for surveillance with an ability to automate the recording of key frames and timestamp). Conventional approaches heavily rely on frame-level annotations and other prior knowledge that can only be applied to limited categories. In this paper, we introduce Generic Action Start Detection (GASD): a new task that aims to detect the taxonomy-free action start in an online manner. Further-more, one novel yet simple design, 3D MLP-mixer based architecture with a multiscaled sampling training strategy, is proposed, which makes the GASD algorithm favorable for edge-device deployment. The GASD task is validated on two large-scale datasets, THUMOS'14 and ActivityNet1.2. Results demonstrate that the proposed architecture achieves the SOTA performance on the GASD task compared with other online action start detection algorithms.