Bayesian Knowledge Distillation for Online Action Detection

Hongji Guo, Hanjing Wang, Qiang Ji

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Online action detection, knowledge distillation, mutual information, uncertainty quantification

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Online action detection aims at identifying the ongoing action in a streaming video without seeing the future. Timely and accurate response is critical for real-world applications. In this paper, we introduce Bayesian knowledge distillation (BKD), an efficient and generalizable framework for online action detection. Specifically, we adopt a teacher-student architecture. During the training, the teacher model is built with a Bayesian neural network to output both the feature mutual information that measures the informativeness of historical features to ongoing action and the detection uncertainty. For efficient online detection, we also introduce a student model based on the evidential neural network that learns the feature mutual information and predictive uncertainties from the teacher model. In this way, the student model can not only select important features and make fast inference, but also efficiently quantify the prediction uncertainty by a single forward pass. We evaluated our proposed method on three benchmark datasets including THUMOS'14, TVSeries, and HDD. Our method achieves competitive performance with much better computational efficiency and much less model complexity. We also demonstrate that BKD generalizes better and is more data-efficient by extensive ablation studies. Finally, we validate the uncertainty quantification of the student model by performing abnormal action detection.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1447