Fine-grained Separation of Action-Background for Point-Level Temporal Action Localization

Yone Huang, Ying Xu, Fusen Li

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Temporal action localization, point-level supervision, weakly-supervised learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Be sure to read the appendix

Abstract: Due to the limitation of coarse-grained video-level labels, the action-background confusion is a tough problem for the weakly-supervised temporal action localization. Point-level temporal action localization recently utilizes point-level labels to overcome this difficulty to some extent. However, considering the sparsity of point-level labels, existing methods still lack the ability to effectively eliminate false positive action proposals. To address this issue, in this paper, we propose a new framework to provide guidance for fine-grained separation of action-background for the model. Specifically, the framework relies on annotated single frame labels to extend the original action features and generate dense pseudo labels, providing the model with more precise position information. Based on this information, the framework generates pseudo segment-level labels from video sequences and utilizes our proposed score contrast module and feature separation module, which are different from the previous works,to amplify the differences in scores and features between segment labels. Extensive experiments on four benchmarks verify the effectiveness of our proposed framework, and demonstrate that our method is significantly superior to previous state-of-the-art methods and obtains 3.9\% performance gains in terms of the average mAP on THUMOS’14.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1718