Motion PointNet: Solving Dynamic Capture in Point Cloud Video Human Action

Zhuoxu Huang, Zhenkun Fan, Tao Xu, Jungong Han

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: 3D point cloud, computer vision, action recognition

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Motion representation plays a pivotal role in understanding video data, thereby elevating the dynamic capture to the forefront of action recognition tasks based on point cloud video. Previous works mainly compute the motion information in an unguided way, e.g. aggregate the spatial variations on adjacent point cloud frames using 4D convolutions or capture a point trajectory with kinematic computation like scene flow. However, the former fails to explicitly consider motion representation in corresponding frames, and the latter's reliance on tracking point trajectories becomes impractical in real-life applications due to the potential inter-frame migration of points. In this paper, we tackle the dynamic capture in point cloud video action by formulating it as solvable partial differential equations (PDEs) in feature space. Based on this intuitive design, we propose Motion PointNet, a novel method that improves the dynamic capture in point cloud video human action by constructing clear guidance for network learning. Motion PointNet is composed of a lightweight yet effective PointNet-like encoder and a PDEs-solving module for dynamic capture. Remarkably, our Motion PointNet, with merely 0.72 M parameters and 0.82 G FLOPs, achieves an impressive accuracy of 97.52 % on the MSRAction-3D dataset, surpassing the current state-of-the-art in all aspects. The code and the trained models will be released for reproduction.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1267