Continual Learning of 3D Point Cloud with Hyperbolic Manifold Replay

Jin Yu, Zihao Xu, Zhengyu Li, Mingsong Chen, Xuan Tang, Xian Wei

Abstract: As an irregular and sparse form of data, 3D point cloud data is widely used in the fields of computer vision and machine learning. Its effective classification and recognition are of great significance. In many cases, task data does not arrive all at once but is acquired in batches over time. However, in deep learning, continual learning faces the problem of catastrophic forgetting, where the model loses memory of old tasks while learning new ones. How to retain old knowledge while learning new knowledge is a significant challenge. To address these challenges, existing replay-based strategies alleviate this issue by storing small portions of old samples. However, this often leads to an imbalance between new and old data, affecting performance. Additionally, point cloud data typically has complex non-Euclidean structures, with potential hierarchical relationships within and between point cloud objects. Current deep learning models based on Euclidean space struggle to capture the hierarchical prior features of point clouds. Therefore, in this paper, we introduce a continual learning method based on a replay mechanism and explore how to incorporate hyperbolic space into continual learning tasks to enhance feature representation capabilities. We propose a manifold replay strategy in hyperbolic space, termed HyMR. Specifically, this paper presents a knowledge distillation strategy that combines global and local information, and utilizes manifold spherical projection to select representative old data for replay. Experiments demonstrate that this method achieves good results in continual learning tasks on point cloud datasets such as ShapeNet and ModelNet.