Point Cloud Dataset Distillation

Deyu Bo, Xinchao Wang

TL;DR: A rotation-invariant dataset distillation method for point cloud datasets

Abstract: This study introduces dataset distillation (DD) tailored for 3D data, particularly point clouds. DD aims to substitute large-scale real datasets with a small set of synthetic samples while preserving model performance. Existing methods mainly focus on structured data such as images. However, adapting DD for unstructured point clouds poses challenges due to their diverse orientations and resolutions in 3D space. To address these challenges, we theoretically demonstrate the importance of matching rotation-invariant features between real and synthetic data for 3D distillation. We further propose a plug-and-play point cloud rotator to align the point cloud to a canonical orientation, facilitating the learning of rotation-invariant features by all point cloud models. Furthermore, instead of optimizing fixed-size synthetic data directly, we devise a point-wise generator to produce point clouds at various resolutions based on the sampled noise amount. Compared to conventional DD methods, the proposed approach, termed DD3D, enables efficient training on low-resolution point clouds while generating high-resolution data for evaluation, thereby significantly reducing memory requirements and enhancing model scalability. Extensive experiments validate the effectiveness of DD3D in shape classification and part segmentation tasks across diverse scenarios, such as cross-architecture and cross-resolution settings.

Lay Summary:

This work addresses the challenge of training deep models on large 3D point cloud datasets by distilling them into a compact, synthetic core that preserves task performance. To overcome the variability of point cloud orientations, we introduce a theoretical analysis showing that aligning rotation-invariant features between real and distilled samples is critical. Building on this insight, we develop a plug-and-play rotator module that automatically reorients all point clouds into a shared canonical frame, ensuring consistent feature learning across rotations.

Rather than directly optimizing a fixed set of synthetic point clouds, we design a lightweight, point-wise generator that produces clouds at arbitrary resolutions by modulating input noise. This flexibility allows models to train efficiently on low-resolution data while still enabling high-resolution evaluation, substantially reducing memory usage without sacrificing accuracy.

Extensive experiments on shape classification and part segmentation demonstrate that our method, DD3D, matches or outperforms existing distillation approaches under diverse architectures and resolution settings. These results validate DD3D’s ability to deliver state-of-the-art performance with a fraction of the data and computational resources typically required.

Primary Area: Applications->Computer Vision

Keywords: Dataset Distillation, Point Cloud

Submission Number: 3335