SMART-PC: Skeletal Model Adaptation for Robust Test-Time Training in Point Clouds

Ali Bahri, Moslem Yazdanpanah, Sahar Dastani, Mehrdad Noori, Gustavo Adolfo Vargas Hakim, David OSOWIECHI, Farzad Beizaee, Ismail Ben Ayed, Christian Desrosiers

TL;DR: SMART-PC enhances 3D point cloud classification under distribution shifts by learning skeletal representations that enable fast, robust, and real-time test-time adaptation.

Abstract:

Test-Time Training has emerged as a promising solution to address distribution shifts in 3D point cloud classification. However, existing methods often rely on computationally expensive backpropagation during adaptation, limiting their applicability in real-world, time-sensitive scenarios. In this paper, we introduce SMART-PC, a skeleton-based framework that enhances resilience to corruptions by leveraging the geometric structure of 3D point clouds. During pre-training, our method predicts skeletal representations, enabling the model to extract robust and meaningful geometric features that are less sensitive to corruptions, thereby improving adaptability to test-time distribution shifts.
Unlike prior approaches, SMART-PC achieves real-time adaptation by eliminating backpropagation and updating only BatchNorm statistics, resulting in a lightweight and efficient framework capable of achieving high frame-per-second rates while maintaining superior classification performance. Extensive experiments on benchmark datasets, including ModelNet40-C, ShapeNet-C, and ScanObjectNN-C, demonstrate that SMART-PC achieves state-of-the-art results, outperforming existing methods such as MATE in terms of both accuracy and computational efficiency. The implementation is available at: \url{https://github.com/AliBahri94/SMART-PC}.

Lay Summary: Modern 3D systems—like those used in robotics—often struggle when real-world conditions change, such as due to weather, lighting, or sensor noise. Our research introduces SMART-PC, a method that helps 3D recognition systems stay accurate even when the input data becomes messy or unexpected. Instead of relying on all surface points of a 3D object, our method first learns a simplified “skeleton” that captures the essential shape. This makes the system better at ignoring noise and generalizing to new conditions. SMART-PC also adapts quickly in real time, making it suitable for practical use in time-sensitive applications. We tested it across several challenging datasets, and it consistently outperformed other methods in both accuracy and speed.

Link To Code: https://github.com/AliBahri94/SMART-PC

Primary Area: Deep Learning->Robustness

Keywords: Point Cloud, Test Time Training, Skeletal Points

Submission Number: 2742