ART-InvRec: Acquiring Rotation Invariance of 3D Object Reconstruction via Adversarial Rotation

Rui Yang, Fei Hu, Zhiye Chen, George Wang

Abstract: 3D surface reconstruction from point clouds has shown promising advancements in the 3D computer vision community. However, most proposed approaches only pursue the reconstruction precision of objects in an aligned pose without considering the rotated inputs, thus failing to reconstruct stable results for rotated point clouds. This study introduces a novel perspective by demonstrating, for the first time, that achieving rotation invariance in 3D object reconstruction is feasible through adversarial training. Our proposed framework, termed ART-InvRec, treats point cloud rotation as an adversarial attack and attains rotation invariance by training the reconstruction network on inputs subjected to Adversarial RoTations. Specifically, we employ the axis-wise angles attack method to efficiently identify the most aggressive rotations and train the target reconstruction model with the rotation pool mechanism. Experiments demonstrate that ART-InvRec gets outstanding results, both qualitatively and quantitatively, for the challenging task of rotation-invariant object reconstruction. Notably, ART-InvRec performs better than state-of-the-art techniques on rotation-invariant reconstruction.