Learning representative viewpoints in 3D shape recognition

Huazhen Chu, Chao Le, Rongquan Wang, Xi Li, Huimin Ma

Abstract: Adopting many viewpoints and mining the relationship between them, 3D shape recognition inferring the object’s category from 2D rendered images has proven effective. However, using a limited number of general representative viewpoints to form a reasonable expression of the object is a task with both practical and theoretical significance. This paper proposes a multi-view CNN architecture with independent viewpoint feature extraction and the unity of importance weights, which can dramatically decrease the number of viewpoints by learning the representative ones. First, the view-based and independent view features are extracted by a deep neural network. Second, the network automatically learns relativity between these viewpoints and outputs the importance weights of views. Finally, view features are aggregated to predict the category of objects. Through iterative learning of these critical weights in instances, global representative viewpoints are selected. We assess our method on two challenging datasets, ModelNet and ShapeNet. Rigorous experiments show that our strategy is competitive with the latest method using only six viewpoints and RGB information as input. Meanwhile, our approach also achieves state-of-the-art performance by using 20 viewpoints as input. Specifically, the proposed approach achieves 99.34% and 97.49% accuracy on the ModelNet10 and ModelNet40, and 80.0% mAP on ShapeNet.