Graph Neural Network based on Geometric and Appearance Attention for 6D Pose Estimation

Tianfu Wang, Hongguang Wang

Abstract: Object 6D pose estimation from RGB-D images requires extracting useful information from two complementary data sources (color and depth). Usually, previous methods extract from two networks (for color and depth) separately and concatenate the final features. In this work, we propose a novel attention based graph neural network to process color and depth images concurrently. In this way, we can learn the interactions between depth and color, leading to an effective fusion. Specifically, our method consists of two stages. First, we convert the segmentation of an RGB-D image into colored point clouds. Second, we estimate the poses of objects by an attention based graph neural network. Our attention mechanism is based on geometric and appearance information. In this way, 3D geometric and 2D appearance information can be fully utilized for better feature learning. The experimental results on LineMOD and Occlusion LineMOD datasets show the effectiveness of our method.