Object SLAM Based on Spatial Layout and Semantic Consistency

Xingwu Ji, Peilin Liu, Haochen Niu, Xiang Chen, Rendong Ying, Fei Wen

Abstract: Visual simultaneous localization and mapping (SLAM) systems face challenges in detecting loop closure under the circumstance of large viewpoint changes. In this article, we present an object-level SLAM system based on the spatial layout and semantic consistency of the 3-D scene graph. Firstly, we propose an object-level data association approach based on the semantic information from semantic labels, intersection over union (IoU), object color, and object embedding. Subsequently, multi-view bundle adjustment (BA) with the associated objects is utilized to jointly optimize the poses of objects and cameras. We represent the refined objects as a 3-D spatial graph with semantics and topology. Then, we propose a graph matching approach to find corresponding objects based on the spatial layout and semantic property similarity of vertices’ neighbors. Finally, we jointly optimize camera trajectories and object poses in an object-level pose graph optimization, which results in a globally consistent map. Experimental results demonstrate that our proposed data association approach can construct more accurate 3-D semantic maps, and our loop closure method is more robust than point-based and object-based methods in circumstances with large viewpoint changes.