Fast Spatial Reasoning of Implicit 3D maps through Explicit Near-Far Sampling Range Prediction

Chaerin Min, Sehyun Cha, Changhee Won, Jongwoo Lim

Abstract: 3D mapping is critical for many robotics applications, such as autonomous navigation and object manipulation. Recently, deep implicit mapping approaches have received much attention for their compactness and ability to represent fine-grained details. However, without explicit guidance, such implicit representations are often cumbersome for searching the full range on the rays to find the object surfaces. As a result, several approaches, including hierarchical sampling, occupancy grids, and zero-level set baking, have been proposed to improve sampling where costly forward passes of the neural network should be performed. However, hierarchical sampling is still suboptimal in that it requires uniform coarse samples. Discrete occupancy grids of Instant NGP and zero-level sets of various baking methods are less suitable for large and noisy real scenes.In this paper, we present a novel framework for adaptively predicting the near-far range for sampling the query positions of the deep implicit map. For this purpose, the truncated signed distance grid for the map is pre-constructed and used to provide hints for near-far prediction during rendering. In addition, our recovery algorithm automatically detects failed near-far predictions and recovers only those rays by directly using the implicit map. We conduct extensive experiments on a synthetic dataset, a public real dataset, and a real dataset captured by our multi-camera robot system. The experimental results show that our algorithm achieves the same rendering quality with surprisingly fewer samples compared to the existing methods, which means that the robot can reason about the image and depth properties of the scene much faster. Finally, a thorough analysis of the sample distribution along the rays is provided to give a better understanding of our method’s strong efficiency, adaptability, and robustness. https://chaerinmin.github.io/TSDF-sampling/