Renderable Street View Map-Based Localization: Leveraging 3D Gaussian Splatting for Street-Level Positioning

Howoong Jun, Hyeonwoo Yu, Songhwai Oh

Abstract:

In this paper, we introduce a new method for street-level localization that first utilizes 3D Gaussian splatting in street-level localization problem. Robust localization with street-level real-world images such as street view is a major issue for autonomous vehicle, augmented reality (AR) navigation, and outdoor mobile robots. The objective is to determine the position and orientation of a query image that matches
a street view database composed of RGB images. However, given the limited information available in the street view images, accurately determining the location solely based on this data presents a significant challenge. To address this challenge, we propose a novel method called renderable street view map-based localization (RSM-Loc). This approach enhances the localization process by augmenting 2D street view images into a renderable map using 3D Gaussian splatting, to resolve street-level localization problems. Upon receiving a query RGB image without geometry information, the proposed method renders 2D images from a pre-made renderable map and compares image
pose similarities between the rendered images and the query image. Through iterations of this process, the proposed method eventually estimates the pose of the given query image. The experimental results demonstrate that RSM-Loc outperforms the baselines with neural-field-based localization. Additionally, we conduct deep analysis on the proposed method to show that our method can serve as a new concept for the street-level localization problem.