Real-time Monocular SLAM with Stochastic Local Newton Optimized 3D Gaussian Splatting

ICLR 2026 Conference Submission21002 Authors

Keywords: Robotics Perception, Gaussian Splatting SLAM

Abstract: We present FSGS, a novel monocular SLAM system that integrates Stochastic Local Newton optimization with 3D Gaussian Splatting (3DGS) for real-time dense reconstruction and accurate camera tracking. While existing methods often struggle with the computational burden of optimizing millions of Gaussian parameters, our approach employs a parameter-specific second-order optimization that substantially improves convergence speed while maintaining mapping quality. By sequentially optimizing position, orientation, scaling, opacity, and color parameters through local Newton solves, we achieve efficient updates without the computational overhead of global Hessian calculations. Our method leverages structured spatial relationships between keyframes through a K-nearest neighbor approach, employing secondary targets as preconditioners to prevent optimization overshoot. Experimental evaluation on TUM RGB-D datasets demonstrates that FSGS achieves competitive tracking accuracy (RMSE ATE < 1.5cm) while providing high-fidelity dense reconstructions at interactive rates. The system's robust performance across various indoor environments highlights the effectiveness of combining second-order optimization techniques with neural rendering for real-time SLAM applications.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 21002