RLD-GS: Reinforcement Learning-Driven Gaussian Splatting for High-Fidelity Neural Rendering

Zexu Huang, Min Xu, Stuart Perry

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a breakthrough approach for real-time neural rendering, excelling in balancing rendering quality and computational efficiency. Despite its advantages, existing methodologies encounter significant challenges, including edge degradation, resolution loss, and inefficient resource allocation. These issues arise primarily from the static nature of kernel formulations and suboptimal primitive distributions, limiting their adaptability to varying scene complexities and viewpoint conditions. To address these limitations, we introduce RLD-GS, a novel framework characterized by two key technical contributions. First, we propose an adaptive multi-kernel Gaussian mechanism utilizing a differentiable parameterization. This method dynamically predicts and adjusts kernel properties, such as size and shape, according to local scene complexities and viewing angles. As a result, it ensures precise preservation of details, particularly enhancing visual quality in complex scene regions. Second, we reformulate the anchor distribution problem as a Markov Decision Process (MDP) and employ a reinforcement learning (RL)-based optimization strategy. This strategy intelligently reallocates computational resources in real-time, prioritizing areas of greater perceptual importance. By doing so, our approach significantly improves visual quality without incurring additional rendering overhead. Comprehensive evaluations conducted on both synthetic datasets and real-world captures demonstrate that RLD-GS achieves superior quantitative and qualitative results compared to existing state-of-the-art techniques, highlighting its effectiveness and practical applicability.

External IDs:doi:10.1007/978-981-95-4378-6_19