Beyond Static Snapshots: Physically-Based Deformable and Relightable 2D Gaussians

Jiaxin LI, Tong Wu, Yi Wei, Tailin Wu, Li Zhang

Keywords: gaussian splatting, computer graphics, simulation and animation, inverse rendering

Abstract: Gaussian splatting (GS), as a cutting-edge technique in 3D reconstruction, has garnered significant attention in VR/AR and digital content creation due to its explicit parameterization and efficient rendering capabilities. However, existing GS-based methods for deformable objects face two key limitations: (i) illumination is erroneously baked into textures, causing physically inconsistent responses under dynamic deformations and lighting changes; (ii) snapshots-based reconstruction restricts post-reconstruction material editing. To address these challenges, we propose Deformable and Relightable (DR-GS), a unified Gaussian framework that integrates physically-based inverse rendering, relighting, and deformation-aware manipulation. Through explicitly disentangling geometry, illumination, and material representations, DR-GS overcomes the limitations of static snapshots, resolving unrealistic appearance under varying conditions while enabling post-reconstruction parameter editing. Experimental results show that DR-GS not only attains state-of-the-art visual fidelity, with particular strength on glossy surfaces, but also creates a fully decoupled geometry-illumination-material pipeline, which enables both high-quality 3D asset creation and comprehensive editing.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 5016