A Novel Framework for Generating Realistic and Semantic Digital Model of Reflective Glass Façades Using RefGlass-3DGS

Zhenyu Liang, Xiao Zhang, Zhaolun Liang, Boyu Wang, Mingzhu Wang, Jack C.P. Cheng

Abstract: Reflective glass façades are a popular choice in modern urban architecture, yet their 3D reconstruction and semantic digitalization pose significant challenges due to two key issues: (1) the view-dependent textures of reflective glass adversely impact geometric and texture reconstruction; and (2) texture interference, coupled with the lack of comparative materials, complicates the accurate segmentation of glass panels. Conventional 3D representations, such as meshes or point clouds, are limited to static textures and struggle to effectively demonstrate dynamic textures. Furthermore, existing glass segmentation methods, including zero-shot approaches like SAM and deep learning-based technologies, are insufficient for segmenting individual glass panels across an entire glass curtain wall. To address these challenges, this study proposes a novel framework for generating realistic and semantic 3D digital models of reflective glass façades using drone imagery. The scientific contributions include (1) a 2D glass panel segmentation method leveraging structural regularities and back-projection, adaptable to multi-view images with varying panel sizes; (2) the RefGlass-3DGS algorithm for generating 3D models with view-dependent textures and semantic annotations; and (3) an optimized UAV view-planning strategy based on full ray coverage, ensuring comprehensive capture of dynamic textures. Validation on a Hong Kong office building demonstrates the framework’s effectiveness in reconstructing and semantically digitalizing reflective glass façades.