Diffusion-Denoised Hyperspectral Gaussian Splatting

Sunil Kumar Narayanan, Lingjun Zhao, Lu Gan, Yongsheng Chen

Keywords: Hyperspectral imaging, Gaussian Splatting, Diffusion models, Novel View Synthesis, 3D Vision

TL;DR: We propose a diffusion-denoised hyperspectral Gaussian splatting (DD-HGS) framework that integrates diffusion-based denoising into 3D Gaussian splatting for accurate and spatially consistent hyperspectral novel view synthesis.

Abstract: Hyperspectral imaging (HSI) has been widely used in agricultural applications for non-destructive estimation of plant nutrient composition and precise quantification of sample nutritional elements. Recently, 3D reconstruction methods, such as Neural Radiance Field (NeRF), have been used to create implicit neural representations of HSI scenes. This capability enables the rendering of hyperspectral channel compositions at every spatial location, thereby helping localize the target object's nutrient composition both spatially and spectrally. However, it faces limitations in training time and rendering speed. In this paper, we propose Diffusion-Denoised Hyperspectral Gaussian Splatting (DD-HGS), which enhances the state-of-the-art 3D Gaussian Splatting (3DGS) method with wavelength-aware spherical harmonics, a Kullback–Leibler divergence-based spectral loss, and a diffusion-based denoiser to enable 3D explicit reconstruction of the hyperspectral scenes for the entire spectral range. We present extensive evaluations on diverse real-world hyperspectral scenes from the Hyper-NeRF dataset to show the effectiveness of our DD-HGS. The results demonstrate that DD-HGS achieves the new state-of-the-art performance compared to all the previously published methods. Project page:https://dragonpg2000.github.io/DDHGS-website/

Submission Number: 354