Pseudo Pansharpening NeRF for Satellite Image Collections

Emilie Pic, Thibaud Ehret, Gabriele Facciolo, Roger Marí

Abstract: The use of NeRF to model 3D scenes from satellite images is becoming increasingly common. However, the models proposed to date assume the availability of pre-processed RGB images as input. This contrasts with the multispectral nature of raw satellite products. Optical satellite sensors do not acquire RGB images but a wider variety of spectral bands, which may have different spatial resolution. We propose a NeRF framework to simultaneously handle panchromatic data and lower resolution spectral bands (e.g., color bands), and investigate the contribution of the low-resolution bands to the output model. Our method achieves comparable or better results with respect to previous approaches that rely on a separate pansharpening step. The model can also be used to generate a pansharpened image surrogate for each input view, as it natively performs super-resolution in the color bands.