Recovering spectral reflectance under commonly available lighting conditions

Jun Jiang, Jinwei Gu

Abstract: Recovering the spectral reflectance of a scene is important for scene understanding. Previous approaches use either specialized filters or controlled illumination where the extra hardware prevents many practical applications. In this paper, we propose a method that accurately recovers spectral reflectance from two images taken with conventional consumer cameras under commonly available lighting conditions, such as daylight at different times over a day, camera flash and ambient light, and fluorescent and tungsten light. Our approach does not require camera spectral sensitivities or the spectra of the illumination, which makes it easy to implement for a variety of practical applications. Based on noise analysis, we also derive theoretical predictors that answer: (1) which two lighting conditions lead to the most accurate spectral recovery overall, and (2) for two given lighting conditions, which spectral reflectance is more likely to be estimated accurately. We implement the method on a variety of cameras from high-end DSLRs to cellphone cameras, and apply the recovered spectral reflectance for several applications such as fine art reproduction, fruit identification, and material classification. Both simulation and experimental results demonstrate the effectiveness of the proposed method.