Bandwise Attention in CycleGAN for Fructose Estimation from Hyperspectral Images

Divyani Tyagi, Tushar Sandhan

Abstract: We propose an innovative approach for generating Near-Infrared (NIR) hyperspectral images from Visible (VIS) hyperspectral imagery using our Bandwise Attention based CycleGAN(BA-CycleGAN). This framework introduces three key enhancements:(i) the integration of a Bandwise Attention mechanism within the architecture to appropriately attend to spectral bands in the hyperspectral images, (ii) the addition of a Spectral Angle Mapper(SAM) based consistency loss to preserve spectral characteristics crucial for hyperspectral imagery, (iii) replacing the convolution block with depthwise separable convolution block to significantly reduce the number of training parameters. The generated NIR images are subsequently concatenated with their corresponding VIS hyperspectral images, producing composite VIS-NIR hyperspectral datasets to improve fructose estimation through the classification of fruit sweetness levels. We conducted comprehensive experiments using three distinct datasets: VIS, NIR, and the combined VIS-NIR. Our findings demonstrate that the NIR images generated by BA-CycleGAN exhibit good spectral fidelity, closely mimicking the characteristics of actual NIR hyperspectral images. Moreover, the combined VIS-NIR dataset outperforms the individual VIS and NIR datasets in classifying fructose or sugar content levels in fruit.

External IDs:dblp:conf/icpr/TyagiS24