Tumor Semantic Segmentation in Hyperspectral Images using Deep Learning

Stojan Trajanovski, Caifeng Shan, Pim J. C. Weijtmans, Susan G. Brouwer de Koning, Theo J. M. Ruers

Keywords: Tumor Segmentation, Hyperspectral imaging, Deep Learning

TL;DR: Improved tumor semantic segmentation in hyperspectral images using U-Net variant, channel selection and both VIS and NIR spectra.

Abstract: Real-time feedback based on hyperspectral images (HSI) to a surgeon can lead to a higher precision and additional insights compared to the standard techniques. To the best of our knowledge, deep learning with semantic segmentation utilizing both visual (VIS) and infrared channels (NIR) has never been exploited with the HSI data with human tumors. We propose using channels selection with U-Net deep neural network for tumor segmentation in hyperspectral images. The proposed method, based on bigger patches, accounts for bigger spatial context and achieves better results (average dice coefficient $0.89 \pm 0.07$ and area under the ROC-curve AUC $0.93 \pm 0.04$) than pixel-level spectral and structural approaches in a clinical data set with tongue squamous cell carcinoma. The importance of VIS channel for the performance is higher, but NIR contribution is non-negligible.

Code Of Conduct: I have read and accept the code of conduct.

Remove If Rejected: Remove submission from public view if paper is rejected.