Weakly supervised lung cancer detection on label-free intraoperative microscopy with higher harmonic generation

Siem de Jong, Marie Louise Groot, Roel L. J. Verhoeven, Erik H. F. M. van der Heijden, Francesco Ciompi

Keywords: higher harmonic generation microscopy, self-supervised pre-training, weakly-supervised learning, classification, attention, explainability, intraoperative, lung cancer

Abstract:

Higher harmonic generation microscopy (HHGM) enables label-free on-site imaging of fresh tissue, potentially allowing a new means of pathology assessment for disease diagnosis.
We investigate the potential of using self-supervised learning (SSL) in combination with weakly-supervised, attention-based, clustering constrained multiple instance learning (CLAM) to detect lung cancer in HHGM images.
First, we tailor encoders to HHGM-specific data domain via both SimCLR and DINO SSL.
Second, we train a CLAM classifier with and without an SSL feature extractor on 100 HHGM images acquired during bronchoscopy procedures.
We show that SSL pre-training with random initialization and CLAM are beneficial to intraoperatively detect lung cancer in HHGM images.

Submission Number: 53