PolypDINO: Adapting DINOv2 for Domain Generalized Polyp Segmentation

Mansoor Ali, Raneem Toman, Gilberto Ochoa-Ruiz, Sharib Ali

Abstract: Despite the effectiveness of colonoscopy as a screening method for lesion detection, there is still a high polyp missing rate due to operator dependence as well as complex polyp morphology variations. Deep learning (DL)-based solutions have demonstrated their potential in assisting clinicians to improve this issue through the use of computer vision-based CAD tools. One of the major bottlenecks in applying these methods in real-world scenarios is their lack of generalizability under domain shifts due to the failure to capture the most salient features from the source training domain. In this work, we propose PolypDINO to exploit the learned parameters of the pre-trained visual foundation model (DINOv2) and adapt with Low-Rank adaption (LoRA) for domain generalized polyp segmentation. Precisely, we fine-tune DINOv2 with LoRA on the Kvasir-SEG dataset and perform generalizability tests on the out-of-distribution (OOD) PolypGen dataset containing data comprising six independent centers. Quantitative results show a consistent improvement in all test data, for instance, outperforming baseline state-of-the-art (SOTA) method by nearly 4% and 3% in terms of mean Intersection-over-Union and mean Dice scores, respectively. Code is available at https://github.com/Mansoor-at/PolypDINO.

External IDs:doi:10.1007/978-3-031-98694-9_14