Lite Class-prompt Tiny-VIT for Multi-Modality Medical Image Segmentation

Haotian Guan, Bingze Dai, Jiajing Zhang

Keywords: Multi-modality, Medical image segmentation, Class prompt, TinyVIT

Abstract: The increasing demand for accurate medical image segmentation is crucial for alleviating the workload of doctors and enhancing diagnostic accuracy, particularly in low-income countries with limited computational resources. This study investigates the application of a novel deep learning model, class-prompt Tiny-VIT, to segment various medical image modalities using a laptop. The primary focus is on the challenges posed by the significant differences across image modalities, which render a unified model ineffective in handling certain modalities like positron emission tomography (PET) with high dice similarity on the segmentation task. Experimental results demonstrate that the class prompt, a simplified yet efficient method, can effectively boost model performance on modalities such as PET and microscopy, achieving improved overall segmentation accuracy. This research holds significant potential for the practical implementation of medical image segmentation in resource-constrained settings, and underlines the importance of developing deep learning algorithms tailored to specific medical imaging modalities.

Submission Number: 14