Demo: A Unified Lesion Detection and Segmentation Pipeline Across Diverse Medical Imaging Modalities

Shivaram V, Jey Praveen Sivaraj, Lakshmi Subramanian

Keywords: unified lesion detection, multi-modality segmentation, medical imaging, CT, Echo, X-Ray, modality-agnostic pipeline, foundation models, TotalSegmentator, MedSAM

Abstract: Accurate lesion detection across diverse medical imaging modalities is critical for clinical decision-making. However, variability in lesion appearance across CT, Echo and X-Ray persists as a challenge. Existing approaches are often modality-specific, requiring extensive retraining. We propose a unified, modality-agnostic pipeline integrating complementary foundation models for comprehensive lesion analysis. Multi-modality inputs undergo modality-specific preprocessing, followed by organ segmentation using TotalSegmentator and lesion segmentation via MedSAM. Detected lesions are mapped to anatomical context through spatial associations with segmented organs. We extract radiomic, morphological, and spatial descriptors from each lesion, subsequently grouped using unsupervised clustering for automated characterisation. Validation across CT, Echo, and X-ray datasets demonstrates high sensitivity and precision in lesion detection and lesion–organ association. Our pipeline matches or outperforms modality-specific models while providing enhanced interpretability through organ context linking and clinically meaningful categorisation.

Submission Number: 18