JI-ADF: Joint–Individual Learning with Adaptive Decision Fusion for Multimodal Skin Lesion Classification

Phan Nguyen, Dat Cao, Hien Quang Kha, Trang Q.T. Pham, Nguyen Quoc Khanh Le

Keywords: Multimodal Learning, Skin Lesion Classification, Adaptive Fusion, Dermoscopic Imaging, Clinical Metadata, Medical Image Analysis

TL;DR: We introduce an adaptive trimodal fusion framework that dynamically weights dermoscopic images, clinical photos, and metadata to achieve state-of-the-art skin lesion classification.

Abstract: Skin lesions encompass a wide spectrum of dermatological conditions, and their accurate classification is critical for early diagnosis and treatment planning. The development of deep learning-based Computer-Aided Diagnosis (CAD) system has shown promise in supporting dermatologists, particularly when leveraging multiple data modalities such as dermoscopic images, clinical images, and patient metadata. However, existing multimodal approaches often rely on late fusion or naive concatenation strategies, which fail to capture fine-grained cross-modal interactions. In this paper, we propose \textbf{JI-ADF} (Joint Individual with Adaptive Decision Fusion), a trimodal deep learning framework designed to address these limitations. Our approach encodes each modality independently, then unifies them via a cross-attention-based fusion mechanism to capture complementary, fine-grained interactions across dermoscopic, clinical and structured metadata views. We evaluate our method on the MILK10k dataset, a newly released multimodal benchmark spanning 11 diagnostic classes. On the hidden test set, JI-ADF achieves a high AUC (0.866) and Accuracy (0.930), along with high Specificity (0.959) and NPV (0.960). These results outperform recent baseline methods and highlight the effectiveness of our model for real-world multimodal dermatological diagnosis.

Primary Subject Area: Detection and Diagnosis

Secondary Subject Area: Application: Dermatology

Registration Requirement: Yes

Visa & Travel: No

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 158