Multimodal Fusion Framework Using Contrastive Learning for Exposure Keratopathy

Gyutae Oh, Yeokyoung Won, Dong Hui Lim, Jitae Shin

Abstract: In this study, we propose a two-stage multimodal fusion learning framework for the automated grading of exposure keratopathy using four complementary imaging modalities: broad-beam, slit-beam, scatter, and blue-light, collected directly at a tertiary care center. In stage 1 (Grade Based Learning&Beam Based Learning), a backbone network is trained to capture and fuse the unique anatomical and pathological features inherent to each modality. In stage 2 (Dynamic Feature Fusion), we leverage the stage 1 pretrained backbone to train a modality agnostic classifier that, given only a single broad-beam image at inference time, implicitly exploits the rich multispectral information of the other three modalities. Experimental results demonstrate that our method achieves an average improvement of over 16% in both F1 score and overall accuracy (ACC) compared to single modality baselines. Ablation studies confirm the significant contribution of each component. By requiring only a single modality at inference, this framework is expected to maintain high diagnostic performance in real-world clinical settings while substantially reducing imaging requirements and patient burden. The code is publicly available at https://github.com/GYUGYUT/Multimodal-Fusion-Framework-Using-Contrastive-Learning-for-Exposure-Keratopathy.

External IDs:dblp:conf/miccai/OhWLS25