MITER: Medical Image-TExt joint adaptive pretRaining with multi-level contrastive learning

Chang Shu, Yi Zhu, Xiaochu Tang, Jing Xiao, Youxin Chen, Xiu Li, Qian Zhang, Zheng Lu

Abstract: Highlights•Novel joint pretraining framework for medical image/text via contrastive learning.•Uni-modal procedure effectively exploits spontaneous relationship.•Effective negative sampling mechanism using Alignment and Uniformity properties.•Significantly outperforms SoTA models on uni-modal and cross-modal tasks.