Weakly supervised multi-modal contrastive learning framework for predicting the HER2 scores in breast cancer

Jun Shi, Dongdong Sun, Zhiguo Jiang, Jun Du, Wei Wang, Yushan Zheng, Haibo Wu

Abstract: Highlights•A novel weakly supervised framework named WSMCL is proposed for HER2 scoring.•A multi-modal contrastive learning module is designed for semantic alignment.•Multi-modal same-class pairing strategy mitigates same-case data acquisition.•WSMCL outperforms other state-of-the-art methods in HER2 scoring tasks.