A multimodal embedding transfer approach for consistent and selective learning processes in cross-modal retrieval

Zhixiong Zeng, Shuyi He, Yuhao Zhang, Wenji Mao

Abstract: Highlights•Propose multimodal embedding transfer to consistently combine class-wise and pair-wise learning in Cross-Modal Retrieval.•Develop unified magin and label relaxation to selectively optimize informative multimodal samples for effective learning.•Design soft contrastive loss to realize multimodal embedding transfer mechanism and strategy for the learning process.•Experimental results demonstrate the superiority of our approach compared to competitive baselines for cross-modal retrieval.