ALMEA: Active Learning-Enhanced Multimodal Entity Alignment with Semantic Modality Imputation

Xizhe Zhang, Meng-Fen Chiang, Jingfeng Zhang

Keywords: Multimodal Entity Alignment, Missing Modality Imputation, Latent Semantics Calibration, Active Learning

TL;DR: We propose a multimodal entity alignment framework that combines semantic calibration and active learning to handle missing modalities and inconsistencies in the low-resource multimodal knowledge graphs settings.

Abstract:

Multimodal knowledge graphs (MMKGs) offer enriched knowledge representation by integrating structural, visual, and textual information from heterogeneous sources. However, existing multimodal entity alignment (MMEA) approaches face significant challenges due to missing modalities and semantic inconsistencies across sources. These limitations compromise alignment robustness, especially in low-resource scenarios with limited seed pairs (i.e., manually annotated aligned entities as supervision).

To bridge the gap, we propose **Active Learning for Multimodal Entity Alignment with Semantic Imputation (ALMEA)**, a MMEA framework that integrates semantic calibration and active learning to improve alignment. Specifically, ALMEA synthesizes embeddings for missing modalities and refines semantic representations to address inconsistencies across MMKGs. This approach iteratively selects optimal candidate pairs within the learnable budget through active learning strategies, thereby acquiring richer modal information in low-resource scenarios.

On the benchmark MMKG dataset, experimental results indicate that ALMEA consistently outperforms state-of-the-art baseline models under the low-resource scenario, achieving average improvements of **5.16% in Mean Reciprocal Rank (MRR)** and **5.57% in Hits at Top-1 (Hits@1)**.

Our anonymized code is available at [github.com/RTX4090123/ALMEA](https://github.com/RTX4090123/ALMEA).

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 5953