Learning Multimodal Representations from Partially Paired, Small-Scale Data

ICLR 2026 Conference Submission18323 Authors

Keywords: multimodal learning, partially paired data, computational pathology

TL;DR: CAMEO is a multimodal fusion model that enables effective learning from small, partially paired biomedical datasets, outperforming unimodal baselines and showing stronger resilience to reduced pairing than existing multimodal baselines.

Abstract: Modeling multimodal data from partially paired samples is critical for advancing domains like biomedicine, where vast unimodal datasets and foundation models exist but paired data remains scarce. Existing fusion methods rely on large-scale paired datasets, limiting their use in scenarios with incomplete pairing. We introduce CAMEO, an adversarial learning-based modality fusion framework that integrates modalities from small, partially paired datasets. Combining any pre-trained unimodal encoder with a cross-modal latent alignment mechanism, CAMEO learns shared representations requiring only minimal paired samples. Evaluated on computational pathology tasks such as niche classification and cell type composition prediction, CAMEO achieves superior data efficiency, outperforming contrastive approaches like CLIP in low paired-data regimes, and highlighting the benefits of adversarial alignment when paired annotations are scarce. To facilitate further research, we additionally release a fully annotated HuggingFace dataset comprising three organs and paired image and gene expression modalities. By extending fusion methods to address limited pairing and small-scale datasets, we provide a framework that advances multimodal learning and broadens its applicability to real-world biomedical problems.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 18323