Explainable Multimodal Regression via Information Decomposition

ICLR 2026 Conference Submission12515 Authors

Keywords: Partial Information Decomposition, Multimodal Regression, Cauchy-Schwarz Divergence, Interpretability.

Abstract: Multimodal regression aims to predict a continuous target from heterogeneous input sources and typically relies on fusion strategies such as early or late fusion. However, existing methods lack principled tools to disentangle and quantify the individual contributions of each modality and their interactions, limiting the interpretability of multimodal fusion. We propose a novel multimodal regression framework grounded in Partial Information Decomposition (PID), which decomposes modality-specific representations into unique, redundant, and synergistic components. The basic PID framework is inherently underdetermined. To resolve this, we introduce inductive biases by enforcing Gaussianity in the joint distribution of latent representations and the response variable, enabling analytical computation of PID terms. Additionally, we derive a closed-form conditional independence regularizer to promote the isolation of unique information within each modality. Experiments on six real-world datasets, including a case study on brain age prediction from multimodal neuroimaging data, demonstrate that our framework outperforms state-of-the-art methods in both predictive accuracy and interpretability, while also enabling informed modality selection and model pruning for efficient inference.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 12515