On the Robustness and Reliability of Late Multi-Modal Fusion using Probabilistic Circuits

Sahil Sidheekh, Pranuthi Tenali, Saurabh Mathur, Erik Blasch, Sriraam Natarajan

Abstract: Multimodal fusion is important for building intelligent systems that exploit patterns across diverse data sources for improved decision-making. However, the reliability and robustness of these systems in safety-critical domains are often compromised by the inherent noise and incompleteness of data. Probabilistic Circuits (PCs) have recently emerged as a promising approach for late (or decision) fusion. Their strength lies in being both expressive and capable of inferring source credibility due to their ability to tractably perform exact probabilistic inference. However, their ability to handle missing data and their reliability in practical scenarios remains underexplored. This work investigates the robustness of PCs as fusion functions in scenarios with missing and noisy data; particularly by examining their impact on the calibration and reliability of the resulting classifiers. Our findings show that PCs not only enable the modeling of complex correlations across modalities but also lead to calibrated and reliable classifiers, highlighting their potential as a robust fusion mechanism in multimodal systems.