OmniFuse: A general modality fusion framework for multi-modality learning on low-quality medical data

Yixuan Wu, Jintai Chen, Lianting Hu, Hongxia Xu, Huiying Liang, Jian Wu

Abstract: Highlights•Highlight drawbacks in multi-modal learning: modality absence, imbalance, and noise.•Design a missing imputation module to handle any extreme modality absence scenario.•Propose dynamic weighted fusion to quantify relationships for imbalanced modalities.•Traceably identify and activate the lazy modality to eliminate noise.•Achieve state-of-the-art performance on various multi-sensor medical datasets.