Multimodal Alignment Reveals Interpretable Gene–Morphology Links in Perineuronal Net Pathology

Wenjun Liu, Saeed Hassanpour

Keywords: Multimodal Learning, Perineuronal nets

Abstract: Perineuronal nets are extracellular matrix structures that enmesh specific neurons, and their disruption has been linked to glioma progression and epilepsy. Yet most studies analyze pathology images, gene expression, or clinical variables in isolation, limiting our understanding of how perineuronal net changes connect to disease. We present a joint multimodal framework that learns aligned embeddings from three inputs: pathology images, RNA expression, and clinical covariates, using a contrastive objective with cross-modality reconstruction and pathway-informed regularization. The approach supports missing modalities via modality dropout and gated fusion at inference, and provides interpretability through pathway enrichment analyses and attention maps that highlight morphology consistent with perineuronal net biology. On a small, patient-level multimodal cohort, the method outperforms early/intermediate/late fusion and unimodal baselines and yields transparent gene--morphology associations, suggesting a practical route to integrating limited multimodal data for perineuronal net pathology.

Submission Number: 85