NeuroHydra: A Generalizable DINOv3–Mamba Framework with Structure-Aware Visual Slices Fusion for Multimodal Biomedical AI

Bo Wang, Yuhan Wang, Christopher Eckstein, Timothy A Pagliaro

Keywords: multimodal biomedical AI, medical image fusion, MRI deep learning, self-supervised vision models, DINOv3, state-space models, Mamba architecture, structure-aware slice fusion, volumetric representation learning, clinical data integration, epilepsy outcome prediction, neuroimaging biomarkers, interpretable AI in healthcare, Grad-CAM explainability, SHAP feature attribution, hierarchical attribution, sequential multimodal reasoning, computationally efficient medical AI, translational neuroimaging, clinical decision support systems

Abstract:

Multimodal integration is central to biomedical AI, yet current approaches often treat imaging
and clinical data as independent streams or rely on computationally expensive 3D
architectures. We present NeuroHydra, a generalizable framework that bridges 2D
self-supervised vision models (DINOv3) with 3D medical imaging and structured clinical
variables. NeuroHydra introduces a Structure-Aware Visual Slice Fusion (AS-VSF) module
that reconstructs volumetric context by learning deformable relationships across MRI slices,
maintaining anatomical continuity without requiring full 3D supervision. Clinical and tabular
features are encoded and fused with imaging representations through a Mamba state-space
integration layer, enabling sequential multimodal reasoning over spatially distributed
pathology patterns. Applied to epilepsy surgical outcome prediction, NeuroHydra
demonstrates improved performance over late-fusion and transformer-based baselines while
remaining computationally efficient. Grad-CAM and SHAP support multi-level attribution,
illustrating how imaging and clinical features jointly influence predictions. The framework is
extensible to segmentation, reconstruction, and broader translational applications. Future
work will include multi-site validation and expanded explainability analyses. NeuroHydra offers
a scalable, interpretable, and modality-aware approach to multimodal biomedical AI

Submission Number: 9