Multimodal diffusion geometry by joint diagonalization of Laplacians

Davide Eynard, Klaus Glashoff, Michael M. Bronstein, Alexander Bronstein

Abstract:

We construct an extension of diffusion geometry to multiple modalities through
joint approximate diagonalization of Laplacian matrices. This naturally extends
classical data analysis tools based on spectral geometry, such as diffusion maps
and spectral clustering. We provide several synthetic and real examples of manifold learning, retrieval, and clustering demonstrating that the joint diffusion geometry frequently better captures the inherent structure of multi-modal data. We also
show that many previous attempts to construct multimodal spectral clustering can
be seen as particular cases of joint approximate diagonalization of the Laplacians.