Many Is Better Than One: Multiple Covariation Learning for Latent Multiview Representation

Yun-Hao Yuan, Pengwei Qian, Jin Li, Jipeng Qiang, Yi Zhu, Yun Li

Abstract: Canonical correlation analysis is a typical multiview representation learning technique, which utilizes within-set and between-set covariance matrices to analyze the correlation between two multidimensional datasets. However, it is quite difficult for the covariance matrix to measure the nonlinear relationship between features because of its linear structure. In this paper, we propose a multiple covariation projection (MCP) method to learn latent two-view representation, which has the ability to model the complicated feature relationship. The proposed MCP first constructs multiple general covariance matrices for modeling diverse feature relations, and then integrates them together via a linear ensemble strategy. At last, an efficient two-stage algorithm is designed for solutions. In addition, we further present a multiview MCP for dealing with the case of multiple (more than two) views. Experimental results on benchmark datasets show the effectiveness of our proposed MCP method in multiview classification and clustering tasks.