Deep Variational Canonical Correlation Analysis

Weiran Wang, Xinchen Yan, Honglak Lee, Karen Livescu

Abstract: We present deep variational canonical correlation analysis (VCCA), a deep multi-view learning model that extends the latent variable model interpretation of linear CCA~\citep{BachJordan05a} to nonlinear observation models parameterized by deep neural networks (DNNs). Computing the marginal data likelihood, as well as inference of the latent variables, are intractable under this model. We derive a variational lower bound of the data likelihood by parameterizing the posterior density of the latent variables with another DNN, and approximate the lower bound via Monte Carlo sampling. Interestingly, the resulting model resembles that of multi-view autoencoders~\citep{Ngiam_11b}, with the key distinction of an additional sampling procedure at the bottleneck layer. We also propose a variant of VCCA called VCCA-private which can, in addition to the ``common variables'' underlying both views, extract the ``private variables'' within each view. We demonstrate that VCCA-private is able to disentangle the shared and private information for multi-view data without hard supervision.

TL;DR: A deep generative model for multi-view representation learning

Conflicts: ucmerced.edu, ttic.edu, umich.edu