Adversarial Representation Learning for Canonical Correlation Analysis

Feng Bao, Steve J. Altschuler, Lani F. Wu

Keywords: Representation Learning, Canonical Correlation Analysis, Adversarial Learining

Abstract: Canonical correlation analysis (CCA) provides a framework to map multimodality data into a maximally correlated latent space. The deep version of CCA has replaced linear maps with deep transformations to enable more flexible correlated data representations; however, this approach requires optimization over all samples for each iteration and poorly scales. Here, we present a deep, adversarial approach to CCA, adCCA, that can be efficiently solved by standard mini-batch training. We reformulate CCA under the constraint that the different modalities are embedded with identical latent distributions, derive a tractable deep CCA target, and use an adversarial framework to efficiently learn the canonical representations. A consequence of the new formation is that adCCA learns maximally correlated representations across multimodalities meanwhile preserves structure within individual modalities. Further, adCCA removes the need for feature transformation and normalization and can be directly applied to diverse modalities and feature encodings. Numerical studies show that the performance of adCCA is robust to data transformations, binary encodings, and corruptions. Together, adCCA provides a scalable approach to align data across modalities without compromising structure within each modality.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning

TL;DR: A reformulation of CCA under the adversarial framework for efficient canonical representation learning.