Task-Oriented Multi-View Representation Learning

Ren Wang, Haoliang Sun, Yuxiu Lin, Yongshun Gong, Xiushan Nie, Yilong Yin

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Multi-view learning; Meta learning; Feature modulation; Task adaptation

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Multi-view representation learning aims to learn a high-quality unified representation for an entity from its multiple observable views to facilitate the performance of downstream tasks. A typical multi-view representation learning framework consists of four main components: View-specific encoding, Single-view learning (SVL), Multi-view learning (MVL), and Fusion. Recent studies achieve promising performance by carefully designing SVL and MVL constraints, but almost all of them ignore the basic fact that \textit{effective representations are different for different tasks, even for the same entity}. To bridge this gap, this work proposes a \textbf{T}ask-\textbf{O}riented \textbf{M}ulti-\textbf{V}iew \textbf{R}epresentation \textbf{L}earning (TOMRL) method, where the key idea is to modulate features in the View-specific encoding and Fusion modules according to the task guidance. To this end, we first design a gradient-based embedding strategy to flexibly represent multi-view tasks. After that, a meta-learner is trained to map the task embedding into a set of view-specific parameters and a view-shared parameter for modulation in the Encoding and Fusion modules, respectively. This whole process is formalized as a nested optimization problem and ultimately solved by a bi-level optimization scheme. Extensive experiments on four multi-view datasets validate that our TOMRL consistently improves the performance of most existing multi-view representation learning approaches.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9283