Barycentric Alignment of Mutually Disentangled Modalities

Nhu-Thuat Tran, Hady W. Lauw

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: mutually disentangled modalities, interest factor alignment

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Discovering explanatory factors of user preferences behind behavioral data has gained increasing attention. As collected behavioral data is often highly sparse, mining other data modalities, e.g., texts, for interest factors and then correlating them with those from behavioral data could provide a pathway to improve recommendation. Nonetheless, two challenges prevail. For one, the unordered set nature of discovered factors and the unavailability of prior alignment information causes a challenge to align revealed interest factors from two modalities. For another, it demands a tailored method to effectively transfer knowledge between interest factors from mutually related modalities. To resolve this, we regard discovered interest factors from ratings and texts as supporting points of two discrete measures. Then, their alignment is formulated as an optimal transport problem, finding an optimal mapping between two probability masses. Next, the mapping probability serves not only as the prior information but also as input of barycentric strategy to match and fuse interest factors, effectively tranferring user preferences between mutually disentangled modalities.
Experiments on real-world datasets verify the advantage of the proposed method over a series of baselines.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4971