Fast and Scalable Method for Efficient Multimodal Feature Extraction with Optimized Maximal Correlation

Hongkang Zhang, Shao-Lun Huang, Ercan Engin KURUOGLU

Keywords: HGR maximal correlation, Soft-HGR, multimodal feature, UniFast HGR, deep learning

TL;DR: UniFast HGR optimizes HGR correlation for large neural networks, reducing complexity and enhancing accuracy.

Abstract: This paper introduces the UniFast HGR framework, a novel method designed to enhance the computation of Hirschfeld-Gebelein-Rényi (HGR) maximal correlation, specifically optimized for large-scale neural networks and multimodal tasks. UniFast HGR introduces a variance constraint and optimizes the trace term, resulting in a more accurate approximation of the original HGR. By replacing traditional covariance-based measures with cosine similarity and eliminating bias from the main diagonal, the approach significantly reduces computational complexity while enhancing overall accuracy. These improvements make UniFast HGR highly scalable and capable of delivering superior performance in diverse, large-scale multimodal learning applications. Building on this foundation, the OptFast HGR method further optimizes performance by reducing the number of normalization steps, achieving efficiency and computational cost comparable to dot product and cosine similarity operations. This advancement accelerates computation without sacrificing performance. Experimental results indicate that UniFast HGR effectively balances efficiency and precision, establishing it as a robust solution for modern deep learning challenges.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10643