Block Coordinate Descent Methods for Optimization under J-Orthogonality Constraints with Applications

Di He, Ganzhao Yuan, Xiao Wang, Pengxiang Xu

Keywords: Orthogonality Constraints, Nonconvex Optimization, Nonsmooth Composite Optimization, Block Coordinate Descent, Convergence Analysis

Abstract: The J-orthogonal matrix, also referred to as the hyperbolic orthogonal matrix, is a class of special orthogonal matrix in hyperbolic space, notable for its advantageous properties. These matrices are integral to optimization under J-orthogonal constraints, which have widespread applications in statistical learning and data science. However, addressing these problems is generally challenging due to their non-convex nature and the computational intensity of the constraints. Currently, algorithms for tackling these challenges are limited. This paper introduces \textbf{JOBCD}, a novel Block Coordinate Descent method designed to address optimizations with J-orthogonality constraints. We explore two specific variants of \textbf{JOBCD}: one based on a Gauss-Seidel strategy (\textbf{GS-JOBCD}), the other on a variance-reduced and Jacobi strategy (\textbf{VR-J-JOBCD}). Notably, leveraging the parallel framework of a Jacobi strategy, \textbf{VR-J-JOBCD} integrates variance reduction techniques to decrease oracle complexity in the minimization of finite-sum functions. For both \textbf{GS-JOBCD} and \textbf{VR-J-JOBCD}, we establish the oracle complexity under mild conditions and strong limit-point convergence results under the Kurdyka-Lojasiewicz inequality. To demonstrate the effectiveness of our method, we conduct experiments on hyperbolic eigenvalue problems, hyperbolic structural probe problems, and the ultrahyperbolic knowledge graph embedding problem. Extensive experiments using both real-world and synthetic data demonstrate that \textbf{JOBCD} consistently outperforms state-of-the-art solutions, by large margins.

Primary Area: optimization

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6221