Efficient Distributed Principal Component Analysis with Parallel Deflation
Keywords: Principal Component Analysis, Distributed Learning
Abstract: We study a distributed Principal Component Analysis (PCA) framework where each worker targets a distinct eigenvector and refines its solution by updating from intermediate solutions provided by peers deemed as "superior". Drawing intuition from the delation methods, which is traditionally used in centralized eigenvalue problems, our method breaks the sequential dependency in between the deflation steps and allows asynchronous updates of workers while incurring only a small communication cost. To our knowledge, a critical gap in the literature --*the theoretical underpinning of such distributed, dynamic interactions among workers*-- has remained unaddressed until now. This paper offers the first theoretical analysis explaining why, how, and when these intermediate, hierarchical updates lead to practical and provable convergence in distributed environments. Our theoretical contributions demonstrate that such a distributed PCA algorithm not only converges effectively but does so in a manner that is favorably scalable. We also demonstrate through experiments that our proposed framework offers comparable performance to EigenGame-$\mu$, the state-of-the-art model-parallel PCA solver.
Supplementary Material: zip
Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8158
Loading