EigenGame: PCA as a Nash Equilibrium

Ian Gemp, Brian McWilliams, Claire Vernade, Thore Graepel

Keywords: pca, principal components analysis, nash, games, eigendecomposition, svd, singular value decomposition

Abstract: We present a novel view on principal components analysis as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm---which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization---is naturally decentralized and hence parallelizable through message passing. We demonstrate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.

One-sentence Summary: We formulate the solution to PCA as the Nash of a suitable game with accompanying algorithm that we demonstrate on a 200TB dataset.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Code: [ 2 community implementations](https://paperswithcode.com/paper/?openreview=NzTU59SYbNq)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2010.00554/code)