Observer Uncertainty of Learning in Games from a Covariance Perspective

Yi Feng, Georgios Piliouras, Xiao Wang

Primary Area: optimization

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: covariance, symplectic Euler method, follow-the-regularized-leader (FTRL) algorithm, uncertainty, zero-sum games

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We investigate observer uncertainty and the evolution of covariances in different learning dynamics of zero-sum games to provide insights on improving prediction accuracy.

Abstract: We investigate the accuracy of prediction in deterministic learning dynamics of zero-sum games with random initializations, specifically focusing on observer uncertainty and its relationship to the evolution of covariances. Zero-sum games are a prominent field of interest in machine learning due to their various applications, such as Generative Adversarial Networks. Concurrently, the accuracy of observation in dynamical systems from mechanics has long been a classic subject of investigation since the discovery of the Heisenberg Uncertainty Principle. This principle employs covariance and standard deviation of particle states to measure observation accuracy. In this study, we bring these two approaches together to analyze the follow-the-regularized-leader (FTRL) algorithm in two-player zero-sum games. We provide growth rates of covariance information for continuous-time FTRL, as well as its two canonical discretization methods (Euler and symplectic). Our analysis and experiments shows that employing symplectic discretization enhances the accuracy of prediction in learning dynamics.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9150