Steering No-Regret Learners to Optimal Equilibria

Brian Hu Zhang, Gabriele Farina, Ioannis Anagnostides, Federico Cacciamani, Stephen Marcus McAleer, Andreas Alexander Haupt, Andrea Celli, Nicola Gatti, Vincent Conitzer, Tuomas Sandholm

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: no-regret learning, extensive-form games, optimal equilibria, mechanism design, information design, payments

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: We consider the problem of steering no-regret-learning agents to play desirable equilibria via nonnegative payments. We show that steering is impossible if the total budget (across iterations) is finite, both in normal- and extensive-form games. However, vanishing average payments are compatible with steering. When players' full strategies are observed at each timestep, constant per-iteration payments permit steering. When only trajectories through the game tree are observable, steering is impossible with constant per-iteration payments in general extensive-form games, but possible in normal-form games or if the maximum per-iteration payment may grow with time, maintaining vanishing average payments. We supplement our theoretical positive results with experiments highlighting the efficacy of steering in large games, and show how our framework relates to optimal mechanism design and information design.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7900