Strategic Recommendations for Improved Outcomes in Congestion Games
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Reinforcement Learning, Congestion Games, Q-learning, Correlated Equilibria
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: Drivers get route recommendations that could reduce congestion, but will they follow?
Abstract: Traffic on roads, packets on the Internet, and electricity on power grids share a structure abstracted in congestion games, where self-interested behaviour can lead to socially sub-optimal results. External recommendations may seek to alleviate these issues, but recommenders must take into account the effect that their recommendations have on the system. In this paper, we investigate the effects that dynamic recommendations have on $Q$-learners as they repeatedly play congestion games. To do so, we propose a novel model of recommendation whereby a $Q$-learner receives a recommendation as a state. Thus, the recommender strategically picks states during learning, which we call the Learning Dynamic Manipulation Problem. We define the \textit{manipulative potential} of these recommenders in repeated congestion games and propose an algorithm for the Learning Dynamic Manipulation Problem designed to drive the actions of $Q$-learners toward a target action distribution. We simulate our algorithm and show that it can drive the system to convergence at the social optimum of a well-known congestion game. Our results show theoretically and empirically that increasing the recommendation space can increase the manipulative potential of the recommender.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8311
Loading