Regret Bounds for Learning State Representations in Reinforcement LearningDownload PDF

Ronald Ortner, Matteo Pirotta, Alessandro Lazaric, Ronan Fruit, Odalric Maillard

06 Sept 2019 (modified: 05 May 2023)NeurIPS 2019Readers: Everyone
Abstract: We consider the problem of online learning in reinforcement learning when several state representations (mapping histories to a discrete state space) are available to the learning agent. At least one of these representations is assumed to induce a Markov decision process (MDP), and the performance of the agent is measured in terms of cumulative regret against the optimal policy giving the highest average reward in this MDP representation. We propose an algorithm (UCB-MS) with O(sqrt(T)) regret in any communicating Markov decision process. The regret bound shows that UCB-MS automatically adapts to the Markov model. This improves over the currently known best results in the literature that gave regret bounds of order O(T^(2/3)).
CMT Num: 6926
0 Replies

Loading