Reinforcement Learning with Lookahead Information
Keywords: Reinforcement Learning, Regret Minimization, Lookahead
TL;DR: We study RL settings where either immediate rewards or transitions are observed before acting and show how to achieve tight regret compared to stronger baselines with similar information.
Abstract: We study reinforcement learning (RL) problems in which agents observe the reward or transition realizations at their current state _before deciding which action to take_. Such observations are available in many applications, including transactions, navigation and more. When the environment is known, previous work shows that this lookahead information can drastically increase the collected reward. However, outside of specific applications, existing approaches for interacting with unknown environments are not well-adapted to these observations. In this work, we close this gap and design provably-efficient learning algorithms able to incorporate lookahead information. To achieve this, we perform planning using the empirical distribution of the reward and transition observations, in contrast to vanilla approaches that only rely on estimated expectations. We prove that our algorithms achieve tight regret versus a baseline that also has access to lookahead information -- linearly increasing the amount of collected reward compared to agents that cannot handle lookahead information.
Primary Area: Reinforcement learning
Submission Number: 2975
Loading