Keywords: offline RL, online RL, exploration, non-reactive, fine-tuning
TL;DR: Policy finetuning using a non-reactive exploration policy with theoretical guarantees
Abstract: In some applications of reinforcement learning,
a dataset of pre-collected experience is already available
but it is also possible to acquire some additional online data to help improve the quality of the policy.
However, it may be preferable to gather additional data with a single, non-reactive exploration policy
and avoid the engineering costs associated with switching policies.
In this paper we propose an algorithm with provable guarantees
that can leverage an offline dataset to design a single non-reactive policy for exploration.
We theoretically analyze the algorithm and measure the quality of the final policy
as a function of the local coverage of the original dataset and the amount of additional data collected.
Supplementary Material: pdf
Submission Number: 2906
Loading