Fair Model-Based Reinforcement Learning Comparisons with Explicit and Consistent Update Frequency

Albert Thomas, Abdelhakim Benechehab, Giuseppe Paolo, Balázs Kégl

Keywords: model-based reinforcement learning, update frequency, MBPO, model-based reinforcement learning benchmark

Blogpost Url: https://iclr-blogposts.github.io/2024/blog/update-frequency-in-mbrl/

Abstract: Implicit update frequencies can introduce ambiguity in the interpretation of model-based reinforcement learning benchmarks, obscuring the real objective of the evaluation. While the update frequency can sometimes be optimized to improve performance, real-world applications often impose constraints, allowing updates only between deployments on the actual system. This blog post emphasizes the need for evaluations using consistent update frequencies across different algorithms to provide researchers and practitioners with clearer comparisons under realistic constraints.

Ref Papers: https://arxiv.org/abs/2006.03647, https://arxiv.org/abs/1906.08253, https://arxiv.org/abs/1805.12114

Id Of The Authors Of The Papers: ~Tatsuya_Matsushima1, ~Michael_Janner1, ~Kurtland_Chua1

Conflict Of Interest: I have no conflict of interest with the papers this blogpost is about.

Submission Number: 28