Reward Bound for Behavioral Guarantee of Model-based Planning Agents

Zhiyu An, Xianzhong Ding, Wan Du

Keywords: model-based reinforcement learning, safety, guarantee

TL;DR: We derived the conditions and the lower bounds for the reward function values to achieve guarantee for model-based reinforcement learning agents to reach a goal, or to enforce specific preference between multiple goals.

Abstract: Recent years have seen an emerging interest in the Verification and Validation (V\&V) of machine learning-based agents in the wild, especially in robotics, to provide safety assurance for the industry. Obtaining behavioral guarantees for these agents remains an important problem. In this work, we focus on guaranteeing a model-based planning agent reaches a goal state within a specific future time step. We show that there exists a lower bound for the reward at the goal state, such that if the said reward is below that bound, it is impossible to obtain such a guarantee. By extension, we show how to enforce preferences over multiple goals.

Submission Number: 54